o11yfest

Supercharge Legacy Application Logs by Injecting OpenTelemetry Span Context

A good cook uses ingredients they have in the kitchen, and observability is no different. A good practitioner should leverage legacy telemetry, even when adopting OpenTelemetry. Learn how to leverage and correlate existing application logs with OTEL Traces by injecting Span Context into your logs.

Unlike neatly packaged demos and “Getting Started” guides, the average user of OpenTelemetry isn’t coming to the project with a complete greenfield. Instead, many of these projects are so-called “brownfields”, with existing metrics, tracing, and logs from a variety of different standards already being emitted by their Applications and used to monitor, debug, and introspect applications. OpenTelemetry, with Tracing Specification(and many SDKs) at 1.0, has done an admirable job ensuring compatibility with older tracing standards, and OpenTelemetry Metrics seem well on their way to ensuring the same broad level of support and inclusivity for existing standards. But existing Logs have been left out in the cold, to some extent, with no standard out of the box way to connect them to your traces across languages.

But, fear not! With a bit of application code, the OpenTelemetry Language SDKs, and some willingness to monkeypatch a library or two, the ability to connect your existing Application Logs with the context of the Span and Trace Telemetry data is easier than ever. In this talk, I’ll take a survey of a few languages (Ruby, Python, and JS), and demonstrate from start to finish how to modify popular Logging Libraries to ensure that every Log line also contains the right span and trace identifiers to correlate them in your vendor or open source backend of choice. I’ll also walk through a scenario or two where connecting Logs and Traces helps improve an end users ability to debug, troubleshoot, and gain insight into their application behavior.