Once you’ve implemented your tracers and confirmed that they are communicating with your Lightstep Satellite, it’s time to add spans to your operations and connect the spans together to create a full trace that you can observe in Lightstep.
Use Context to Join Spans Together
Traces are all about following transactions through an application, not simply monitoring individual processes or operations. To deliver the goal of an end-to-end trace of your highest-priority operations, you will need to join those operations’ spans together by propagating trace span context along with the transaction. You will want to connect the dots both within processes and between processes.
Spans have a parent/child relationship with other spans in the trace, creating the flow from the root span at the beginning of the request, to the lowest level span at the end of the request, before the flow returns back to the root. Each span (except the root) carries a reference to its parent span. This reference (along with others) is carried in the context of the process.
Many languages and frameworks provide some form of
context object that can be used to propagate the hierarchical relationship and other identifiers (for example, a request, transaction, or operation). For example, Go has
context.Context and Django has template contexts. Where possible, use the approach that is idiomatic for the platform to propagate context throughout a process.
Here’s an example of using
context.Context in Go to propagate transaction context from parent to child spans.
1 2 3 // Start a new span as a child of the span in the current context // Returns the span and a new context.Context containing the current span span, ctx := opentracing.StartSpanFromContext(ctx, "my_operation_name")
Extract feature serializes and deserializes the span’s context. This context can be sent over the wire, for example as HTTP headers. Use these methods to propagate your Span Context when crossing process boundaries.
Use Trace Assembly Tags
Sometimes you may not be able to use
Extract.When that’s the case, Lightstep offers an additional correlation mechanism called Trace Assembly Tags.
If you already propagate a de facto transaction ID (perhaps referred to as a
trace_id, etc) around your distributed system, Lightstep can use it to assemble distributed traces. In order to take advantage of this feature, add an OpenTracing tag to your spans with a key string prefixed by
"guid:" and the given transaction/correlation/etc ID as the value. For instance, given a
requestId member variable, you might write:
1 2 3 4 5 6 7 // Set a guid tag on an opentracing.Span object using the SetTag method and by // giving the key a name that starts with "guid:". // // Lightstep will automatically include any other Span (throughout the // distributed system) that uses this same Tag key and value as part of the // same distributed trace. span.SetTag("guid:request_id", requestId)
"guid:" tags allow you to reuse existing propagation mechanisms to assemble traces in Lightstep, and in so doing can greatly reduce the initial integration time for codebases and systems that have not yet instrumented with OpenTracing.
"guid:" trace assembly tags are useful in production, conventional OpenTracing
Extract propagation actually encodes more information about span relationships and provides Lightstep with more information to use when visualizing or analyzing traces. Trace assembly tags are far better than nothing, but conventional OpenTracing propagation is still a best practice when it’s a convenient option.
Spans may have multiple
"guid:" tags, and traces are assembled by taking the union of these assembly hints and any conventional OpenTracing span-to-span references.
Check out our Quick Start guides for your language to get your tracers configured.
Follow these language-specific guides to get started instrumenting at the service level:
- C++: Add Spans
- C#: Add Spans
- Go: Add Spans
- Java: Add Spans
- Objective-C: Add Spans
- PHP: Add Spans
- Python: Add Spans
- Ruby: Add Spans
If you want to add instrumentation to your framework as a head start, check out Instrument Your Framework.
Once you’ve completed your instrumentation, be sure to check out your Instrumentation Quality score.
Don’t send any sensitive data to your tracers! Lightstep is GDPR-compliant. Our terms-of-service state you cannot send personally identifiable information (PII) to Lightstep. If PII is sent, Contact Lightstep immediately to get help in deleting the non-compliant data.