Golang Manual Instrumentation with OpenTracing

Want to use OpenTelemetry instead? Read these docs to get started!

This Quickstart will have you configure your tracer to communicate with the Cloud Observability Microsatellites and create a single span on your service. You install both the OpenTracing API and Cloud Observability tracer and then use the OpenTracing and Cloud Observability APIs to instrument your code.

To ensure you can access all Cloud Observability functionality, including infrastructure metrics reporting, update your tracer to the latest release. To update, simply follow the instructions for installing the tracer. No code changes are needed.

While Cloud Observability offers tracers and APIs specific to its tracing software, you will still use the OpenTracing API to fully instrument your code. Be sure you have read and are familiar with both Cloud Observability tracers and the OpenTracing specification in your application’s language.

Golang Instrumentation for OpenTracing

  1. Find your access token in Cloud Observability. You’ll need this to configure your Cloud Observability tracer.
    • Click the Project Settings button.
    • In the Access Tokens table, click the Copy icon to copy your access token to the clipboard.

  2. Install the OpenTracing and Cloud Observability tracer packages.

    1
2

    go get 'github.com/lightstep/lightstep-tracer-go'
go get 'github.com/opentracing/opentracing-go'

  3. In your application code, add a reference to the Cloud Observability Tracer and to the OpenTracing API.

    1
2
3
4

    import (
  "github.com/opentracing/opentracing-go"
  "github.com/lightstep/lightstep-tracer-go"
)

  4. Early in your application’s initialization, configure the Cloud Observability tracer and register it as the OpenTracing Global Tracer. As part of the configuration, you need to add your access token and add a tag to hold the value of your service’s name.

    Use the right code for your environment!
    When initializing the tracer, you pass in properties to the Cloud Observability Microsatellites that collect the span data. Be sure to use the code for your Cloud Observability Microsatellite environment - On-Premise, Cloud Observability Public Microsatellite pool, or Developer Mode Satellite.

     

    on-premise

    1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16

     package main. . .
 func main() {
   ...
   lightStepTracer := lightstep.NewTracer(lightstep.Options{
     Collector: lightstep.Endpoint{
       Host:"{your_load_balancer_DNS_name_or_IP_address}",
       Plaintext: true,
     },
     AccessToken: "YOUR_ACCESS_TOKEN",
     Tags: map[string]interface{}{
       lightstep.ComponentNameKey: "YOUR_SERVICE_NAME",
     },
   })
   opentracing.SetGlobalTracer(lightStepTracer)
   ...
 }

    public

    1
2
3
4
5
6
7
8
9
10
11
12
13

     package main
 . . .
 func main() {
   ...
   lightStepTracer := lightstep.NewTracer(lightstep.Options{
     AccessToken: "YOUR_ACCESS_TOKEN",
     Tags: map[string]interface{}{
       lightstep.ComponentNameKey: "YOUR_SERVICE_NAME",
     },
   })
   opentracing.SetGlobalTracer(lightStepTracer)
   ...
 }

    developer

    1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19

     package main
 . . .
 func main() {
 ...
 lightStepTracer := lightstep.NewTracer(lightstep.Options{
   Collector: lightstep.Endpoint{
     Host:      "localhost",
     Port:      8360,
     Plaintext: true,
   },
   AccessToken: "developer",
   Tags: map[string]interface{}{
     lightstep.ComponentNameKey: "YOUR_SERVICE_NAME",
   },
 })
 opentracing.SetGlobalTracer(lightStepTracer)
 ...
  }
​​​​​​​

 

  1. Test that everything is connected by sending a test span.

    1
2
3
4
5
6
7
8
9
10

    tracer := opentracing.GlobalTracer()
span := tracer.StartSpan("my-first-span")
// for tag value, use either "client" or "server" depending on whether
// this service receives or creates requests
span.SetTag("kind", "client")
span.LogKV("message", "what a lovely day")
span.Finish()

// remember to close the tracer in order to ensure spans are sent
lightStepTracer.Close(context.Background()))

  2. Before shutdown, be sure to close the tracer so that any remaining spans will be sent.

    1

       lightStepTracer.Close(context.Background())

  3. Run the app.

  4. Open Cloud Observability. You should see your service in the Service directory list.
    Multiple services appear in the Service Directory's sidebar.

It may take a few moments for your service to display in the Service Directory. To see data immediately, click the Explorer tab to view data in the histogram.

Add More Spans

Now that you’ve begun instrumentation of your app, you can create more spans and connect them together.

  1. Start adding code to create a span for your service’s operation. Start with an operation at an edge (and that calls an operation or operations in response). This will be your root span. For the operation name, use a unique name that clearly represents the function. This is what will display in Cloud Observability, so understandable operation names are important.

    1
2
3
4
5
6
7
8
9
10

     	func xyz() {
     ...
     sp := opentracing.StartSpan("operation_name")
     defer sp.Finish()

     // Sleep for 1 second before returning to give the tracer
     // buffer time to flush the span to Lightstep
     time.Sleep(1 * time.Second)
     ...
 }

  2. You then connect the root span to subsequent operations’ spans to create a full trace. Don’t forget to call Finish.

    1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23

     // If you use context.Context in your application, OpenTracing's Go library will happily rely on it for Span propagation. To start a new (blocking child) Span, you can use StartSpanFromContext.

  func abc(ctx context.Context, ...) {
     ...
     span, ctx := opentracing.StartSpanFromContext(ctx, "operation_name")
     defer span.Finish()
     span.LogFields(
         log.String("event", "soft error"),
         log.String("type", "cache timeout"),
         log.Int("waited.millis", 1500))
     ...
 }

 // OR create a child Span given an existing parent Span.

 func abc(parentSpan opentracing.Span, ...) {
     ...
     sp := opentracing.StartSpan(
         "operation_name",
         opentracing.ChildOf(parentSpan.Context()))
     defer sp.Finish()
     ...
 }

  3. If the trace needs to continue over the wire, you use GlobalTracer to inject the TraceContext into HTTP headers.

    1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17

     func makeSomeRequest(ctx context.Context) ... {
     if span := opentracing.SpanFromContext(ctx); span != nil {
         httpClient := &http.Client{}
         httpReq, _ := http.NewRequest("GET", "http://myservice/", nil)

         // Transmit the span's TraceContext as HTTP headers on our
         // outbound request.
         opentracing.GlobalTracer().Inject(
             span.Context(),
             opentracing.HTTPHeaders,
             opentracing.HTTPHeadersCarrier(httpReq.Header))

         resp, err := httpClient.Do(httpReq)
         ...
     }
     ...
 }

  4. And then deserialize using extract like this.

    1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21

     http.HandleFunc("/", func(w http.ResponseWriter, req *http.Request) {
     var serverSpan opentracing.Span
     appSpecificOperationName := ...
     wireContext, err := opentracing.GlobalTracer().Extract(
         opentracing.HTTPHeaders,
         opentracing.HTTPHeadersCarrier(req.Header))
     if err != nil {
         // Optionally record something about err here
     }

     // Create the span referring to the RPC client if available.
     // If wireContext == nil, a root span will be created.
     serverSpan = opentracing.StartSpan(
         appSpecificOperationName,
         ext.RPCServerOption(wireContext))

     defer serverSpan.Finish()

     ctx := opentracing.ContextWithSpan(context.Background(), serverSpan)
     ...
 }

You’ll find details about working with the OpenTracing API here:

See also

Updated Mar 2, 2020