Send traces to Cloud Observability with Java

This Quick Start guide shows you how to configure your Java applications to send OpenTelemetry traces to Cloud Observability.

This guide does not provide documentation on application instrumentation. For Java-specific information on instrumentation, please see the Java OpenTelemetry getting started guide.

The sections below contain code snippets only. For full code listings, please see java/otlp and java/launcher in the Cloud Observability OpenTelemetry examples repository.

Sending OpenTelemetry data directly to Cloud Observability without a Collector for most developer setups will suffice. For non-development setups, however, it is highly recommended that you send OpenTelemetry data to Cloud Observability by way of the OpenTelemetry Collector. This can be done with or without a Launcher, as we’ll see below.

Pre-Requisites

Before you get started with sending OpenTelemetry data to Cloud Observability, you will need the following:

Install OpenTelemetry Packages

In your application code, you will need to install dependencies and import OpenTelemetry packages before you can send data to Cloud Observability.

Start by installing the Java auto-instrumentation agent. Do this by opening up a terminal window and pasting the following:

1

curl -L -O https://github.com/open-telemetry/opentelemetry-java-instrumentation/releases/latest/download/opentelemetry-javaagent.jar

You will also need to add the following OpenTelemetry dependencies to your pom.xml:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16

...
<dependencies>
    ...
    <dependency>
        <groupId>io.opentelemetry</groupId>
        <artifactId>opentelemetry-api</artifactId>
        <version>1.18.0</version>
    </dependency>
    <dependency>
        <groupId>io.opentelemetry.instrumentation</groupId>
        <artifactId>opentelemetry-instrumentation-annotations</artifactId>
        <version>1.18.0-alpha</version>
    </dependency>
    ...
</dependencies>
...

You can find the latest version of the OpenTelemetry Java here.

If you are using Gradle, your dependencies in build.gradle would look like this:

1
2
3
4
5
6
7
8

...
dependencies {
    ...
    implementation("io.opentelemetry:opentelemetry-api:1.18.0")
    implementation("io.opentelemetry:opentelemetry-instrumentation-annotations:1.18.0")
    ...
}
...

Code Setup

Although auto-instrumentation is a great way to get started quickly with instrumenting your Java application, it is often not sufficient. In order to gain greater insights into your application code, you should also add manual instrumentation to your business logic. More on auto-instrumentation vs manual instrumentation here.

In order to manually instrument your Java code, you need to acquire a Tracer. The Tracer is responsible for creating Spans and interacting with the Context.

Import OpenTelemetry Packages

Before you can acquire a Tracer, you must first import the appropriate package. Open up your application code, and add the following import to your .java file:

1

import io.opentelemetry.api;

Acquire a Tracer

Now you are ready to acquire your tracer. This should be declared as a private, class-level variable so that it can be easily accessed by other methods in your class.

1
2
3
4
5
6

...
public class MyClass {
...
    private static final Tracer tracer = GlobalOpenTelemetry.getTracer("CloudObsExample");
...
}

Where CloudObsExample can be replaced by anything. More info here.

Start Auto-Instrumentation

The Java auto-instrumentation agent is used to configure OpenTelemetry and auto-instrument your code. It dynamically injects bytecode to capture telemetry from many popular libraries and frameworks. A full list of auto-instrumented Java libraries can be found here.

Start tabs

Direct (gRPC)

1
2
3
4
5
6
7
8
9
10

export OTEL_EXPORTER_OTLP_TRACES_HEADERS="lightstep-access-token=<LS_ACCESS_TOKEN>"

java -javaagent:opentelemetry-javaagent.jar \
           -Dotel.service.name=<service_name> \
           -Dotel.traces.exporter=logging,otlp \
           -Dotel.metrics.exporter=logging,otlp \
           -Dotel.exporter.otlp.protocol=grpc \
           -Dotel.exporter.otlp.endpoint="https://ingest.lightstep.com:443" ## US data center \
           ## -Dotel.exporter.otlp.endpoint="https://ingest.eu.lightstep.com:443" ## EU data center \
           -jar path/to/your/app.jar

Collector (gRPC)

1
2
3
4
5
6
7

java -javaagent:opentelemetry-javaagent.jar \
    -Dotel.service.name=<service_name> \
    -Dotel.traces.exporter=logging,otlp \
    -Dotel.metrics.exporter=logging,otlp \
    -Dotel.exporter.otlp.protocol=grpc \
    -Dotel.exporter.otlp.endpoint="http://0.0.0.0:4317" \
    -jar path/to/your/app.jar

End tabs

To view traces in your Cloud Observability project, click explorer in the left navigation bar, and then click on any span in the Trace Analysis table.

Notes: Direct (OTLP from application code)

  • Replace <LS_ACCESS_TOKEN> with your own Cloud Observability access token.
  • otel.service.name sets the name of the service. This is the value that will show up in the Cloud Observability service explorer. Be sure to replace <service_name> with your own service name.
  • otel.traces.exporter and otel.metrics.exporter specify where to send traces and metrics, respectively. In this case, they are being sent to the console (stdout) and to otlp. The otlp option tells the Java agent to send it to an endpoint that accepts OTLP. More information can be found here.
  • otel.exporter.otlp.protocol specifies whether to send OTLP requests via gRPC or HTTP.

  • otel.exporter.otlp.endpoint specifies where to send the traces and metrics to OTLP endpoint https://ingest.lightstep.com:443 (i.e. Cloud Observability).

    Organizations using the EU data center should use ingest.eu.lightstep.com:443.

Include the https:// prefix for the gRPC endpoint, otherwise, you will get an error.

To use HTTP instead of gRPC, update the call to the Java agent as follows:

1
2
3
4
5
6
7
8
9
10
11

java -javaagent:opentelemetry-javaagent.jar \
    -Dotel.service.name=<service_name> \
    -Dotel.traces.exporter=logging,otlp \
    -Dotel.metrics.exporter=logging,otlp \
    -Dotel.exporter.otlp.traces.protocol=http/protobuf \
    -Dotel.exporter.otlp.metrcs.protocol=grpc \
    -Dotel.exporter.otlp.traces.endpoint="https://ingest.lightstep.com/traces/otlp/v0.9" \ #US data center 
    -Dotel.exporter.otlp.metrics.endpoint="https://ingest.lightstep.com:443" \ #US data center 
    #-Dotel.exporter.otlp.traces.endpoint="https://ingest.eu.lightstep.com/traces/otlp/v0.9" \ ## EU data center 
    #-Dotel.exporter.otlp.metrics.endpoint="https://ingest.eu.lightstep.com:443" \ ## EU data center 
    -jar path/to/your/app.jar

Noteworthy items:

  • The HTTP traces endpoint is https://ingest.lightstep.com/traces/otlp/v0.9 instead of https://ingest.lightstep.com:443 (https://ingest.eu.lightstep.com/traces/otlp/v0.9 for the EU data center).
  • The otel.traces.exporter uses http/protobuf instead of grpc.
  • Cloud Observability currently does not support an HTTP metrics endpoint, so we cannot use otel.exporter.otlp.endpoint for both. Instead, in this example, we must define separate endpoints for traces (via HTTP) and metrics (via gRPC).

Expandable end

Notes: OpenTelemetry Collector

  • Do not set the OTEL_EXPORTER_OTLP_TRACES_HEADERS environment variable, because the LS_ACCESS_TOKEN is already configured in the Collector’s config.yml file.
  • If the otel.exporter.otlp.endpoint option is left out, the agent assumes that you have a Collector running locally at http://0.0.0.0:4317. If you wish to use a different Collector endpoint, simply include the otel.exporter.otlp.endpoint option with your Collector’s gRPC endtpoint.
  • otel.service.name sets the name of the service. This is the value that will show up in the Cloud Observability service explorer. Be sure to replace <service_name> with your own service name.

To use HTTP instead of gRPC, update the call to the Java agent as follows:

1
2
3
4
5
6

java -javaagent:opentelemetry-javaagent.jar \
    -Dotel.service.name=java-collector-server-http \
    -Dotel.traces.exporter=otlp \
    -Dotel.metrics.exporter=otlp \
    -Dotel.exporter.otlp.protocol=http/protobuf \
    -jar target/lightstep-otlp-server.jar com.lightstep.otlp.server.ExampleServer

Which is the equivalent of:

1
2
3
4
5
6
7
8
9

java -javaagent:opentelemetry-javaagent.jar \
    -Dotel.service.name=java-collector-server-http \
    -Dotel.traces.exporter=otlp \
    -Dotel.metrics.exporter=otlp \
    -Dotel.exporter.otlp.traces.protocol=http/protobuf \
    -Dotel.exporter.otlp.metrics.protocol=http/protobuf \
    -Dotel.exporter.otlp.traces.endpoint="http://0.0.0.0:4318/v1/traces" \
    -Dotel.exporter.otlp.metrics.endpoint="http://0.0.0.0:4318/v1/metrics" \
    -jar target/lightstep-otlp-server.jar com.lightstep.otlp.server.ExampleServer

Noteworthy items:

  • The otel.exporter.otlp.protocol uses http/protobuf instead of grpc.
  • If otel.exporter.otlp.endpoint is omitted when using HTTP, it is the equivalent of saying -Dotel.exporter.otlp.traces.endpoint="http://0.0.0.0:4318/v1/traces" and -Dotel.exporter.otlp.metrics.endpoint="http://0.0.0.0:4318/v1/metrics"
  • otel.exporter.otlp.protocol=http/protobuf is the equivalent of saying otel.exporter.otlp.traces.protocol=http/protobuf and otel.exporter.otlp.metrics.protocol=http/protobuf
  • If you wish to use a different Collector HTTP endpoint, you will need to specify separate endpoints for traces (otel.exporter.otlp.traces.endpoint) and metrics (otel.exporter.otlp.metrics.endpoint). You will also need to include the /v1/traces suffix for traces, and the /v1/metrics for metrics as part of the endpoint.

Noteworthy items:

  • Do not set LS_ACCESS_TOKEN, since that’s already configured in the Collector’s config.yml file.
  • If you attempt to override otel.exporter.otlp.endpoint to send traces to a Collector, the traces will be sent directly to ingest.lightstep.com:443 (or ingest.eu.lightstep.com:443 for the EU) instead of via the Collector. Instead, you need to override otel.exporter.otlp.traces.endpoint for traces, and otel.exporter.otlp.metrics.endpoint for metrics.
  • If you wish to use a different Collector gRPC endpoint, simply configure otel.exporter.otlp.traces.endpoint and otel.exporter.otlp.metrics.endpoint, using your own Collector’s endpoint.
  • If you wish to use a different Collector HTTP endpoint, you will need to include the /v1/traces suffix for traces, and the /v1/metrics for metrics as part of the endpoint.

Expandable end

Troubleshooting

Debugging

To see Java agent debug logs, you can pass in -Dotel.javaagent.debug=true:

1
2
3
4

java -javaagent:opentelemetry-javaagent.jar \
...
    -Dotel.javaagent.debug=true
...

Note that these are quite verbose.

Expandable end

Endpoints Connectivity Issues

If you are having trouble sending traces to your endpoint (whether it is to Cloud Observability or to your Collector, please ensure that you include the http:// or https:// prefix for your endpoints, otherwise the Java agent will err out.

If you choose to use the OpenTelemetry Collector over HTTP, your endpoint will need to include the /v1/{signal} suffix. You will also need to ensure that you set otel.exporter.otlp.traces.protocol to http/protobuf. For example:

1
2
3
4
5
6
7
8
9
10

java -javaagent:opentelemetry-javaagent.jar \
...
    # Traces
    -Dotel.exporter.otlp.traces.protocol=http/protobuf \
    -Dotel.exporter.otlp.traces.endpoint="http://0.0.0.0:4318/v1/traces" \

    # Metrics
    -Dotel.exporter.otlp.metrics.protocol=http/protobuf \
    -Dotel.exporter.otlp.metrics.endpoint="http://0.0.0.0:4318/v1/metrics" \
...

For more information, and configuration options please see Java agent OpenTelemetry documentation.

Expandable end

Check for multiple versions of OpenTelemetry

If multiple versions of OpenTelemetry are installed, traces are not created or propagated correctly. Check your dependencies to ensure that only a single version of OpenTelemetry is installed.

Expandable end

Check that security is configured correctly

Cloud Observability’s public Microsatellites only accept spans via a secure endpoint. If you see security related errors (e.g., Netty TLS errors with Java), you may not have the necessary root certificate installed on the machine where the tracer is running. To add a root certificate, see the documentation about encrypted connections.

Expandable end

Check that Metrics are enabled in the OpenTelemetry Collector

Not seeing metrics come through the OpenTelemetry Collector? Make sure that you have defined a Metrics pipeline in your Collector’s YAML config file.

Cloud Observability access token

See also

Send traces to Cloud Observability with Python

Send traces to Cloud Observability with Go

Send traces to Cloud Observability with .NET

Updated May 19, 2023