This Quick Start guide shows you how to configure your Python applications to send OpenTelemetry traces to Cloud Observability.
This guide does not provide documentation on application instrumentation. For Python-specific information on instrumentation, please see the Python OpenTelemetry getting started guide.
The sections below contain code snippets only. For full code listings, please see python/opentelemetry
and python/launcher
in the Cloud Observability OpenTelemetry examples repository.
Sending OpenTelemetry data directly to Cloud Observability without a Collector for most developer setups will suffice. For non-development setups, however, it is highly recommended that you send OpenTelemetry data to Cloud Observability by way of the OpenTelemetry Collector. This can be done with or without a Launcher, as we’ll see below.
Before you get started with sending OpenTelemetry data to Cloud Observability, you will need the following:
In your application code, you will need to install dependencies and import OpenTelemetry packages before you can send data to Cloud Observability.
Start by installing the Python OpenTelemetry packages. Do this by opening up a terminal window and pasting the following snippet:
Start tabs
Direct (gRPC)
1
2
3
4
pip install opentelemetry-distro \
opentelemetry-exporter-otlp
opentelemetry-bootstrap -a install
Collector
1
2
3
4
pip install opentelemetry-distro \
opentelemetry-exporter-otlp
opentelemetry-bootstrap -a install
Launcher
1
2
3
4
pip install opentelemetry-launcher \
protobuf==3.20.1
opentelemetry-bootstrap -a install
End tabs
The opentelemetry-bootstrap -a install
command will read through the packages installed in your active site-packages
folder, and will install the applicable auto-instrumentation libraries. For example, if you already installed the flask
package, running opentelemetry-bootstrap -a install
will install opentelemetry-instrumentation-flask
for you. If you leave out -a install
, it will simply list out the recommended auto-instrumentation packages to be installed.
For more information, check out the official OpenTelemetry docs here.
The Python package installs require gcc
and gcc-c++
, which you may need to install if you’re running a slim version of Linux. More info here.
opentelemetry-distro (Non-Launchers): The opentelemetry-distro
package installs the API, SDK, and the opentelemetry-bootstrap
and opentelemetry-instrument
tools.
Launchers only: It is necessary to force a specific version of protobuf because of Launcher compatibility issues with newer versions. This has already been fixed in opentelemetry-python.
Although auto-instrumentation is a great way to get started quickly with instrumenting your Python application, it is often not sufficient. In order to gain greater insights into your application code, you should also add manual instrumentation to your business logic. More on auto-instrumentation vs manual instrumentation here.
In order to manually instrument your Python code, you need to acquire a Tracer. The Tracer is responsible for creating Spans and interacting with the Context.
Before you can acquire a Tracer, you must first import the appropriate package. Open up your application code, and add the following import to your .py
file:
1
from opentelemetry import trace
Now you are ready to acquire your tracer. This should be declared as a global variable in your .py
file.
1
tracer = trace.get_tracer_provider().get_tracer(__name__)
The opentelemetry-instrument
tool is used to configure OpenTelemetry and auto-instrument your code. It is used as a wrapper around your Python program.
The tool will automatically detect packages used by your Python program and when possible, apply automatic tracing instrumentation on them. This means that your program will get automatic distributed tracing where applicable for free without having to make any code changes at all.
A full list of auto-instrumented Python libraries can be found here.
Start tabs
Direct (gRPC)
1
2
3
4
5
6
7
8
export OTEL_EXPORTER_OTLP_TRACES_HEADERS="lightstep-access-token=<LS_ACCESS_TOKEN>"
opentelemetry-instrument \
--traces_exporter console,otlp_proto_grpc \
--metrics_exporter console,otlp_proto_grpc \
--service_name <service_name> \
--exporter_otlp_endpoint "ingest.lightstep.com:443" \
python <your_app>.py
Collector (gRPC)
1
2
3
4
5
opentelemetry-instrument \
--traces_exporter console,otlp \
--metrics_exporter console,otlp \
--service_name test-py-auto-collector-server \
python <your_app>.py
Launcher
1
2
3
4
5
export LS_ACCESS_TOKEN="<LS_ACCESS_TOKEN>"
opentelemetry-instrument \
--service_name <service_name> \
python <your_app>.py
End tabs
To view traces in your Cloud Observability project, click explorer in the left navigation bar, and then click on any span in the Trace Analysis table.
<LS_ACCESS_TOKEN>
with your own Lightstep access token.traces_exporter
and metrics_exporter
specify which trace exporter and which metrics to use, respectively. In this case, traces and metrics are being exported to console
(stdout) and to otlp_proto_grpc
. The otlp_proto_grpc
option tells opentelemetry-instrument
to send it to an endpoint that accepts OTLP via gRPC. The full list of available options for traces_exporter
can be found here.service_name
sets the name of the service. This is the value that will show up in the Lightstep service explorer. Be sure to replace <service_name>
with your own service name.exporter_otlp_endpoint
tells opentelemetry-instrument
to send the traces to gRPC endpoint ingest.lightstep.com:443
(i.e. Lightstep).To use HTTP instead of gRPC, make sure that opentelemetry-exporter-otlp-proto-http
is installed. This should be already be installed as part of installing opentelemetry-exporter-otlp
.
Next, opentelemetry-instrument
must be altered as follows:
1
2
3
4
5
6
opentelemetry-instrument \
--traces_exporter console,otlp_proto_http \
--metrics_exporter console \
--service_name <service_name> \
--exporter_otlp_traces_endpoint "https://ingest.lightstep.com:443/traces/otlp/v0.9" \
python <your_app>.py
Noteworthy items:
traces_exporter
uses otlp_proto_http
instead of otlp_proto_grpc
.exporter_otlp_traces_endpoint
endpoint is https://ingest.lightstep.com/traces/otlp/v0.9
(see docs, instead of ingest.lightstep.com:443
).otlp_proto_http
and there is no exporter_otlp_metrics_endpoint
option, which is why metrics are being sent to console
only.Notice that the endpoint isn’t specified. That’s because it assumes that you are using the default Collector gRPC endpoint, 0.0.0.0:4317
. The above command is the equivalent of saying:
1
2
3
4
5
6
7
opentelemetry-instrument \
--traces_exporter console,otlp \
--metrics_exporter console,otlp\
--service_name test-py-auto-collector-server \
--exporter_otlp_endpoint "0.0.0.0:4317" \
--exporter_otlp_insecure true \
python server.py
If you specify the endpoint, you must also specify --exporter_otlp_insecure true
if a certificate isn’t configured with your Collector.
Noteworthy items:
otlp
, used in configuring traces_exporter
and metrics_exporter
, is equivalent to using otlp_proto_grpc
--exporter_otlp_insecure true
OTEL_EXPORTER_OTLP_TRACES_HEADERS
, because that’s already configured in the Collector’s config.yml file.service_name
sets the name of the service. This is the value that will show up in the Lightstep service explorer. Be sure to replace <service_name>
with your own service name.If you wish to use HTTP instead of gRPC, the command would then look like this:
1
2
3
4
5
opentelemetry-instrument \
--traces_exporter console,otlp_proto_http \
--metrics_exporter console,otlp_proto_http \
--service_name test-py-auto-collector-server \
python server.py
Which is the same as saying:
1
2
3
4
5
6
7
opentelemetry-instrument \
--traces_exporter console,otlp_proto_http \
--metrics_exporter console,otlp_proto_http \
--service_name test-py-auto-collector-server \
--exporter_otlp_endpoint "http://0.0.0.0:4318" \
--exporter_otlp_insecure true \
python server.py
Again, if you wish to use your own Collector endpoint, simply replace the value in exporter_otlp_endpoint
, making sure that you prefix it with http://
or https://
. Remember to add --exporter_otlp_insecure true
if you don’t have a Certificate configured with your Collector.
OTEL_EXPORTER_OTLP_TRACES_HEADERS="lightstep-access-token=<LS_ACCESS_TOKEN>"
, the Launcher allows us to simplify it to LS_ACCESS_TOKEN="<LS_ACCESS_TOKEN>"
.exporter_otlp_traces_endpoint
, because that is already implicitly set to ingest.lightstep.com:443
.service_name
sets the name of the service. This is the value that will show up in the Lightstep service explorer. Be sure to replace <service_name>
with your own service name.To send data to Lightstep via an OpenTelmetry Collector using gRPC, opentelemetry-instrument
would be configured as follows:
1
2
3
4
5
opentelemetry-instrument \
--service_name <service_name> \
--exporter_otlp_traces_endpoint "0.0.0.0:4317" \
--exporter_otlp_traces_insecure true \
python <your_app>.py
Noteworthy items:
LS_ACCESS_TOKEN
, since that’s already configured in the Collector’s config.yaml file.exporter_otlp_endpoint
to send traces to a Collector, the traces will be sent directly to ingest.lightstep.com:443
instead of via the Collector. Instead, you need to override exporter_otlp_traces_endpoint
,exporter_otlp_traces_endpoint
sends traces to a Collector running on 0.0.0.0:4317
(gRPC). If you wish to use a different Collector endpoint, simply include exporter_otlp_traces_endpoint
, using your own Collector’s endpoint.exporter_otlp_traces_insecure
is set to true
. This is required if you are using a Collector and if a certificate isn’t configured in the Collector.The Python package installs require gcc
and gcc-c++
, which you may need to install if you’re running a slim version of Linux (e.g., CentOS):
Start tabs
CentOS
1
2
yum -y install python3-devel
yum -y install gcc-c++
Debian/Ubuntu
1
2
apt install -y python3-dev
apt install -y build-essential
Alpine
1
2
apk add python3-dev
apk add build-base
End tabs
To debug Python gRPC connectivity issues, set the following gRPC debug environment variables:
1
2
3
export GRPC_VERBOSITY=debug
export GRPC_TRACE=http,call_error,connectivity_state
opentelemetry-instrument python <your_app>.py
If multiple versions of OpenTelemetry are installed, traces are not created or propagated correctly. Check your dependencies to ensure that only a single version of OpenTelemetry is installed.
Cloud Observability’s public Microsatellites only accept spans via a secure endpoint. If you see security related errors (e.g., Netty TLS errors with Java), you may not have the necessary root certificate installed on the machine where the tracer is running. To add a root certificate, see the documentation about encrypted connections.
Not seeing metrics come through the OpenTelemetry Collector? Make sure that you have defined a Metrics pipeline in your Collector’s YAML config file.
Updated Sep 13, 2022