This Quick Start guide shows you how to configure your Python applications to send OpenTelemetry traces to Lightstep Observability.
This guide does not provide documentation on application instrumentation. For Python-specific information on instrumentation, please see the Python OpenTelemetry getting started guide.
The sections below contain code snippets only. For full code listings, please see python/opentelemetry
and python/launcher
in the Lightstep OpenTelemetry examples repository.
Sending OpenTelemetry data directly to Lightstep without a Collector for most developer setups will suffice. For non-development setups, however, it is highly recommended that you send OpenTelemetry data to Lightstep Observability by way of the OpenTelemetry Collector. This can be done with or without a Launcher, as we’ll see below.
Pre-Requisites
Before you get started with sending OpenTelemetry data to Lightstep, you will need the following:
- A Lightstep Observability account: create a free account here.
- A Lightstep access token for the Lightstep Observability project you would like to use.
- The OpenTelemetry Getting Started guide for Python.
Install OpenTelemetry Packages
In your application code, you will need to install dependencies and import OpenTelemetry packages before you can send data to Lightstep Observability.
Start by installing the Python OpenTelemetry packages. Do this by opening up a terminal window and pasting the following snippet:
Start tabs
Direct (gRPC)
1
2
3
4
pip install opentelemetry-distro \
opentelemetry-exporter-otlp
opentelemetry-bootstrap -a install
Collector
1
2
3
4
pip install opentelemetry-distro \
opentelemetry-exporter-otlp
opentelemetry-bootstrap -a install
Launcher
1
2
3
4
pip install opentelemetry-launcher \
protobuf==3.20.1
opentelemetry-bootstrap -a install
End tabs
The opentelemetry-bootstrap -a install
command will read through the packages installed in your active site-packages
folder, and will install the applicable auto-instrumentation libraries. For example, if you already installed the flask
package, running opentelemetry-bootstrap -a install
will install opentelemetry-instrumentation-flask
for you. If you leave out -a install
, it will simply list out the recommended auto-instrumentation packages to be installed.
For more information, check out the official OpenTelemetry docs here.
The Python package installs require gcc
and gcc-c++
, which you may need to install if you’re running a slim version of Linux. More info here.
opentelemetry-distro (Non-Launchers): The opentelemetry-distro
package installs the API, SDK, and the opentelemetry-bootstrap
and opentelemetry-instrument
tools.
Launchers only: It is necessary to force a specific version of protobuf because of Launcher compatibility issues with newer versions. This has already been fixed in opentelemetry-python.
Code Setup
Although auto-instrumentation is a great way to get started quickly with instrumenting your Python application, it is often not sufficient. In order to gain greater insights into your application code, you should also add manual instrumentation to your business logic. More on auto-instrumentation vs manual instrumentation here.
In order to manually instrument your Python code, you need to acquire a Tracer. The Tracer is responsible for creating Spans and interacting with the Context.
Import OpenTelemetry Packages
Before you can acquire a Tracer, you must first import the appropriate package. Open up your application code, and add the following import to your .py
file:
1
from opentelemetry import trace
Acquire a Tracer
Now you are ready to acquire your tracer. This should be declared as a global variable in your .py
file.
1
tracer = trace.get_tracer_provider().get_tracer(__name__)
Start Auto-Instrumentation
The opentelemetry-instrument
tool is used to configure OpenTelemetry and auto-instrument your code. It is used as a wrapper around your Python program.
The tool will automatically detect packages used by your Python program and when possible, apply automatic tracing instrumentation on them. This means that your program will get automatic distributed tracing where applicable for free without having to make any code changes at all.
A full list of auto-instrumented Python libraries can be found here.
Start tabs
Direct (gRPC)
1
2
3
4
5
6
7
8
export OTEL_EXPORTER_OTLP_TRACES_HEADERS="lightstep-access-token=<LS_ACCESS_TOKEN>"
opentelemetry-instrument \
--traces_exporter console,otlp_proto_grpc \
--metrics_exporter console,otlp_proto_grpc \
--service_name <service_name> \
--exporter_otlp_endpoint "ingest.lightstep.com:443" \
python <your_app>.py
Collector (gRPC)
1
2
3
4
5
opentelemetry-instrument \
--traces_exporter console,otlp \
--metrics_exporter console,otlp \
--service_name test-py-auto-collector-server \
python <your_app>.py
Launcher
1
2
3
4
5
export LS_ACCESS_TOKEN="<LS_ACCESS_TOKEN>"
opentelemetry-instrument \
--service_name <service_name> \
python <your_app>.py
End tabs
To view traces in your Lightstep Observability project, click explorer in the left navigation bar, and then click on any span in the Trace Analysis table.
Notes: Direct (OTLP from application code)
- Replace
<LS_ACCESS_TOKEN>
with your own Lightstep access token. traces_exporter
andmetrics_exporter
specify which trace exporter and which metrics to use, respectively. In this case, traces and metrics are being exported toconsole
(stdout) and tootlp_proto_grpc
. Theotlp_proto_grpc
option tellsopentelemetry-instrument
to send it to an endpoint that accepts OTLP via gRPC. The full list of available options fortraces_exporter
can be found here.service_name
sets the name of the service. This is the value that will show up in the Lightstep service explorer. Be sure to replace<service_name>
with your own service name.exporter_otlp_endpoint
tellsopentelemetry-instrument
to send the traces to gRPC endpointingest.lightstep.com:443
(i.e. Lightstep).
To use HTTP instead of gRPC, make sure that opentelemetry-exporter-otlp-proto-http
is installed. This should be already be installed as part of installing opentelemetry-exporter-otlp
.
Next, opentelemetry-instrument
must be altered as follows:
1
2
3
4
5
6
opentelemetry-instrument \
--traces_exporter console,otlp_proto_http \
--metrics_exporter console \
--service_name <service_name> \
--exporter_otlp_traces_endpoint "https://ingest.lightstep.com:443/traces/otlp/v0.9" \
python <your_app>.py
Noteworthy items:
- The
traces_exporter
usesotlp_proto_http
instead ofotlp_proto_grpc
. - The
exporter_otlp_traces_endpoint
endpoint ishttps://ingest.lightstep.com/traces/otlp/v0.9
(see docs, instead ofingest.lightstep.com:443
). - There is currently no metrics support for
otlp_proto_http
and there is noexporter_otlp_metrics_endpoint
option, which is why metrics are being sent toconsole
only.
Notes: OpenTelemetry Collector
Notice that the endpoint isn’t specified. That’s because it assumes that you are using the default Collector gRPC endpoint, 0.0.0.0:4317
. The above command is the equivalent of saying:
1
2
3
4
5
6
7
opentelemetry-instrument \
--traces_exporter console,otlp \
--metrics_exporter console,otlp\
--service_name test-py-auto-collector-server \
--exporter_otlp_endpoint "0.0.0.0:4317" \
--exporter_otlp_insecure true \
python server.py
If you specify the endpoint, you must also specify --exporter_otlp_insecure true
if a certificate isn’t configured with your Collector.
Noteworthy items:
otlp
, used in configuringtraces_exporter
andmetrics_exporter
, is equivalent to usingotlp_proto_grpc
- To use a different Collector endpoint, simply replace it with your own. If you don’t have a Certificate configured with your Collector, remember to add
--exporter_otlp_insecure true
- You don’t need to set
OTEL_EXPORTER_OTLP_TRACES_HEADERS
, because that’s already configured in the Collector’s config.yml file. service_name
sets the name of the service. This is the value that will show up in the Lightstep service explorer. Be sure to replace<service_name>
with your own service name.
If you wish to use HTTP instead of gRPC, the command would then look like this:
1
2
3
4
5
opentelemetry-instrument \
--traces_exporter console,otlp_proto_http \
--metrics_exporter console,otlp_proto_http \
--service_name test-py-auto-collector-server \
python server.py
Which is the same as saying:
1
2
3
4
5
6
7
opentelemetry-instrument \
--traces_exporter console,otlp_proto_http \
--metrics_exporter console,otlp_proto_http \
--service_name test-py-auto-collector-server \
--exporter_otlp_endpoint "http://0.0.0.0:4318" \
--exporter_otlp_insecure true \
python server.py
Again, if you wish to use your own Collector endpoint, simply replace the value in exporter_otlp_endpoint
, making sure that you prefix it with http://
or https://
. Remember to add --exporter_otlp_insecure true
if you don’t have a Certificate configured with your Collector.
Notes: Launcher
- Instead of setting the Lightstep access token via
OTEL_EXPORTER_OTLP_TRACES_HEADERS="lightstep-access-token=<LS_ACCESS_TOKEN>"
, the Launcher allows us to simplify it toLS_ACCESS_TOKEN="<LS_ACCESS_TOKEN>"
. - There is no need to specify
exporter_otlp_traces_endpoint
, because that is already implicitly set toingest.lightstep.com:443
. service_name
sets the name of the service. This is the value that will show up in the Lightstep service explorer. Be sure to replace<service_name>
with your own service name.
To send data to Lightstep via an OpenTelmetry Collector using gRPC, opentelemetry-instrument
would be configured as follows:
1
2
3
4
5
opentelemetry-instrument \
--service_name <service_name> \
--exporter_otlp_traces_endpoint "0.0.0.0:4317" \
--exporter_otlp_traces_insecure true \
python <your_app>.py
Noteworthy items:
- Do not set
LS_ACCESS_TOKEN
, since that’s already configured in the Collector’s config.yaml file. - If you attempt to override
exporter_otlp_endpoint
to send traces to a Collector, the traces will be sent directly toingest.lightstep.com:443
instead of via the Collector. Instead, you need to overrideexporter_otlp_traces_endpoint
, exporter_otlp_traces_endpoint
sends traces to a Collector running on0.0.0.0:4317
(gRPC). If you wish to use a different Collector endpoint, simply includeexporter_otlp_traces_endpoint
, using your own Collector’s endpoint.exporter_otlp_traces_insecure
is set totrue
. This is required if you are using a Collector and if a certificate isn’t configured in the Collector.- There is currently no HTTP support for Python Launchers.
Troubleshooting
Python package installs
The Python package installs require gcc
and gcc-c++
, which you may need to install if you’re running a slim version of Linux (e.g., CentOS):
Start tabs
CentOS
1
2
yum -y install python3-devel
yum -y install gcc-c++
Debian/Ubuntu
1
2
apt install -y python3-dev
apt install -y build-essential
Alpine
1
2
apk add python3-dev
apk add build-base
End tabs
gRPC Connectivity
To debug Python gRPC connectivity issues, set the following gRPC debug environment variables:
1
2
3
export GRPC_VERBOSITY=debug
export GRPC_TRACE=http,call_error,connectivity_state
opentelemetry-instrument python <your_app>.py
Check for multiple versions of OpenTelemetry
If multiple versions of OpenTelemetry are installed, traces are not created or propagated correctly. Check your dependencies to ensure that only a single version of OpenTelemetry is installed.
Check that security is configured correctly
Lightstep’s public Microsatellites only accept spans via a secure endpoint. If you see security related errors (e.g., Netty TLS errors with Java), you may not have the necessary root certificate installed on the machine where the tracer is running. To add a root certificate, see the documentation about encrypted connections.
Check that Metrics are enabled in the OpenTelemetry Collector
Not seeing metrics come through the OpenTelemetry Collector? Make sure that you have defined a Metrics pipeline in your Collector’s YAML config file.