This topic walks you through how to install an OpenTelemetry Collector with a sample configuration for both metrics and traces that serves as a base for other guides.
While there are many ways to deploy a Collector on Kubernetes, we recommend using the OpenTelemetry Operator. The Operator installs custom resources into your cluster, allowing you to create OpenTelemetry Collectors and have the operator handle their coordination.
You can create a pre-built dashboard to monitor the Operator from the Dashboard list view. Or use the Cloud Observability Terraform Provider to create a dashboard.
If you’re interested in learning more about the trade-offs between different deployment modes, read more here.
To install the Collector, you need to (in this order):
These instructions install the Collector to a Kubernetes cluster as a single replica Kubernetes Deployment (also called “standalone” mode) using the Operator. For any questions, please contact your Customer Success representative.
You must be able to run ValidatingWebhookConfigurations
and MutatingWebhookConfigurations
within your Kubernetes cluster; these are used to verify the Collector configuration.
Run the following command to add the following Helm respositories and pull the latest charts:
1
2
3
4
5
helm repo add jetstack https://charts.jetstack.io
helm repo add open-telemetry https://open-telemetry.github.io/opentelemetry-helm-charts
helm repo add prometheus https://prometheus-community.github.io/helm-charts
helm repo add lightstep https://lightstep.github.io/otel-collector-charts
helm repo update
Next, install the cert-manager charts on your cluster. The Cert Manager manages certificates needed by the Operator to subscribe to in-cluster Kubernetes events.
1
2
3
4
5
6
helm install \
cert-manager jetstack/cert-manager \
--namespace cert-manager \
--create-namespace \
--version v1.8.0 \
--set installCRDs=true
0.35.1
with the Cloud Observability Helm chart.
1
2
3
helm install \
opentelemetry-operator open-telemetry/opentelemetry-operator \
-n default --version 0.35.1
deployed
:
1
helm list -A
Kubernetes has built-in support for hundreds of useful metrics that help teams understand the health of their containers, pods, nodes, workloads, and internal system components. Cloud Observability provides a Helm chart to automatically configure collectors to send these metrics to Cloud Observability.
Create a secret that holds your Cloud Observability Access Token.
1
2
export LS_TOKEN='<your-token>'
kubectl create secret generic otel-collector-secret -n default --from-literal="LS_TOKEN=$LS_TOKEN"
Install the collector-k8s
chart. This chart automatically creates collectors to pull Kubernetes metrics and send them to your Cloud Observability project. We recommend you also specify the name of your cluster when installing the chart, which your can use by setting the clusterName
variable:
1
helm install kube-otel-stack lightstep/kube-otel-stack -n default --set metricsCollector.clusterName=your-cluster-name
Verify the pods from the charts have been deployed with no errors:
1
kubectl get pods
You should see pods for a node exporter, the operator, kube-state-metrics, and multiple collectors.
The default OTLP Exporter from a Collector enables gzip
compression and TLS.
Depending on your network configuration, you may need to enable or disable
certain other gRPC features. This
page
contains a complete list of configuration parameters for the Collector gRPC
client.
In the event that you are unable to establish a gRPC connection to the Cloud Observability
Observability platform, you can use the
grpcurl tool to ensure connectivity
from your network to our public satellites. Run the following command, replacing
<YOUR_ACCESS_TOKEN>
with your project’s access
token:
1
grpcurl -H 'lightstep-access-token:<YOUR_ACCESS_TOKEN>' ingest.lightstep.com:443 list
You should see the following output, or something similar:
1
2
3
4
5
grpc.reflection.v1alpha.ServerReflection
jaeger.api_v2.CollectorService
lightstep.collector.CollectorService
lightstep.egress.CollectorService
opentelemetry.proto.collector.trace.v1.TraceService
If you do not see this output, or the request hangs, then something is blocking
gRPC traffic from transiting your network to ours. Please ensure that any
proxies are passing through the lightstep-access-token
header.
For additional troubleshooting recommendations, see Troubleshooting Missing Data in Cloud Observability.
Now that you’ve succesfully installed an OpenTelemetry Operator and Collector to your cluster, there are many ways to tune your setup to your needs.
Performance test and tune the Collector
Updated Oct 21, 2022