This topic provides instructions for scaling the OpenTelemetry Collector deployment for tracing, metrics, or logs in Kubernetes.
There are two ways to scale your deployment: vertically and horizontally. In Kubernetes, vertical scaling entails increasing the memory and CPU requests/limits of your Kubernetes pod. Horizontal scaling means adding more pods (collector instances) to handle your traffic.
We recommend you use the pre-built dashboard for the OpenTelemetyr Collector to monitor and tune resource usage of collectors.
Define a new
values.yaml for your collectors deployed using the Kubernetes Quickstart. Adjust the following blocks, as needed.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 # Vertically scale the tracesCollector tracesCollector: resources: limits: cpu: CPU_LIMIT_HERE memory: MEMORY_LIMIT_HERE requests: cpu: CPU_LIMIT_HERE memory: MEMORY_LIMIT_HERE # Vertically scale the singleton collector for cluster-level metrics data clusterCollector: resources: limits: cpu: CPU_LIMIT_HERE memory: MEMORY_LIMIT_HERE requests: cpu: CPU_LIMIT_HERE memory: MEMORY_LIMIT_HERE # Vertically scale the log collector logsCollector: resources: limits: cpu: CPU_LIMIT_HERE memory: MEMORY_LIMIT_HERE requests: cpu: CPU_LIMIT_HERE memory: MEMORY_LIMIT_HERE
Ensure that your memory and CPU values are well formatted for Kubernetes
values.yaml for your collector deployment, adjust the replica count:
1 2 3 4 5 tracesCollector: hpa: minReplicas: 1 maxReplicas: 3 targetMemoryUtilization: 70
To upgrade the otel-cloud-stack installed in the Kubernetes Quickstart, run the following:
1 helm upgrade otel-cloud-stack lightstep/otel-cloud-stack -f values.yaml -n opentelemetry --install
Your new Collectors should appear in your OpenTelemetry Collector dashboard.
Updated Oct 21, 2022