Cilium metrics with Hubble

The OpenTelemetry Collector, when configured with a Prometheus receiver, provides an integration with Cilium to scrape Prometheus metrics. The Collector uses the Prometheus Receiver to fetch metrics from the configured path in the Cilium configuration file. From there, the metrics are processed and exported to Cloud Observability. This integration also includes using Kubernetes operators for both the Collector and Cilium - the Cilium Operator ensures that services requiring monitoring expose relevant metrics.

To complete the integration, you will:

Configure Cilium to use the Prometheus exporter.
Configure the Collector to use the Cilium endpoint as a scrape target for the Prometheus receiver.
Enable the integration by adding it to a pipeline.

Prerequisites

Cilium with Hubble running in a Kubernetes environment.
A deployed Kubernetes operator for both the Collector and Cilium.

See also Use Prometheus with an OpenTelemetry Collector on Kubernetes.
(Cloud Observability documentation)
You’ve configured the Collector to export metric data to Cloud Observability.

Configure Cilium reporting

When running Cilium with Hubble and a Cilium operator in Kubernetes environment, you can export Prometheus metrics from:

Prometheus
The Cilium Operator
Hubble

You need to enable metrics for each of these and optionally configure ports (if you can’t use the default).

Following are the Cilium settings needed to publish the metrics at the default ports on a service called cilim-agent.

prometheus.enabled=true  
operator.prometheus.enabled=true
hubble.prometheus.enabled=true

You can find additional details, such as how to configure custom ports and using other configuration formats, see the Cilium documentation.

Configure the Collector receiver

In the Collector configuration file, configure the Prometheus receiver to use the Cilium Prometheus endpoint as a scrape target.

This example uses the service names and ports sent in the Cilium Operator.

receivers:
  prometheus:
    config:
      scrape_configs:
        - job_name: 'otel-cilium-eg'
          static_configs:
            - targets: 
                - cilium-agent.kube-system.svc.cluster.local:9962
                - cilium-agent.kube-system.svc.cluster.local:9963
                - hubble-metrics.kube-system.svc.cluster.local:9965

The OpenTelemetry repo’s readme provides additional details about Prometheus receiver configuration.

More details about the Prometheus scrape configuration can be found here.

Enable the Collector receiver

Once the Cilium receiver is configured, enable it by adding it to one or more pipelines as described in the Collector configuration documentation.

Validate metrics are reporting to Cloud Observability

You can validate that metrics are reporting to Cloud Observability on the Metrics details page in Settings.

In Cloud Observability, click Settings > Metric details.
Search for Cilium metric names.

See the Cilium docs for a complete list of emitted metrics.
If needed, click on the metric to edit the description and how the units are displayed in Cloud Observability.

Create a dashboard for the metrics

You can create a pre-built dashboard for this integration from the Dashboard list view. Or use the Cloud Observability Terraform Provider to create a dashboard.

Additional resources

For a more complete example that’s ready to run, see the Cilium integration in Cloud Observability OpenTelemetry Examples.
OpenTelemetry Operator documentation