The OpenTelemetry Collector provides an integration with Elasticsearch to ingest metrics. The Elasticsearch receiver fetches node and cluster metrics for Elasticsearch operators from the configured path in the Collector. From there the metrics are processed and exported to Lightstep Observability.

To complete the integration, you will:

  • Configure and enable the Elasticsearch receiver for the Collector

Prerequisites

Configure the Collector receiver

In the Collector configuration file, add Elasticsearch as a receiver and set the following:

  • endpoint: The base URL where the Collector can scrape metrics.
  • tls: ca_file: Path to the CA certificate for TLS.
  • username: A username with monitoring permissions to identify the Collector.
  • password: The password, which must be configured in the Elasticsearch keystore.
1
2
3
4
5
6
7

receivers:
  elasticsearch:
    endpoint: "https://elasticsearch:9200"
    tls:
      ca_file: /elastic-ca.crt
    username: elastic
    password: $ELASTICSEARCH_PASSWORD

While the default settings collect all cluster and node level metrics, there are additional options to enable specified monitoring of each metric type. For more on how to customize metric collections or omit certain node or cluster metrics, see the Elasticsearch Receiver repo’s readme.

Enable the Collector receiver

Once the Elasticsearch receiver is configured, enable it by adding it to one or more pipelines as described in the Collector configuration documentation.

Validate metrics are reporting to Lightstep

You can validate that metrics are reporting to Lightstep on the Metrics details page in Project settings.

  1. In Lightstep, click Project settings > Metric details.

  2. Search for Elasticsearch metric names. Search for metric

    See the receiver’s metadata file for a complete list of emitted metrics.

  3. If needed, click on the metric to edit the description and how the units are displayed in Lightstep.

Create a dashboard for the metrics

Use the Lightstep Terraform Provider to create a dashboard for the metrics.

Additional resources

Lightstep example For a more complete example that’s ready to run, see the Elasticsearch integration in Lightstep OpenTelemetry Examples.

Elasticsearch automated configuration examples