The OpenTelemetry Collector provides an integration with Elasticsearch to ingest metrics. The Elasticsearch receiver fetches node and cluster metrics for Elasticsearch operators from the configured path in the Collector. From there the metrics are processed and exported to Lightstep Observability.
To complete the integration, you will:
- Configure and enable the Elasticsearch receiver for the Collector
Prerequisites
- Elasticsearch v7.9 or later
- A user configured in the Elasticsearch keystore with monitor or manage cluster privileges.
- You’ve configured the Collector to export metric data to Lightstep Observability.
Configure the Collector receiver
In the Collector configuration file, add Elasticsearch as a receiver and set the following:
endpoint
: The base URL where the Collector can scrape metrics.tls: ca_file
: Path to the CA certificate for TLS.username
: A username with monitoring permissions to identify the Collector.password
: The password, which must be configured in the Elasticsearch keystore.
1
2
3
4
5
6
7
receivers:
elasticsearch:
endpoint: "https://elasticsearch:9200"
tls:
ca_file: /elastic-ca.crt
username: elastic
password: $ELASTICSEARCH_PASSWORD
While the default settings collect all cluster and node level metrics, there are additional options to enable specified monitoring of each metric type. For more on how to customize metric collections or omit certain node or cluster metrics, see the Elasticsearch Receiver repo’s readme.
Enable the Collector receiver
Once the Elasticsearch receiver is configured, enable it by adding it to one or more pipelines as described in the Collector configuration documentation.
Validate metrics are reporting to Lightstep
You can validate that metrics are reporting to Lightstep on the Metrics details page in Project settings.
-
In Lightstep, click Project settings > Metric details.
-
Search for Elasticsearch metric names.
See the receiver’s metadata file for a complete list of emitted metrics.
-
If needed, click on the metric to edit the description and how the units are displayed in Lightstep.
Create a dashboard for the metrics
Use the Lightstep Terraform Provider to create a dashboard for the metrics.
Additional resources
Lightstep example For a more complete example that’s ready to run, see the Elasticsearch integration in Lightstep OpenTelemetry Examples.
Elasticsearch automated configuration examples