Elasticsearch metrics

The OpenTelemetry Collector provides an integration with Elasticsearch to ingest metrics. The Elasticsearch receiver fetches node and cluster metrics for Elasticsearch operators from the configured path in the Collector. From there the metrics are processed and exported to Cloud Observability.

To complete the integration, you will:

Configure and enable the Elasticsearch receiver for the Collector

Prerequisites

Elasticsearch v7.9 or later
A user configured in the Elasticsearch keystore with monitor or manage cluster privileges.
You’ve configured the Collector to export metric data to Cloud Observability.

Configure the Collector receiver

In the Collector configuration file, add Elasticsearch as a receiver and set the following:

endpoint: The base URL where the Collector can scrape metrics.
tls: ca_file: Path to the CA certificate for TLS.
username: A username with monitoring permissions to identify the Collector.
password: The password, which must be configured in the Elasticsearch keystore.

receivers:
  elasticsearch:
    endpoint: "https://elasticsearch:9200"
    tls:
      ca_file: /elastic-ca.crt
    username: elastic
    password: $ELASTICSEARCH_PASSWORD

While the default settings collect all cluster and node level metrics, there are additional options to enable specified monitoring of each metric type. For more on how to customize metric collections or omit certain node or cluster metrics, see the Elasticsearch Receiver repo’s readme.

Enable the Collector receiver

Once the Elasticsearch receiver is configured, enable it by adding it to one or more pipelines as described in the Collector configuration documentation.

Validate metrics are reporting to Cloud Observability

You can validate that metrics are reporting to Cloud Observability on the Metrics details page in Project settings.

In Cloud Observability, click Project settings > Metric details.
Search for Elasticsearch metric names.

See the receiver’s metadata file for a complete list of emitted metrics.
If needed, click on the metric to edit the description and how the units are displayed in Cloud Observability.

Create a dashboard for the metrics

Use the Cloud Observability Terraform Provider to create a dashboard for the metrics.

Additional resources

Cloud Observability example For a more complete example that’s ready to run, see the Elasticsearch integration in Cloud Observability OpenTelemetry Examples.

Elasticsearch automated configuration examples