The OpenTelemetry Collector provides an integration with Elasticsearch to ingest metrics. The Elasticsearch receiver fetches node and cluster metrics for Elasticsearch operators from the configured path in the Collector. From there the metrics are processed and exported to Cloud Observability.
To complete the integration, you will:
In the Collector configuration file, add Elasticsearch as a receiver and set the following:
endpoint: The base URL where the Collector can scrape metrics.
tls: ca_file: Path to the CA certificate for TLS.
username: A username with monitoring permissions to identify the Collector.
password: The password, which must be configured in the Elasticsearch keystore.
1 2 3 4 5 6 7 receivers: elasticsearch: endpoint: "https://elasticsearch:9200" tls: ca_file: /elastic-ca.crt username: elastic password: $ELASTICSEARCH_PASSWORD
While the default settings collect all cluster and node level metrics, there are additional options to enable specified monitoring of each metric type. For more on how to customize metric collections or omit certain node or cluster metrics, see the Elasticsearch Receiver repo’s readme.
Once the Elasticsearch receiver is configured, enable it by adding it to one or more pipelines as described in the Collector configuration documentation.
You can validate that metrics are reporting to Cloud Observability on the Metrics details page in Project settings.
In Cloud Observability, click Project settings > Metric details.
Search for Elasticsearch metric names.
See the receiver’s metadata file for a complete list of emitted metrics.
If needed, click on the metric to edit the description and how the units are displayed in Cloud Observability.
Use the Cloud Observability Terraform Provider to create a dashboard for the metrics.
Cloud Observability example For a more complete example that’s ready to run, see the Elasticsearch integration in Cloud Observability OpenTelemetry Examples.
Elasticsearch automated configuration examples
Updated Dec 1, 2022