Lightstep Observability’s metrics integration with Amazon uses CloudWatch Metric Streams to send metric data into its system. In addition, Lightstep Observability supports collecting resource metadata from the following AWS products:

API Gateway ApplicationELB DynamoDB
ECS EC2 ELB
ElastiCache Elastic File System Kinesis
Lambda S3 SNS
SQS RDS ...more coming soon

You can add EC2 resource tags and other useful EC2 metadata as labels to any metric from an EC2 container, and you can also add ECS, RDS, ElastiCache, ELB, and ApplicationELB resource tags as labels to CloudWatch metrics.

Once Lightstep Observability is ingesting metrics from CloudWatch, you can use Terraform to create a number of pre-built dashboards.

In order to take full advantage of Lightstep Observability’s Change Intelligence features, tag your resources with an appropriate service.name tag.

AWS Tag Management

You need to create a Cloudwatch Metric Stream to push metric data to Lightstep Observability. Additionally, you need to create an AWS role and policy for Lightstep Observability and then grant Lightstep Observability read-only access to your resource metadata. Lightstep strongly recommends using Terraform for this integration. If you are unable to use Terraform, please contact your Technical Account Manager for manual integration instructions.

About CloudWatch metric streams

Amazon offers CloudWatch Metric Streams as a way to continually stream metrics to Lightstep Observability and similar services. Metric streams offer near-real-time delivery and an expected latency of as low as 3 minutes. AWS has embraced the OpenTelemtry standard and currently streams metrics in OTLP v0.7. Metric streams are a reliable and cost-effective way to deliver your CloudWatch metrics to Lightstep Observability.

Use Terraform to integrate

The Terraform repository creates a metric stream, a kinesis firehose, along with the necessary roles and policies for the stream as well as role and the associated policy that grant Lightstep Observability access to your AWS resource metadata.

Requirements

You need the following to use the Terraform repository:

Download and run the Terraform repository

  1. Download and unzip the repository. Download Terraform Repository

  2. Thoroughly review the README and example.tfvars. They contain instructions and detailed information regarding all options. Basic installation instructions:

    1
    2
    3
    4
    5
    6
    7
    8
    9
    10
    11
    12
    13
    14
    15
    16
    17
    18
    19
    20
    21
    22
    23
    
     # You may also download and unzip via the command line.
     % wget https://github.com/lightstep/aws-cloudwatch-metric-stream-terraform/archive/main.zip
     % unzip main.zip
     % cd aws-cloudwatch-metric-stream-terraform-main
    
     # Configure your AWS credentials.
     % export AWS_ACCESS_KEY_ID=<access-key-id>
     % export AWS_SECRET_ACCESS_KEY=<secret-access-key>
    
     # Initialize your terraform backend.
     % terraform init
    
     # Apply terraform, entering your Lightstep Observability project access token when prompted.
     % terraform apply
     var.lightstep_access_token
       Lightstep Observability project access token
    
       Enter a value: <your-lightstep-access-token>
    
     # -----
     # The above is the minimal install with default values.
     # For custom install, copy example.tfvars (eg: `cp example.tfvars my-vars.tfvars`), edit the new file, then run:
     % terraform apply -var-file="my-vars.tfvars"  
    
  3. The script outputs an Integration Role ARN and an External ID (a random string used to provide extra security). Copy these as you’ll need to paste them into the UI in the next step.

Add credentials to Lightstep Observability

To complete the integration, you need to add the AWS credential information to Lightstep Observability.

  1. In Lightstep Observability, click Settings in the left navigation menu to open the Project Settings page.

  2. On the Project Settings page, scroll down to Metric integrations. Metric integrations on the Project Settings page

  3. Click Enable and enter the following AWS credential information in the dialog:
    • The Integration Role ARN
    • External ID

      These values are included in the output after running the Terraform script. Terraform output

    Enter AWS credentials

  4. Click Enable.

    Your metric data is now enabled in Lightstep Observability. AWS Metrics in Lightstep Observability Dashboard

    It may take up to 30 minutes for metric data to appear in Lightstep Observability using the default configuration. This may vary if you set custom values for buffer_size and/or buffer_interval.

    It may take up to 30 minutes for changes to resource metadata to propagate to Lightstep Observability. For example, if you add a new custom tag to an EC2 instance, that tag may not appear in Lightstep Observability for up to 30 minutes.

Update the integration

You may reconfigure your integration at any time by re-running terraform apply with different configuration values. We recommend using an external .tfvars file.

1
2
3
4
5
6
% cd /<path>/<to>/aws-cloudwatch-metric-stream-terraform-main
% cp example.tfvars my-vars.tfvars

# Edit my-vars.tfvars in your favorite text editor.

% terraform apply -var-file="my-vars.tfvars"  

It may take up to 30 minutes for configuration changes to propagate to Lightstep Observability. This may vary if you set custom values for buffer_size and/or buffer_interval.

Remove the integration

You may remove your metric stream integration at any time using terraform.

1
2
% cd /<path>/<to>/aws-cloudwatch-metric-stream-terraform-main
% terraform destroy

Notes

  • Percentiles are not included in CloudWatch Metric Streams.
  • CloudWatch Metric Streams do not include metrics with timestamps older than two hours - for example, metrics that are aggregated on a daily basis.

If you have previously integrated your CloudWatch metrics with Lightstep Observability prior to the release of our metric streams integration, we strongly recommend transitioning to the streaming integration. It is lower latency, more reliable, and cheaper! Please contact your Technical Account Manager who will be happy to guide you through the upgrade.