Once you’ve integrated with AWS Cloudwatch, you have access to metrics from AWS Kinesis Data Firehose, which is a completely managed service for sending real-time streaming data to locations like Lightstep, AWS data storage services, and other monitoring platforms.

All available AWS integrations

To verify metrics are reporting, search for the metrics in the Metric details section of the Project Settings page.

The following table shows the Kinesis Data Firehose metrics ingested by Lightstep.

Metric Name Unit Description
aws.kinesisfirehose.partition_count count The number of active partitions, or the number of partitions that are currently being processed.
aws.kinesisfirehose.partition_count_exceeded count The number of partition count cap gone over.
aws.kinesisfirehose.jq_processing_duration milliseconds The JQ Lambda function's return value is the time it took to execute a JQ expression.
aws.kinesisfirehose.per_partition_throughput bytes The number of bytes throughput is being processed for each partition.
aws.kinesisfirehose.delivery_to_s3_object_count count The number of items are being delivered to your S3 bucket.
aws.kinesisfirehose.delivery_to_amazon_open_search_service_bytes bytes The number of bytes indexed to OpenSearch Service during the defined time frame.
aws.kinesisfirehose.delivery_to_amazon_open_search_service_data_freshness seconds The age of the oldest record in Kinesis Data Firehose, measured from its entry to the present.
aws.kinesisfirehose.delivery_to_amazon_open_search_service_records count The total number of documents that OpenSearch Service has indexed during the time frame.
aws.kinesisfirehose.delivery_to_amazon_open_search_service_success count The total of records that were successfully indexed over the total number of records that were attempted.
aws.kinesisfirehose.delivery_to_redshift_bytes count The number of data copied to Amazon Redshift during the time frame specified.
aws.kinesisfirehose.delivery_to_redshift_records count The number of data copied to Amazon Redshift during the time frame.
aws.kinesisfirehose.delivery_to_redshift_success count The sum of all Amazon Redshift COPY commands divided by the total number of successful COPY commands.
aws.kinesisfirehose.delivery_to_s3_bytes bytes The number of bytes sent to Amazon S3 during the defined time frame.
aws.kinesisfirehose.backup_to_s3_records count The total number of records sent to Amazon S3 for backup during the designated time frame.
aws.kinesisfirehose.delivery_to_s3_data_freshness seconds The age of the oldest record in Kinesis Data Firehose, measured from its entry to the present.
aws.kinesisfirehose.delivery_to_s3_records count The number of records sent to Amazon S3 during the time frame specified.
aws.kinesisfirehose.backup_to_s3_success count The sum of all Amazon S3 backup put commands is greater than the sum of the Amazon S3 put commands that were successful.
aws.kinesisfirehose.delivery_to_splunk_bytes bytes The number of data sent to Splunk during the designated time frame.
aws.kinesisfirehose.delivery_to_splunk_data_ack_latency seconds The latency it takes for Splunk to acknowledge receiving data from Kinesis Data Firehose.
aws.kinesisfirehose.delivery_to_splunk_data_freshness seconds Age of the oldest record in Kinesis Data Firehose, measured from its entry to the present.
aws.kinesisfirehose.delivery_to_splunk_records count The number of records sent to Splunk during the designated time frame.
aws.kinesisfirehose.delivery_to_splunk_success count The sum of records that were successfully indexed over the total number of records that were attempted.
aws.kinesisfirehose.delivery_to_s3_success count The sum of the total number of Amazon S3 put commands over those that were successful.
aws.kinesisfirehose.backup_to_s3_bytes count The number of data sent to Amazon S3 for backup during the predetermined time frame.
aws.kinesisfirehose.backup_to_s3_data_freshness seconds Age of the oldest record in Kinesis Data Firehose, measured from its entry to the present.
aws.kinesisfirehose.delivery_to_http_endpoint_bytes bytes The number of data successfully sent to the HTTP endpoint in bytes.
aws.kinesisfirehose.delivery_to_http_endpoint_records Counts The number of records that were successfully delivered to the HTTP endpoint.
aws.kinesisfirehose.delivery_to_http_endpoint_data_freshness seconds Age of the oldest Kinesis Data Firehose record.
aws.kinesisfirehose.delivery_to_http_endpoint_success count The sum of all successful requests for data delivery to the HTTP endpoint.
aws.kinesisfirehose.delivery_to_http_endpoint_processed_bytes count The total amount of processed bytes that were attempted, including retries.
aws.kinesisfirehose.delivery_to_http_endpoint_processed_records count The total number of tries, including retries.
aws.kinesisfirehose.data_read_from_kinesis_stream_bytes bytes The number of bytes read from a Kinesis data stream when that is the data source.
aws.kinesisfirehose.data_read_from_kinesis_stream_records count The number records were read from a Kinesis data stream when that was the data source.
aws.kinesisfirehose.throttled_describe_stream count The total number of times the DescribeStream operation has been slowed down when a Kinesis data stream is the data source.
aws.kinesisfirehose.throttled_get_records count The total number of times the GetRecords operation is slowed down when a Kinesis data stream is used as the data source.
aws.kinesisfirehose.throttled_get_shard_iterator count The total number of times the GetShardIterator operation is slowed down when a Kinesis data stream is used as the data source.
aws.kinesisfirehose.bytes_per_second_limit count The number of current top speed at which a delivery stream can ingest data before throttling.
aws.kinesisfirehose.incoming_bytes bytes The total number of bytes successfully ingested into the delivery stream over the predetermined time frame following throttling.
aws.kinesisfirehose.incoming_put_requests count The number of PutRecord and PutRecordBatch requests that were successful over a given time frame.
aws.kinesisfirehose.incoming_records count The total number of records that were successfully ingested into the delivery stream after throttling during the specified time frame.
aws.kinesisfirehose.kinesis_millis_behind_latest millisecond The number of milliseconds the most recent record in the Kinesis data stream is behind the most recently read record when the data source is a Kinesis data stream.
aws.kinesisfirehose.records_per_second_limit count The number of maximum records per second that a delivery stream can currently ingest before being throttled.
aws.kinesisfirehose.throttled_records count The number of records that were throttled because one of the delivery stream limits was reached during data import.
aws.kinesisfirehose.describe_delivery_stream_latency milliseconds The latency required for each DescribeDeliveryStream operation over the predetermined time frame.
aws.kinesisfirehose.describe_delivery_stream_requests count Total requests for DescribeDeliveryStream.
aws.kinesisfirehose.list_delivery_streams_latency milliseconds The latency required for each ListDeliveryStream operation, calculated over the given time frame.
aws.kinesisfirehose.list_delivery_streams_requests count The total number of ListFirehose requests.
aws.kinesisfirehose.put_record_bytes bytes The number of data that was sent via PutRecord over the chosen time frame to the Kinesis Data Firehose delivery stream in bytes.
aws.kinesisfirehose.put_record_latency milliseconds The number of time required for each PutRecord operation, calculated over the given time frame.
aws.kinesisfirehose.put_record_requests count The sum of all PutRecord requests, which is the same as the sum of all records produced by PutRecord operations.
aws.kinesisfirehose.put_record_batch_bytes bytes The number of data sent via PutRecordBatch over the designated time period to the Kinesis Data Firehose delivery stream.
aws.kinesisfirehose.put_record_batch_latency milliseconds The latency required for each PutRecordBatch operation, calculated over the given time frame.
aws.kinesisfirehose.put_record_batch_records count The total number of records from PutRecordBatch operations.
aws.kinesisfirehose.put_record_batch_requests count The total number of PutRecordBatch requests.
aws.kinesisfirehose.put_requests_per_second_limit count The highest rate at which a delivery stream can accept put requests before throttling.
aws.kinesisfirehose.update_delivery_stream_latency milliseconds The latency required for each UpdateDeliveryStream operation, calculated over the given time frame.
aws.kinesisfirehose.update_delivery_stream_requests count The total number of UpdateDeliveryStream requests.
aws.kinesisfirehose.execute_processing_duration milliseconds The duration of each Kinesis Data Firehose call to a Lambda function.
aws.kinesisfirehose.execute_processing_success count The sum of the total number of successful Lambda function calls to the total number of unsuccessful calls.
aws.kinesisfirehose.succeed_processing_records count The total number of records that were successfully processed during the time frame specified.
aws.kinesisfirehose.succeed_processing_bytes bytes The total number of bytes that were successfully processed during the time frame.
aws.kinesisfirehose.succeed_conversion_records count The number of successfully converted records.
aws.kinesisfirehose.succeed_conversion_bytes bytes The size of the successfully converted records.
aws.kinesisfirehose.failed_conversion_records count The number of records that could not be converted.
aws.kinesisfirehose.failed_conversion_bytes bytes The size of the records that could not be converted.
aws.kinesisfirehose.kms_key_access_denied count The number of times the delivery stream's KMSAccessDeniedException is encountered by the service.
aws.kinesisfirehose.kms_key_disabled count The number of times the delivery stream's KMSDisabledException is encountered by the service.
aws.kinesisfirehose.kms_key_invalid_state count The number of times the delivery stream's KMSInvalidStateException is encountered by the service.
aws.kinesisfirehose.kms_key_not_found count The number of KMSNotFoundExceptions that the delivery stream's service encounters.