Once you’ve integrated with AWS Cloudwatch, you have access to metrics from AWS Kinesis Data Firehose, which is a completely managed service for sending real-time streaming data to locations like Cloud Observability, AWS data storage services, and other monitoring platforms.
See all AWS integrations.
To verify metrics are reporting, search for the metrics in the Metric details section of the Settings page.
The following table shows the Kinesis Data Firehose metrics ingested by Cloud Observability.
| Metric Name | Unit | Description |
|---|---|---|
| aws.kinesisfirehose.partition_count | count | The number of active partitions, or the number of partitions that are currently being processed. |
| aws.kinesisfirehose.partition_count_exceeded | count | The number of partition count cap gone over. |
| aws.kinesisfirehose.jq_processing_duration | milliseconds | The JQ Lambda function's return value is the time it took to execute a JQ expression. |
| aws.kinesisfirehose.per_partition_throughput | bytes | The number of bytes throughput is being processed for each partition. |
| aws.kinesisfirehose.delivery_to_s3_object_count | count | The number of items are being delivered to your S3 bucket. |
| aws.kinesisfirehose.delivery_to_amazon_open_search_service_bytes | bytes | The number of bytes indexed to OpenSearch Service during the defined time frame. |
| aws.kinesisfirehose.delivery_to_amazon_open_search_service_data_freshness | seconds | The age of the oldest record in Kinesis Data Firehose, measured from its entry to the present. |
| aws.kinesisfirehose.delivery_to_amazon_open_search_service_records | count | The total number of documents that OpenSearch Service has indexed during the time frame. |
| aws.kinesisfirehose.delivery_to_amazon_open_search_service_success | count | The total of records that were successfully indexed over the total number of records that were attempted. |
| aws.kinesisfirehose.delivery_to_redshift_bytes | count | The number of data copied to Amazon Redshift during the time frame specified. |
| aws.kinesisfirehose.delivery_to_redshift_records | count | The number of data copied to Amazon Redshift during the time frame. |
| aws.kinesisfirehose.delivery_to_redshift_success | count | The sum of all Amazon Redshift COPY commands divided by the total number of successful COPY commands. |
| aws.kinesisfirehose.delivery_to_s3_bytes | bytes | The number of bytes sent to Amazon S3 during the defined time frame. |
| aws.kinesisfirehose.backup_to_s3_records | count | The total number of records sent to Amazon S3 for backup during the designated time frame. |
| aws.kinesisfirehose.delivery_to_s3_data_freshness | seconds | The age of the oldest record in Kinesis Data Firehose, measured from its entry to the present. |
| aws.kinesisfirehose.delivery_to_s3_records | count | The number of records sent to Amazon S3 during the time frame specified. |
| aws.kinesisfirehose.backup_to_s3_success | count | The sum of all Amazon S3 backup put commands is greater than the sum of the Amazon S3 put commands that were successful. |
| aws.kinesisfirehose.delivery_to_splunk_bytes | bytes | The number of data sent to Splunk during the designated time frame. |
| aws.kinesisfirehose.delivery_to_splunk_data_ack_latency | seconds | The latency it takes for Splunk to acknowledge receiving data from Kinesis Data Firehose. |
| aws.kinesisfirehose.delivery_to_splunk_data_freshness | seconds | Age of the oldest record in Kinesis Data Firehose, measured from its entry to the present. |
| aws.kinesisfirehose.delivery_to_splunk_records | count | The number of records sent to Splunk during the designated time frame. |
| aws.kinesisfirehose.delivery_to_splunk_success | count | The sum of records that were successfully indexed over the total number of records that were attempted. |
| aws.kinesisfirehose.delivery_to_s3_success | count | The sum of the total number of Amazon S3 put commands over those that were successful. |
| aws.kinesisfirehose.backup_to_s3_bytes | count | The number of data sent to Amazon S3 for backup during the predetermined time frame. |
| aws.kinesisfirehose.backup_to_s3_data_freshness | seconds | Age of the oldest record in Kinesis Data Firehose, measured from its entry to the present. |
| aws.kinesisfirehose.delivery_to_http_endpoint_bytes | bytes | The number of data successfully sent to the HTTP endpoint in bytes. |
| aws.kinesisfirehose.delivery_to_http_endpoint_records | Counts | The number of records that were successfully delivered to the HTTP endpoint. |
| aws.kinesisfirehose.delivery_to_http_endpoint_data_freshness | seconds | Age of the oldest Kinesis Data Firehose record. |
| aws.kinesisfirehose.delivery_to_http_endpoint_success | count | The sum of all successful requests for data delivery to the HTTP endpoint. |
| aws.kinesisfirehose.delivery_to_http_endpoint_processed_bytes | count | The total amount of processed bytes that were attempted, including retries. |
| aws.kinesisfirehose.delivery_to_http_endpoint_processed_records | count | The total number of tries, including retries. |
| aws.kinesisfirehose.data_read_from_kinesis_stream_bytes | bytes | The number of bytes read from a Kinesis data stream when that is the data source. |
| aws.kinesisfirehose.data_read_from_kinesis_stream_records | count | The number records were read from a Kinesis data stream when that was the data source. |
| aws.kinesisfirehose.throttled_describe_stream | count | The total number of times the DescribeStream operation has been slowed down when a Kinesis data stream is the data source. |
| aws.kinesisfirehose.throttled_get_records | count | The total number of times the GetRecords operation is slowed down when a Kinesis data stream is used as the data source. |
| aws.kinesisfirehose.throttled_get_shard_iterator | count | The total number of times the GetShardIterator operation is slowed down when a Kinesis data stream is used as the data source. |
| aws.kinesisfirehose.bytes_per_second_limit | count | The number of current top speed at which a delivery stream can ingest data before throttling. |
| aws.kinesisfirehose.incoming_bytes | bytes | The total number of bytes successfully ingested into the delivery stream over the predetermined time frame following throttling. |
| aws.kinesisfirehose.incoming_put_requests | count | The number of PutRecord and PutRecordBatch requests that were successful over a given time frame. |
| aws.kinesisfirehose.incoming_records | count | The total number of records that were successfully ingested into the delivery stream after throttling during the specified time frame. |
| aws.kinesisfirehose.kinesis_millis_behind_latest | millisecond | The number of milliseconds the most recent record in the Kinesis data stream is behind the most recently read record when the data source is a Kinesis data stream. |
| aws.kinesisfirehose.records_per_second_limit | count | The number of maximum records per second that a delivery stream can currently ingest before being throttled. |
| aws.kinesisfirehose.throttled_records | count | The number of records that were throttled because one of the delivery stream limits was reached during data import. |
| aws.kinesisfirehose.describe_delivery_stream_latency | milliseconds | The latency required for each DescribeDeliveryStream operation over the predetermined time frame. |
| aws.kinesisfirehose.describe_delivery_stream_requests | count | Total requests for DescribeDeliveryStream. |
| aws.kinesisfirehose.list_delivery_streams_latency | milliseconds | The latency required for each ListDeliveryStream operation, calculated over the given time frame. |
| aws.kinesisfirehose.list_delivery_streams_requests | count | The total number of ListFirehose requests. |
| aws.kinesisfirehose.put_record_bytes | bytes | The number of data that was sent via PutRecord over the chosen time frame to the Kinesis Data Firehose delivery stream in bytes. |
| aws.kinesisfirehose.put_record_latency | milliseconds | The number of time required for each PutRecord operation, calculated over the given time frame. |
| aws.kinesisfirehose.put_record_requests | count | The sum of all PutRecord requests, which is the same as the sum of all records produced by PutRecord operations. |
| aws.kinesisfirehose.put_record_batch_bytes | bytes | The number of data sent via PutRecordBatch over the designated time period to the Kinesis Data Firehose delivery stream. |
| aws.kinesisfirehose.put_record_batch_latency | milliseconds | The latency required for each PutRecordBatch operation, calculated over the given time frame. |
| aws.kinesisfirehose.put_record_batch_records | count | The total number of records from PutRecordBatch operations. |
| aws.kinesisfirehose.put_record_batch_requests | count | The total number of PutRecordBatch requests. |
| aws.kinesisfirehose.put_requests_per_second_limit | count | The highest rate at which a delivery stream can accept put requests before throttling. |
| aws.kinesisfirehose.update_delivery_stream_latency | milliseconds | The latency required for each UpdateDeliveryStream operation, calculated over the given time frame. |
| aws.kinesisfirehose.update_delivery_stream_requests | count | The total number of UpdateDeliveryStream requests. |
| aws.kinesisfirehose.execute_processing_duration | milliseconds | The duration of each Kinesis Data Firehose call to a Lambda function. |
| aws.kinesisfirehose.execute_processing_success | count | The sum of the total number of successful Lambda function calls to the total number of unsuccessful calls. |
| aws.kinesisfirehose.succeed_processing_records | count | The total number of records that were successfully processed during the time frame specified. |
| aws.kinesisfirehose.succeed_processing_bytes | bytes | The total number of bytes that were successfully processed during the time frame. |
| aws.kinesisfirehose.succeed_conversion_records | count | The number of successfully converted records. |
| aws.kinesisfirehose.succeed_conversion_bytes | bytes | The size of the successfully converted records. |
| aws.kinesisfirehose.failed_conversion_records | count | The number of records that could not be converted. |
| aws.kinesisfirehose.failed_conversion_bytes | bytes | The size of the records that could not be converted. |
| aws.kinesisfirehose.kms_key_access_denied | count | The number of times the delivery stream's KMSAccessDeniedException is encountered by the service. |
| aws.kinesisfirehose.kms_key_disabled | count | The number of times the delivery stream's KMSDisabledException is encountered by the service. |
| aws.kinesisfirehose.kms_key_invalid_state | count | The number of times the delivery stream's KMSInvalidStateException is encountered by the service. |
| aws.kinesisfirehose.kms_key_not_found | count | The number of KMSNotFoundExceptions that the delivery stream's service encounters. |
See also
Updated Jan 13, 2023