Once you’ve integrated with AWS CloudWatch, you have access to all metrics for the AWS Glue,which is a service that helps users integrate data from multiple sources, making it easier for them to analyze, prepare, and move that data. It can be used for analytics, machine learning, and app development. It also comes with tools for managing data operations and creating and running jobs, as well as for implementing business processes.

All available AWS integrations

To verify metrics are reporting, search for the metrics in the Metric details section of the Project Settings page.

The following table shows the Glue metrics ingested by Lightstep.

Metric Name Unit Description
aws.glue.glue_driver_aggregate_bytes_read bytes The total number of bytes read from all data sources by completed Spark tasks on all executors.
aws.glue.glue_driver_aggregate_elapsed_time milliseconds The total time in milliseconds for ETL processing (does not include job bootstrap time).
aws.glue.glue_driver_aggregate_num_completed_stages count The number of completed stages in the job.
aws.glue.glue_driver_aggregate_num_completed_tasks count The number of completed tasks in the job.
aws.glue.glue_driver_aggregate_num_failed_tasks count The number of failed tasks.
aws.glue.glue_driver_aggregate_num_killed_tasks count The number of terminated tasks.
aws.glue.glue_driver_aggregate_records_read count The total number of records read from all data sources by completed Spark tasks on all executors.
aws.glue.glue_driver_aggregate_shuffle_bytes_written bytes The total number of bytes written by all executors for shuffling data between them in the past minute.
aws.glue.glue_driver_aggregate_shuffle_local_bytes_read bytes The total number of bytes read by all executors for shuffling data between them in the past minute.
aws.glue.glue_driver_block_manager_disk_disk_space_used_mb megabytes The total number of megabytes of disk space used by all executors.