Lightstep Observability accepts Prometheus Counters, Gauges, and Histograms through the OpenTelemetry Collector. However, there may be differences in how these metric instruments and measurements are displayed in Lightstep Observability as opposed to Prometheus. This document collects these differences for reference.
Prometheus cumulative counters
Lightstep Observability accepts a Counter, but for technical and philosophical reasons, we do not provide the underlying cumulative count to our graphs and instead plot them as deltas. The resolution of these deltas varies based on the age of the data - for example, 45 second buckets for data recorded in the past hour, 2 minute buckets for data recorded between one and four hours ago, and so forth.
For an example of how this might manifest, consider the promhttp_metric_handler_requests_total
counter that Prometheus exposes for itself (at the /metrics
endpoint). Assuming a regular scape cadence of 1 scrape per 10 seconds, graphing this metric in Prometheus would result in a plot of values increasing steadily, up and to the right. In Lightstep Observability, the same metric represented as a count would result in a straight line over time assuming a consistent rate of scrape requests.
Default grouping for rate calculations
When taking the rate
of a counter in Prometheus, the default plot will implicitly group the plot by attributes (tabs/labels). In Lightstep Observability, you must explicitly specify the attribute to group by.