Your dashboards can contain charts that display your metric and span data. Each chart represents a query against that data.

Using Terraform? You can use the Lightstep Terraform provider to create and manage your dashboards and charts. You can also use it to export existing dashboards into the Terraform format.

Before you create charts, read how Lightstep displays metric data to understand the different metric kinds and how your configuration choices affect how your data is displayed.

Create a chart

You create charts by querying your metric or span data. You can add multiple queries and also add formulas.

  1. From an open dashboard, click Add a Chart. Add a chart

  2. In the Chart Builder window, click into the title to name your chart. You can rename it at anytime by clicking into the title. Enter chart name

  3. Build your query using the Unified Query Builder.

  4. Click Save to save your changes. The chart now displays on the dashboard. New chart on dashboard

We cap the number of time series displayed to 1,000 to avoid unnecessary noise. If your query returned more than that, note that some were omitted. If you want to view all series, you may want to omit using “group by” to reduce the cardinality.

Data retention

Data for metric queries is saved for 13 months, and is not configurable.

By default, the span data returned by a query for a dashboard chart is continuously collected and saved as a Stream. The Stream’s data and and any associated traces are retained for as long as your data retention policy.

Instead, you can choose to only display data from the Microsatellites’ retention window, (by default, the last three days of data).

If data is beyond the retention window, the chart displays hashed lines to show that data is no longer available. Beyond retention window

To control retention of a span query:

By default, span queries are retained and saved as Streams. You can unselect the Create Stream checkbox to keep the data from being retained. Retained span query

Span queries that use a group-by or that use regex or Contains and Does not contain, can’t be retained.

If you chose to not retain a span query when you created the chart, you can change it later by clicking Edit. Edit to retain a query

If you choose to retain a query and later want it to use only what is in the retention window, you can delete its underlying Stream.

When a chart uses multiple queries, each query is retained separately.

If you add a chart with a retained Stream query to a dashboard that uses global filters, the query is retained without those filters applied.

Change a chart’s display

You can change the time period that a chart reports for and for metric data, also change the type of chart to use.

Change the time range

By default, the data shown is from the last hour. You can change that to a number of different time periods using the dropdown. Use the < > controls to move backwards and forwards through time. Change the time range

If data is beyond the retention window, the chart displays hashed lines to show that data is no longer available. Beyond retention window

Changing the time window may affect the alignment of the data points and show different results.

Change the chart type

By default, line charts are used to display values. You can change the chart type using the View as dropdown.

Depending on the metric kind or type, only certain chart types may be available.

  • Line: Available for all charts
    Displays a line that connects the charted values.Line chart

  • Heatmap Available for span latency and distribution-type metric charts in dashboards and notebooks. Multiple queries are not supported.
    Heatmaps allow you to see the distribution of values over time. Unlike a line chart, you can see the number of data points for a particular value, at a particular point in time. Color saturation in the map represents the amount of data points - the darker the color, the more data points for that value, at the point in time.

    For example, in the following heatmap, you can see that there are 300 points with a value between 500ms and 600ms around 9:10 am. Heatmap with many data points

    But starting at around 9:18 am, the distribution of data points shifts. There are now a number of points in the higher range of latency and fewer data points at the lower range (many of the dark blue cells changed to a lighter blue). The legend on the right shows the range of color values. Heatmap change

  • Bar: Available for metric charts in dashboards and notebooks
    Displays a bar for the value at a given time point. Bar charts are good for displaying discrete values, like the number of requests.Bar chart

  • Area: Available for metric charts in dashboards and notebooks
    Displays a shaded area for the value. Area charts are helpful when you’re charting more than one metric.Area chart

  • Big number: Available for metric charts dashboards and notebooks
    Displays an aggregated value over time as a number. Big number charts are useful when you only need to know the current value of a metric and don’t expect to investigate any changes. When creating a big number chart, note the following:

    • To determine the number to display, select an aggregation method from the dropdown and a time period for that aggregation. For example, if you want to show the maximum value over the last four hours, you’d select maximum value and The Past 4 Hours.

      This aggregation is different than the aggregation used when building the query. For a query, the data is aggregated to determine the value for each data point in a time series. For a big number chart, those values are aggregated a second time to determine the final value to display.

    • Click under the number to add a subtitle.
    • Because a big number chart displays a single value, you can not group by an attribute. If you use more than one metric for your chart, you need to combine them using a formula. Big number chart

Delete a chart

To delete a chart, use the More ( ⋮ ) icon and choose Delete. Delete a chart

Edit a chart

You access and edit a chart from the Dashboard.

  1. From the navigation bar, click Dashboards and open the dashboard your chart is on. Dashboard list view

  2. Hover over the chart, click the More ( ⋮ ) icon and select Edit. Edit icon for a chart The chart opens in the builder where you can edit it.

Add a chart to a notebook

You can add a chart to a notebook for when, during an investigation, you want to be able to run ad hoc queries, take notes, and save your analysis for use in postmortems or runbooks. Notebooks allow you to view metric and trace data from different places in Lightstep Observability together, in one place. Notebook

To add to a notebook, click Add to notebook and search to choose an existing notebook or create a new notebook.

Add chart to notebook

When you add to a notebook, a chart is created using the same query. The annotation is a link back to the original, so you can quickly return to the origin of your investigation.

Add chart to notebook

Learn more about notebooks.

Investigate a deviation in the chart

When you notice a deviation in a chart and don’t want to start a notebook, you can start your investigation directly from the chart. Metric charts use Change Intelligence. Span charts allow you to view exemplar traces and span data.

Investigate a metric deviation

Once you have the chart open in the editor, you can use Change Intelligence to determine what cased the change in performance. Change Intelligence links metric data with trace data to find components in your system whose performance changed at the same time as the change, allowing you to find the root cause without leaving Lightstep Observability.

You can’t use Change Intelligence with big number charts.

Change Intelligence

If you’ve made any edits to the chart, you need to save those changes before using Change Intelligence.

Investigate a span data issue

By default, the chart shows lines for each series (group-by), and dots for sampled spans. Triangles represent spans that have errors. Charted query

Use the Show span samples toggle to turn these off. Turn sample spans off

With spans turned off, when you hover over a line, you can see it’s value and percentile. Span data

With span samples displayed, when you hover over a point, you can see the value at the point along with the group-by value. View span data

Clicking the point takes you to its full trace, where you can view the full request path that the span participated in. In this case, the error is coming from the GET operation on the store-server service. Full trace

Below the chart, a table displays the data for each line in the chart.

Table displays data

With sample spans displayed, the Value column shows the latest value for that series.

You can collapse both the query and the table.