This page describes how to manage and work with Lightstep alerts. Read the sections below to learn how to do the following:

  • View alerts and their status.
  • Run Change Intelligence on alerts.
  • Delete and snooze alerts.

View alerts

You can get a quick overview of alerts, and if they are in compliance or not, from the Alerts page. Open the Alerts page by clicking Alerts from the navigation bar.

By default, the Alerts view lists all alerts by their status, with those in violation at the top. Alerts can have the following statuses:

  • Red/Critical: The alert is in violation of the Critical threshold.

  • Green/OK: The alert is active and is currently in a non-triggered state. All inbound traffic is being analyzed against the criteria you have set over the defined evaluation window.
  • Light gray/Unknown: The alert has been created but it’s not yet in an active monitoring state. If an alert remains light gray, either:
    • Refresh the page to get the alert’s latest status.
    • Click the alert and click Alert configuration. If Lightstep is waiting to activate the alert, the page shows this message: Your alert will activate in X days when Lightstep has enough data to run the alert query.
  • Dark gray/No data: There is no data reporting for this query.
  • Grayed out: The alert is active, but is currently in a snoozed state. This means that ingested traffic is being analyzed against the criteria defined in the alert, but alerts will not be generated if a threshold is breached.

You can filter alerts by name, status, date, or alert type.

Click on an alert to open it, view the associated data, edit it, and snooze it.

Add an alert’s chart to a notebook

You can add to a notebook for when, during an investigation, you want to be able to run ad hoc queries, take notes, and save your analysis for use in postmortems or runbooks. Notebooks allow you to view metric and trace data from different places in Lightstep Observability together, in one place. Notebook

To add to a notebook, click Add to notebook and search to choose an existing notebook or create a new notebook.

When you add to a notebook, a chart is created using the same query. The annotation is a link back to the original, so you can quickly return to the origin of your investigation.

Learn more about notebooks.

Delete an alert

To delete an alert, from the Alerts view use the gear icon to choose Delete.

Run Change Intelligence on an alert

Once you have the alert open in the editor, you can use Change Intelligence to help determine what caused the change in performance. For metric data, Change Intelligence looks at traces that include spans from key operations on the service that emitted the metric. For span data, it searches traces that match the chart’s query. In both cases, Change Intelligence surfaces the attributes that appeared in traces with performance issues occurring at the same time as the spike in the chart.

You can’t use Change Intelligence with big number charts.

Change Intelligence

To run Change Intelligence, click Analyze deviation or click directly in the chart and select Analyze deviation.

Change Intelligence opens in the side panel, where you can begin your investigation. You can also click View full Change Intelligence to navigate to the expanded view.

Follow these instructions to get started.

Snooze an alert

You can snooze an alert when needed, for example if you know a team is working on a fix and don’t need to be further notified.

To snooze an alert:

  1. From the Alerts view, click the alert to open it in the editor.

  2. Click Snooze, choose the amount of time to snooze the alert for, and click Save. Snooze an alert The alert now displays in the Alert view as snoozed. When you hover over the snooze icon, a tooltip displays the time when the alert will reactivate. Snoozed alert

To un-snooze an alert:

You remove a snooze by returning to the editor using the Snooze button to choose Off.