What are Streams and alerts?
The Pre-Deploy action relies on alerts on your Streams to determine if there are currently any issues in your system before you deploy. So you need to have those alerts and Streams set up before using the action.
Streams provide a time-series chart based on a query. You can query on a service, operation, and attributes. The chart shows you the latency, error rate, and operation rate for a time range, using a scatter plot to chart exemplar traces. Once you create a Stream, your Lightstep Microsatellites collect and persist the span data that matches your query.
Alerts are thresholds you set on a Stream that when crossed, trigger a notification. An example might be if latency for a stream in the 99th percentile reaches 100ms for over 2 minutes. For the Pre-Deploy action, you need to create alerts for anything you want to be warned about before you commit a pull request.
If you already have Streams and alerts, then you can skip to Step 2
Create a Stream
For this learning path, you’ll create one Stream. Remember for the action to be valuable, you’ll need to create Streams for anything that should keep you from deploying.
From the Stream list, click Create Stream.
Enter a query to base your Stream on and click Run. Lightstep Observability runs the query and returns example spans.
Learn how to run a query
To run a query using the Query Builder:
Click into the search bar to open the Query Builder. You can build queries for services, operations, and attributes. Use
NOT INto build the query. When you click into the Service or Operation field, the builder displays valid values.
When you add multiple values to the Operation field, spans that match either value (
ORoperation) are returned.
Learn more about running queries
Supported query keys
Query multiple keys and values
If the results are satisfactory, click Create stream. The Stream is added to the list of Streams. By default, the Stream’s name is the query. You can rename it.
To view the Stream, click VIEW STREAM in the banner or click its name in the list.
Lightstep Observability starts collecting data for Streams once you create them, so when you create a Stream, it will initially be blank.
Create an alert
Now you’ll create an alert that the action will use to determine if it’s safe to deploy.
Open a Stream either from Streams in the navigation bar or from a Dashboard.
Click Create Alert.
If you already have alerts on this stream, click Alerts to open the panel and click New.
Enter the following:
- Signal: Select the metric Lightstep Observability should analyze for this alert. Choose from Latency, Operation rate, or Error percentage.
- Name: Enter a name for this alert.
- Threshold: Create a threshold that should trigger the alert. Configuration for the threshold depends on the signal you chose.
- Latency: Lightstep Observability analyzes the latency in units of time (m, s, ms, for microseconds put a number < 1ms) for a certain percentile bucket, and determines if the latency during the evaluation window is above or below the threshold.
- Error Percentage: Lightstep Observability checks every ingested span for the presence of the OpenTelemetry
errorattribute. The error percentage signal is the percentage of spans in a Stream population that contain an error. Lightstep Observability can create an alert if the error percentage is above or below the threshold.
- Operation Rate: Lightstep Observability collects basic throughput information about Stream operations and determines if the rate is above or below that threshold.
- Evaluation Window: The period of time during which you want Lightstep Observability to perform the threshold calculations to determine if an alert should be created. This can be thought of as a moving time frame, trailing the current point in time, during which relative percentages are calculated for signals. Allowed units include: d (days), h (hours), m (minutes). The minimum value is 2 minutes.
Higher values can be useful if you’d like to “smooth” out the data to avoid alerts during brief metric spikes.
While you can click Create now, alerts won’t happen because you haven’t defined a notification destination. Continue on to create the actual alert.
You don’t have to create a notification destination because you won’t be using this alert to send a notification to a third-party app. You can use this same alert to do that if you want.
What did we learn?
- Streams proactively collect span data based on their query. They report on latency, error rate, and operation rate.
- Alerts set thresholds on a Stream that when crossed, provide a trigger. For the Pre-Deploy action, they let you know that it may not be safe to commit your pull request.