When you are in a performance investigation, you often need to run ad hoc queries to see the performance of your system, both at the infrastructure level and app level, and then share your findings with your team. Instead of using multiple tabs or tools, and creating “throw-away” dashboards, Lightstep Observability provides notebooks that allow you to query and save both your infrastructure (metrics) and app performance (traces) data in one place. You can also add charts from other pages in Lightstep Observability. You can add text blocks to note and explain your findings and share your notebook (either a live version or a snapshot in time) with anyone with a Lightstep Observability account.

Notebooks require that your environment uses Microsatellites.

Notebooks view

You use the Unified Query Builder to create and filter charts for your notebook for both metric and trace data. From metric charts, you can run Change Intelligence directly from a notebook to begin your investigation.Run Change Intelligence

From trace data charts, you can view example traces that have high latency, error, or operation rate. View exemplar traces

To view a trace, make sure the Show span samples toggle is on. Spans with errors are shown as a triangle. Spans without errors are shown as dots. Click on either one to view the associated trace.

When to use notebooks

Notebooks are helpful anytime during an investigation when you want to create ad-hoc queries, especially when you want to query both metric and trace data. They are also useful when you want a record of your investigation for postmortems or runbooks.

Here are some examples of how you might use notebooks:

  • Ad-hoc queries: You can build ad-hoc queries in a notebook and compare the charts and run Change Intelligence or view traces from there.
  • Collaboration: When your team needs to collaborate during an investigation, you can each add your own charts and text blocks to the notebook and work with the same data.

    When someone adds a chart or text block to a notebook, you will need to refresh your page to see any additions/changes. If you’ve shared your notebook and know that someone else may be editing it, best practice is to refresh the page after each change you make to ensure you don’t overwrite each other’s edits.

  • Postmortems and runbooks: When you use notebooks during an investigation, you can add your steps to resolution to the notebook once you’ve resolved the issue.

Users with Viewer permissions can create, view, and copy any notebook, but can only edit and delete their own.

Data retention in notebooks

The data for metric-based charts is stored for 13 months and for your trace-based charts, data is kept live for three days and then auto-saved, so once you mitigate an issue, you can go back to the data you worked with to fully investigate and remediate.

Auto-saved span data in notebooks

By default you can explore up to the past three days of span data. But because your notebooks become useful tools in rituals like post-mortems, you often want to have a record of the data you worked with as you investigated an issue. Lightstep automatically saves the data from when you created the chart, so you can continue to view the chart and click on an exemplar (a dot or triangle) to view the associated trace. To see more recent data, you can rerun the query on a new chart by clicking Duplicate with latest data.

If you try to change the query once it’s auto-saved, you see a warning stating that you will lose data if you save the change. Data loss will happen because the data that was auto-saved is based on the retention window from the original query. Changing the query requires gathering new data which will overwrite the original. Instead, you can duplicate the original query to a new chart using data from the current retention window and edit that instead.

The auto-saved version is retained for as long as your data retention policy.

Notebook snapshots

Additionally, sharing information becomes important during an investigation. You can share a read-only snapshot of a notebook at any point in time and that snapshot is also saved for the period of your data retention. Read more about sharing notebooks.

Work with notebooks

You access notebooks from the Notebook icon in the navigation bar. Notebook icon in nav bar

Existing notebooks are listed in the sidebar. Any notebooks that you’ve favorited are shown in the first section. Notebooks that you’ve created are in the second section, and all other notebooks are listed in the last section, in the order they were created (latest on top). You can see when the notebook was last edited and by who.

You can use the Search field to find a notebook by searching on name or user. Search notebooks

You can favorite a notebook by clicking the star. Your favorite notebooks are moved to Starred notebooks section of the sidebar. Favorite a notebook

Create a notebook

You can create a notebook from the Notebooks view and then query metric or trace data to add charts.

You can also add charts from existing charts, queries, or time series.

To create a notebook:

  1. From the navigation bar, click Notebooks.

    Existing notebooks for the projects are shown on the left. Notebook icon in nav bar

    A new (unsaved) notebook shows the query builder. New notebook

    If an existing notebook is displayed, click Create notebook from the sidebar. Create notebook

  2. Click into the fields to add a title and an optional description for the notebook. Notebook title

    If you don’t add a title, Lightstep Observability automatically names the notebook after the query, along with a timestamp. Auto title Click Save notebook. Save notebook

    You only need to explicitly save a notebook when you create it. All changes are auto-saved after that point.

You can now add a metric or span chart, text blocks, and share the notebook.

Add charts to a notebook

You can add charts based on queries to both metric and span data. You can also add charts from other parts of Lightstep Observability where you may have started your investigation.

Add a chart

  1. If this is the first chart in the notebook, the query builder is open.

    If you’re adding a subsequent chart, to open the query builder click the + icon and choose Build another chart. Add chart

  2. Build a metric query or a span query.

You can add another query to the chart and you can use formulas to perform arithmetic on the time series.

Add an existing chart, query, or time series

You can add charts from these places where you may have started your investigation:

  • A chart on a dashboard.
  • A metric chart on an alert.
  • An Explorer query.
  • A time series chart from the Deployments tab in the Service Directory.
  • An operation’s overview in the Service Directory. This creates a chart for that operation’s performance.
  • A Stream listed for a service in the Service Directory. This creates a chart using the same query as the Stream.

When you add an existing query to a notebook, a chart is created using the same query. The annotation is a link back to the original, so you can quickly return to the origin of your investigation. Explorer query in a notebook

Work with charts

  • You can change the query for any chart.

    Edit history for notebooks is not saved, so you can’t “undo” a change. If you think you might want to save a query but also want to try variations of the query, best practice is to duplicate the chart first (see bullet below).

  • The time range for charts imported from dashboards or alerts uses the time range from the original chart. Charts added directly to the notebook use the same time range as the chart directly above the new chart.

    You can change the time range using the time picker. When you change the range, you can apply it to just that chart or to all charts in the notebook (the default). Change time range

    When you apply a new time range, the time window is fixed and does not continuously update.
    When you use the < > controls to move backward and forward through time, the time range changes for only that chart.

  • Delete a chart clicking the More ( ⋮ ) icon Delete a chart
  • Duplicate a chart clicking the More ( ⋮ ) icon Duplicate a chart

    The chart title is automatically populated with Copy of [Chart title]

Add text blocks

You can add text blocks to enhance your investigation.

Users with Viewer permissions can edit only their own notebooks.

To add a text block, click the plus icon above or below an existing notebook entry. Add a chart or text block

Work with text blocks

  • Text blocks are saved as plain text.
  • URLs resolve to clickable links.
  • Duplicate a text block by clicking the More ( ⋮ ) icon. Duplicate a note
  • Delete the contents of a text block by clicking the X icon. Delete note's contents
  • Delete the entire text block by clicking the More ( ⋮ ) icon Delete a note

Duplicate a notebook

You can copy any existing notebook to start a new one.

To copy a notebook, click the More ( ⋮ ) icon and choose Duplicate notebook. Duplicate notebook

The notebook is saved as Copy of [Notebook title]. Click into the field to edit it.

Delete a notebook

You can delete a notebook. When you do, all included charts and text blocks are also deleted.

Users with Viewer permissions can delete only their own notebooks.

To delete a notebook, click the More ( ⋮ ) icon and choose Delete. Delete notebook

Share a notebook

You can share a notebook with any other Lightstep Observability user in your organization. You can share a live version of the notebook and authorized users can edit the notebook, or you can create a read-only snapshot of the notebook to share.

Share a live notebook

When you share a live notebook, users with Admin or Member roles can edit the notebook, including changing queries, time ranges, and text blocks (users with Viewer roles will see a read-only version).

To share a live notebook, click the Share button, select Share this notebook and copy the link. Share a notebook

When someone adds a chart or text block to a notebook, you will need to refresh your page to see any additions/changes. If you’ve shared your notebook and know that someone else may be editing it, best practice is to refresh the page after each change you make to ensure you don’t overwrite each other’s edits.

Share a snapshot

When you don’t want other users to be able to edit the chart (or if you want to save off a notebook at a certain point in time), you can create a snapshot. The snapshot is saved for the length of your data retention policy.

Read only notebook

To share a snapshot, click the Share button, select Share a read-only snapshot, and copy the link. Share a snapshot

Once you create a snapshot, you can access it from the Share button. Snapshot list