You use the Trace view to see a full trace from beginning to end of a request. The Trace view shows you a flame graph of the full trace (each service a different color) and below that, each span is shown in a hierarchy, allowing you to see the parent-child relationship of all the spans in the trace. Errors are shown in red.
Clicking a span shows details in the right panel, based on the span’s metadata. Whenever you view a trace in Lightstep, it’s persisted for the length of your Data Retention policy so it can be bookmarked, shared, and reviewed by your team at any time. The selected span is part of the URL, so it will remain selected when the URL is visited.
When viewing a “live” trace, Lightstep view pulls span data from the Satellites` recall window and builds the trace as the data comes in (that’s why you’ll often see a notice about trace assembly as the data loads). Your view into the trace is compiled from real-time data, giving you an accurate view into a request. Traces built from historical data (such as Snapshots or links) load much quicker.
Access Trace View
You access a Trace view by clicking on a span from several places within Lightstep:
- Trace Analysis tabe in Explorer: Click a span in the table to open. The span is selected in the trace.
- Streams: Click a dot in the scatterplot to open the trace.
- Dashboards: Click a dot in the scatterplot to open the trace.
- Service diagram: Click an operation’s span in the left panel.
- Historical spans: Click Trace in the Historical Spans table to view a trace used in a Stream.
- Developer Mode: Click View Trace to see traces sent by the Developer Satellite.
- Alerts: Alerts include a link to a trace that is an exemplar of the current issue.
View a Timeline
At the top of the Trace view is a timeline for the entire trace, along with a timestamp. When you click on an individual span, a blue bar indicates where that span fits into the timeline and shows its start time.
View Trace Map
At the top of the Trace view, a map displays the overall trace. Services participating in the trace are displayed in different colors to give a sense of the overall service interactions. Each service is a different color, and you can click on a colored bar to select the corresponding span and see its details.
You can toggle the trace map between two modes. The “icicle graph” provides a view of the overall shape of the trace.
Click the Minimize/Maximize icon to view a single-line condensed view designed to take up minimal screen space.
Spans highlighted with a red dashed box denote missing spans; their parent span wasn’t found in the data from the Satellites.
Missing spans can happen when either the Satellite dropped the span (for example, your Satellite pool is not auto-scaling to keep up with traffic) or the tracer dropped the span (for example, your application crashed or never called
span.flush() ). When you see missing spans, check the Reporting Status page to find the culprit.
See the Critical Path
Lightstep automatically runs a heuristic to determine the critical path of a given trace. The critical path is highlighted in yellow to help identify bottlenecks in the overall transactions.
When errors are found on a span, Lightstep displays the span in red in the flame graph, and the operation and service name are also in red. The details panel shows that there’s an error, and logs should help determine the nature of the error.
Modify Display of Spans
The Trace view shows a hierarchical representation of spans in parent/child relationships in the order that they participated in a request. The number to the left of the span tells you how many children that span has. You can collapse all or part of the hierarchy, focus on just one span and its hierarchy, and filter to see only specific spans.
Collapse the Hierarchy
By default, the hierarchy is completely expanded, showing every trace.
You can collapse the hierarchy to the level of the selected span. Select a span and click the Collapse/Expand button at the top of the span table. Parent spans with collapsed children display a black box.
To expand the hierarchy to reshow all spans, re-click the Collapse/Expand button.
To collapse at a single point in the hierarchy, click on that span’s number box. Re-click to expand.
Focus on a Single Span’s Hierarchy
You can show just the spans in a sub-hierarchy to get a closer view. Select the span to focus on and click Focus on selected span. The span table filters to show only that hierarchy. Click Reset Focus to show the full span table.
Filter the Span Table
You can show only specific operations or services. Enter a name into the search bar and Lightstep filters the table.
View Details of a Span
The panel to the right of the span table shows details for the selected span.
Operation and Service
The top of the panel shows the operation and service names. The service name is in the color that its associated span is shown in. Click the clock icon to run a query in Explorer for that operation or service.
Workflow Links allow you to create dynamic links to information outside of this page, allowing access to all the info you need when you need it. For example, say you want to jump directly to logs from the time when an issue occurred or to quickly view your playbook’s instructions when the span includes a certain error code. Lightstep can construct these customized links automatically, using tags and other metadata from a span in the trace.
Logging information displays, based on your instrumentation. Payloads capture detailed information for transactions of interest.
The rest of the panel shows details captured from your instrumentation, such as span timestamps, latency, IDs, platform, and other tags you may have added. For tags, you can click the clock icon to run a query in Explorer for that tag. For the client library, you can click a link to visit that version’s GitHub page.
Share a Trace Preview in Slack
By default, you can share the URL for the trace and anyone who clicks the URL is taken to Lightstep to see that exact trace. But what if not everyone in your org is a Lightstep user? And while a URL is helpful, it’s not as helpful as being able to immediately show interesting information about that trace.
When you integrate Lightstep with Slack, you can share previews of the Trace View page in any channel of your workspace. Other Slack members can see information about the selected span, its parent span, critical path information, and errors. Lightstep users can click View Trace to jump right to that page.