Use dependency maps

You can use a dependency map to get an aggregate view of trace data as a request travels through your system. The map provides a visual, interactive, and hierarchical representation of a system’s behavior for a given point in time, based on a span query. You create a dependency map by querying on any span data (for example service, operation, or attribute key), and you can add filters to narrow in on operations or other attributes.

Dependency maps also provide a clear visualization of inter-service relationships and insight into the performance of distributed software. You can see services (and optionally operations) both upstream and downstream from the queried service and pinpoint services or operations that contribute to the latency of the request. Maps also allow you to easily visualize a complex system architecture, identify services or operations with errors, and quickly formulate or eliminate hypotheses.

You can also configure Lightstep to recognize and display inferred services in the map. Inferred services are external services, libraries, or dependencies that haven’t been instrumented, like a database or a third-party API. Lightstep recognizes these leaf spans (the request can’t continue to another service) and reports on their error counts, span counts, and average latencies.

The diagram edges represent relationships between spans and is dependent on the quality of your instrumentation. Missing spans may cause edges to appear when there is no parent-child relationship.
The following can cause missing spans:
* The Microsatellite dropped the span (for example, your Microsatellite pool is not auto-scaling to keep up with traffic)
* The tracer dropped the span (for example, your application crashed or never called span.flush() )
* There is an issue with your instrumentation (context was dropped, or the service with a parent span is not instrumented).

When you see missing spans, check the Reporting Status page to find the culprit.

In the map, services/operations that contribute to latency are shown with a yellow halo - the larger the halo, the larger the latency. Services/operations with errors are shown with a red halo. Inferred services have a light blue inner halo.

Dependency maps refresh every 5 minutes.

Add a dependency map

You can add a dependency map to a notebook or a dashboard.

1. Query your span data.

   Note the following:

   • Queries can be on services, operations, or attributes.
   • Queries can’t include an aggregation, group-by, or latency percentiles.
   • You can’t use more than one query or a formula

   You can also use the Editor and UQL to create a dependency map.

2. Select Dependency map from the View as dropdown. Lightstep builds the dependency map from sampled data on your spans. The span table below the map shows exemplar spans used to build the map. You can click a span row to open it in the Trace view

   Hover over a service or operation in the map to view information about it. Dotted lines show direct ancestors and dependencies

Change the dependency map display

This image shows how to change the map’s display. The steps below describe the procedure in more detail.

1. By default, the map shows relationships between services. You can change it to display operations by clicking the Operations button.

2. Also by default, the map shows all dependencies. Use the dropdown to view immediate, upstream, or downstream dependencies.

3. Zoom in and out or center in on a selected service.

4. Move the diagram by clicking and dragging.

5. Toggle the display of inferred services.

   Filter span table

You can filter the span table to show only spans from a selected service or operation. Click the node (selected nodes display a blue circle) and the table populates with spans only from the selection. Return to the full set of spans by clicking Remove filter.

You can’t filter on inferred services.

Add inferred services

Before inferred services display in the diagram, you need to add attributes to your instrumentation that tell Lightstep the request is going to an inferred service. You then need to add the attribute to Lightstep settings and set a display name.

For example, you might create (or have) an attribute that returns the database type for a span. You can use this attribute and its value to identify spans coming from specific types of databases. Lightstep Observability will collect information for any span that uses that attribute key/value pair. You might also use that attribute to display the database type, so that any database service is labeled with the database type value (for example, sql or cassandra).

You might also need to use different attributes for recognizing the inferred service and the display name for that service. For example, you might use span.type="sql" to tell Lightstep to collect data for any spans from any SQL database, and then use the attribute sequel.db.vendor to return the value and use that as the label in the display, for example, "MySQL" and DB2.

Inferred services must be a leaf node in the resulting trace - that is, it can’t call out to another service.

To add inferred services:

1. If you don’t already have one, add an attribute to your instrumentation that can identify an inferred service.
   For example, if a request is calling out to a SQL database, you might create an attribute with the name db.type that takes a value of sql when calling out to that database.

2. In Project settings, click Inferred services.

3. In the Identify inferred services field, start typing to select the attribute and value for the inferred service you want to add.

4. In the Label inferred service field, select the attribute whose value should be used as the display name.

For example, you might select db.type as the label, and db.type:"sql", db.type:"memcached" and db.type:"cassandra" as all the values to collect and display inferred service information for.