View service hierarchy and performance

Service diagrams provide an aggregated view of span data across your system. They are drawn based on a sample of up to 1000 traces that match your query and selected time period.

Service diagrams provide a clear visualization of inter-service relationships and insight into the performance of distributed software. You can see services both upstream and downstream from the queried service and pinpoint services or operations that contribute to the latency of the request. They also allow you to easily visualize a complex system architecture, identify services or operations with errors, and quickly formulate or eliminate hypotheses.

Service diagrams are useful in the following situations:

  • Identifying relationships between services
  • Visualizing error, rate, and latency contribution in your system
  • Understanding how those issues are affecting services up and downstream
  • Pinpointing the source of those issues

Service diagram tab shows service relationships and performance.

Service diagrams can also display inferred services. Inferred services are external services, libraries, or dependencies that haven’t been instrumented with traces, like a database or a third-party API. Cloud Observability checks leaf spans (spans that don’t connect to another instrumented service) against any inferred service rules you’ve configured. When Cloud Observability infers a service from those rules, it reports the that service’s error counts, span counts, and average latencies.

View the Service diagram

You access the Service diagram by clicking the Service Diagram tab in Explorer.

Each node on the diagram represents a service in the path. Each path shows the edge between two nodes.

The following annotations are used to describe system behavior:

  • Orange-to-dark-red circles: Show error percentages; darker shades indicate higher error contributions.
  • Yellow halos: Indicate contribution to the p50 latency; larger halos mean a higher contribution of latency to the critical path of the trace.
  • Gray shading: Depicts the traffic between nodes (edges); wider shading represents higher traffic.
  • Blue halos: Represent inferred services.

Hover over a node to view performance details for that node in a popup. Click View time series charts in notebook to add time series charts for the selected node to a notebook.

Animations show the request paths from the edges on that node to other nodes.

Hover over a path to view performance information for that part of the request.

Click the center of a service to select it and see its information in the side panel. The panel shows operations on the currently selected service with exemplar spans. You can click a span to see it in context in the Trace view.

For inferred services, the panel shows details for the service.

Annotated map points to the circles, halos, and shading described above.

Service diagrams refresh every 5 minutes.

Edges represent relationships between spans and is dependent on the quality of your instrumentation. Missing spans may cause edges to appear when there is no parent-child relationship.
The following can cause missing spans:
* The Collector or Microsatellite dropped the span (for example, your Microsatellite pool is not auto-scaling to keep up with traffic)
* The tracer dropped the span (for example, your application crashed or never called span.flush() )
* There is an issue with your instrumentation (context was dropped, or the service with a parent span is not instrumented).

When you see missing spans, check the Reporting Status page to find the culprit.

Change the Service diagram’s display

You can change how the service diagram is displayed.

  • Use the controls to:
    • Zoom in and out (you can also use your mouse or track pad to zoom)
    • Expand the diagram to fit the current container
    • Open the diagram in full-screen mode
    • Center in on a selected service
  • Select and center on a service to populate the side panel with information about that service.
  • Select a node and click Focus on service to see just the immediate upstream and downstream services.
  • Move the diagram by clicking and dragging.
  • Use the switch to turn the display of inferred services off and on.

Filter results

Similar to how you can filter the spans from the Trace Analysis table, you can filter spans from the side panel of the Service diagram.

When you click a filter icon on a service or operation, the Trace Analysis table repopulates to show spans from the results that match the filter. Results are taken from all spans that participate in the same traces as the original query.

You can’t filter on inferred services.

See also

Service health

Query real-time span data

Monitor Microsatellites, tracers, and service reporting

Updated May 8, 2024