When your instrumentation includes attributes (key/value pairs that carry metadata descriptions about your spans), Lightstep Observability can use them to help quickly pinpoint root causes of issues. You can see when spans with certain attribute values have errors or latency, while others with different values don’t.
A span can have zero or more key/value attributes. Attributes allow you to create metadata about the span. For example, you might create attributes that hold a customer ID, or information about the environment that the request is operating in, or an app’s release. Attributes do not reflect any time-based events (log events handle events). The OpenTelemetry spec defines several standard attributes.
You can also implement your own attributes and events. The following are attributes and log events that work very well with Lightstep Observability in addition to the OpenTelemetry semantic conventions. Any attribute that you add to your span data will enable more segmentation, making it easier to find, filter, and group your span data in Lightstep Observability. Lightstep doesn’t have cardinality limitations, so the more attributes you use, the greater your insights will be.
Learn more about attributes.
Use the OpenTelemetry semantic attributes
Start with the OpenTelemetry semantic span attributes. These are standardized attributes that provide much of the functionality you’ll need.
Use attributes to identify issues related to your business values
Attributes help identify issues that are affecting business flows and assets that are important to you. For example, if your business’s number one concern is customers, then create an attribute to carry the customer’s name so you can quickly see when issues hit your highest priority customers.
Similarly, attributes that describe the device type or OS can help determine how widespread an issue is. Is it only a problem for customers on a particular combination of OS and device? Is that a large part of your customer base? Being able to make that determination quickly can help prioritize mitigation.
Catch system changes
Knowing immediately when a deploy affects your system, or when an a/b segment of your environment is affected means you can react quickly by starting a rollback or moving customers to a different segment.
When you use the service.version
attribute, Lightstep Observability automatically creates markers showing when deployments occurred. Here’s an example of increased latency, likely caused by the deployment immediately before it.
In this graphic, you can see that the attribute service.version
with a value of 10.8.585
has many errors during the regression time range.
When you filter on the service.version:10.8.585
attribute to see only spans with that attribute value, and then in the Trace Analysis table, group by the service
attribute, you can see which services were deployed with that version.
In this example, only spans on the store-server
service have the version value of 10.8.585
, which tells you that a deploy of that service is causing the errors. And you can look at the event log messages to see what those errors are.
Because of the service.version
and service
attributes, you were able to quickly pinpoint the source of errors!
Label infrastructure
Knowing where an issue is occurring, and just as important, ruling out where it’s not, can save a tremendous amount of time. In this example, you can see that the attribute db.type=cassandra
is on almost every span at the high-end of latency.
When you group the results by the db.type
attribute, you can see that the Cassandra database is experiencing much higher latency than others.
What did we learn?
-
Attributes are valuable when looking for root causes. Lightstep Observability can correlate attributes with error and latency rate, and you can group and filter by attributes to narrow down the data and find issues fast.
-
Using the
service.version
attribute allows Lightstep Observability to show you when deploys occur, letting you associate a deploy with a regression at a glance. -
You should create attributes related to valued business functionality so you can easily monitor flows and assets important to you.