Workflow Links cookbook

This page illustrates common uses for Workflow Links. Each scenario includes a template, followed by an example with real values.

You can choose many of these recipes as templates when you create your links in the Cloud Observability UI. These recipes are noted below.

For recipes not included as templates in Cloud Observability, you can copy these examples directly into Cloud Observability to create your own links.

Tips:

  • Replace values in ALL CAPS with your own text.
  • Values in {{brackets}} are variables. In these examples, lowercase values are Cloud Observability variables and should be left as is. Values in {{ALL_CAPS}} represent user-defined variables – you should either verify that your span contains these attributes, or replace them with an appropriate attribute from your service.

Cloud Observability will automatically URL encode all link definitions and attribute values. This means if you provide values that have already been encoded, they may end up double-encoded and not render properly.

Email

This link will open an email to the owner of a specific set of services.

Template

Name Contact: {{$service}} owner
URL mailto:SOME_TEAM@MYCOMPANY.COM
Rules {
  "$service": ["MY_SERVICE_NAME","MY_OTHER_SERVICE_NAME"]
}

Example
This link will open an email addressed to the Client Team, and will be shown on spans from the ios-client and the react-client services only.

Name Contact: {{$service}} owner
URL mailto:team_client@acme.com
Rules {
  "$service": ["ios_client","react_client"]
}

Slack

Included as a template in Cloud Observability.

This link will open a Slack channel for spans with a specific attribute value.

Template

Name Contact: #MY_CHANNEL_NAME
URL https://slack.com/app_redirect?channel=MY_SLACK_CHANNEL
Rules {
  "MY_ATTRIBUTE": ["VALUE_A","VALUE_B"]
}

Example
This link displays for spans when the customer_region_code is east-coast, ny, or bos and opens the #customer_support_east_coast Slack channel.

Name Contact: #customer-support-east-coast
URL https://slack.com/app_redirect?channel=customer_support-east-coast
Rules {
  "customer-region-code": ["east-coast","ny", "bos"]
}

Wiki playbook

This link points to an alert playbook page and appears when the span has an HTTP error code indicating an authorization error.

Example

Name Playbook: Auth Error
URL http://wiki.com/playbook/auth_error#{{http.status_code}}
Rules {
  "http.status_code": ["401", "403"]
}

Generic dashboard

This definition creates a link for spans of all services that have release and environment attributes, and navigates to an internal dashboard for that release and environment, around the time when the span started.

Template

Name {{$service}} Dashboard at {{$startTimeISO}}
URL https://INTERNAL.DASHBOARD.COM?service={{$service}}&release_attribute={{RELEASE_ATTRIBUTE}}&environment={{ENVIRONMENT_ATTRIBUTE}}&beginTimestamp={{$startTimeSeconds-1hr}}&endTimestamp={{$startTimeSeconds+1hr}}
Rules {
  "$service": null,
  "RELEASE_ATTRIBUTE": null,
  "ENVIRONMENT_ATTRIBUTE": null
}

Example

This link displays for spans of all services in any release to the staging or production environment and navigates to a dashboard that shows data within +/- 1 hour of the span start time.

Name {{$service}} Dashboard at {{$startTimeISO}}
URL https://INTERNAL.DASHBOARD.COM?service={{$service}}&release_attribute={{release}}&environment={{evironment}}&beginTimestamp={{$startTimeSeconds-1hr}}&endTimestamp={{$startTimeSeconds+1hr}}
Rules {
  "$service": null,
  "release": null,
  "environment": ["staging", "production"]
}

GCP Stackdriver logs

Included as a template in Cloud Observability.

This definition creates a link for spans that have a defined attribute and searches Stackdriver logs for that attribute in a specific project.

Template

Do not URL encode the advancedFilter text below. Cloud Observability encodes links when displayed on the trace page.

Name Logs for {{MY_INTERESTING_ATTRIBUTE}} at {{$startTimeISO}}
URL https://console.cloud.google.com/logs/viewer?project=MY_PROJECT_NAME&advancedFilter=MY_SEARCH_TERMS&timestamp={{$startTimeISO}}&interval=JUMP_TO_TIME&scrollTimestamp={{$startTimeISO}}
Rules {
  "MY_INTERESTING_ATTRIBUTE": null
}

Example

This link searches logs for this customer_id in the logs for the current service (in the production environment). The link will only be displayed for spans that have an attribute called customer_id.

Name Logs for {{customer_id}} at {{$startTimeISO}}
URL https://console.cloud.google.com/logs/viewer?project=production&advancedFilter=labels.customer_id="{{customer_id}}" logName="project/logs/{{$service}}"&timestamp={{$startTimeISO}}&interval=JUMP_TO_TIME&scrollTimestamp={{$startTimeISO}}
Rules {
  "customer_id": null
}

GCP Kubernetes node

Included as a template in Cloud Observability.

This link will take you directly to the Kubernetes Dashboard page for the Node (VM) that hosts the pod that generated this span.

To use this template, you’ll need several pieces of information: the GCP Project ID and Zone, and GKE Cluster and Node ID. These can be hard-coded in the URL or added as attributes in your instrumentation, then included as variables in the URL.

Template

Name Kubernetes Node (VM)
URL https://console.cloud.google.com/kubernetes/node/{{GCP_ZONE}}/{{GKE_CLUSTER}}/{{GKE_NODE}}?project={{GCP_PROJECT_ID}}
Rules {
  "$service": null,
  "GCP_PROJECT_ID": null,
  "GCP_ZONE": ["ZONE_A", "ZONE_B"],
  "GKE_CLUSTER": null,
  "GKE_NODE": null
}

Example

This link displays for spans of all instances in the us-central1-f or us-east1 GCP zones, and navigates to the Kubernetes Node where the instrumented service instance is running.

Name Kubernetes Node (Machine)
URL https://console.cloud.google.com/kubernetes/node/{{zone}}/{{cluster}}/{{node}}?project={{project_id}}
Rules {
  "$service": null,
  "project": null,
  "zone": ["us-central1-f", "us-east1"],
  "cluster": null,
  "node": null
}

AWS logs

This definition creates a link for spans of all services that have a region attribute and navigates to the AWS logs for that service.

Template

Name CloudWatch Logs
URL https://console.aws.amazon.com/cloudwatch/home?region={{REGION_ATTRIBUTE}}#logEventViewer:group=/aws/lambda/{{$service}}
Rules {
  "$service": null,
  "REGION_ATTRIBUTE": null
}

Example

This link displays for spans of all services that have an attribute called region, navigates to AWS logs for that service and region.

Name CloudWatch Logs
URL https://console.aws.amazon.com/cloudwatch/home?region={{region}}#logEventViewer:group=/aws/lambda/{{$service}}
Rules {
  "$service": null,
  "region": null
}

EC2 dashboard

Included as a template in Cloud Observability.

This definition creates a link for spans of all services that have a region attribute and navigates to the Amazon EC2 dashboard for the span’s region.

Template

Name EC2 Dashboard
URL https://console.aws.amazon.com/cloudwatch/home?region={{REGION_ATTRIBUTE}}#cw:dashboard=EC2
Rules {
  "REGION_ATTRIBUTE": null
}

Example

This link displays for all spans with a region attribute (in any region) and navigates to the EC2 dashboard for that region.

Name EC2 Dashboard
URL https://console.aws.amazon.com/cloudwatch/home?region={{region}}#cw:dashboard=EC2
Rules {
  "region": null
}

GitHub search for an operation

Included as a template in Cloud Observability.

This definition creates a link for spans of all services and searches a defined GitHub repo for all instances of the span’s operation.

Template

Name GitHub Search: {{$operation}}
URL https://github.com/MY_REPO/search?q={{$operation}}
Rules {
  "$operation": null
}

Example

This link displays for all spans and searches the Cloud Observability GitHub repo for the span’s operation.

Name GitHub Search: {{$operation}}
URL https://github.com/lightstep/lightstep-tracer-go/search?q={{$operation}}
Rules {
  "$operation": null
}

Cloud Observability dashboard

Because you can create links for anything with a valid URL, you can even create a link into another page in Cloud Observability. This definition creates a link to a defined service’s dashboard at the span’s start time.

Template

Name {{$service}} LS Dashboard
URL https://app.lightstep.com/PROJECT_NAME/dashboard/DASHBOARD_NAME/DASHBOARDID?range=8880&anchor={{$startTimeMicros}}
Rules {
  "$service": ["CORRESPONDING_SERVICE"]
}

Example

This link displays for spans with the ios-client service and navigates to the Cloud Observability dashboard for the ios-client starting at the time the span started.

Name {{$service}} LS Dashboard
URL https://app.lightstep.com/lightstep-demo/dashboard/ios-client/rSWgMWKy?range=8880&anchor={{$startTimeMicros}}
Rules {
  "$service": ["ios-client"]
}

See also

Create links to important information

Create and manage Workflow Links

Use variables in Workflow Links

Updated Dec 1, 2019