Create ingestion rules to control metric-ingestion costs.

With ingestion rules, you block low-value metrics from Lightstep based on metric names and other attributes. Ingestion rules can help you:

  • Ingest only the metrics you need.
  • Lower your monthly Active Time Series (ATS) usage.
  • Stay within your budget.

This page shows how to view, create, and delete ingestion rules. It also shows how Lightstep estimates a rule’s impact on your monthly ATS usage.

Before you begin

The sections below assume you’ve:

The sections are also primarily for users with the Billing admin role. Billing admin users can do everything with ingestion rules, including creating, renaming, and deleting rules. Other roles (Admin, Member, and Viewer) can only do the following:

  • View active ingestion rules.
  • View ingestion-rule activity.

View ingestion rules

To view ingestion rules in Lightstep, click Account management > Metric ingestion rules. The Metric ingestion rules page has two tabs: Active rules and Activity.

Active rules

The Active rules tab lists the ingestion rules working in Lightstep. It shows these details about each rule:

  • Name - The name of the ingestion rule.
  • Project - The Lightstep project the rule applies to.
  • Created on - The date the rule was first activated.

You can also click a rule to learn more about it, including its configuration and who created it.

Activity

The Activity tab shows all ingestion-rule user activity. The tab is organized by time, and you can use it to see when users create, delete, and update rules.

The table in the Activity tab includes this information:

  • Date - The date of the event.
  • Event - The action that happened: rule created, rule updated, or rule deleted.
  • Rule - The ingestion rule’s name.
  • User - The email of the user who did the action.

You can also click a rule to learn more about it, including its configuration and who created it.

Create ingestion rules

Ingestion rules block data from Lightstep, helping you lower your ATS usage. When creating ingestion rules, keep in mind the following:

  • Lightstep can’t recover blocked data.
  • Lightstep doesn’t delete any historical data.
  • Lightstep blocks future data, so you can create rules matching data you’re not sending yet.

Follow these steps to create an ingestion rule:

  1. In Lightstep, click Account management > Metric ingestion rules and then click Create new rule.
  2. On the Rule configuration page:
    1. Enter a name for your rule. Example: Block supernova-host requests.
    2. Select the relevant project in the project drop-down.

    3. Next, create a filter for your rule by selecting attribute keys and values. You can add several filters to your rule by clicking add filter. Example:

      The rule in the image below blocks all metrics named requests where the customer attribute isn’t sweetpines. Note that when you use !=, you block everything except that value.

      Ingestion rule filter example

    4. Preview the estimated effects of your rule on your ATS usage.

      Lightstep uses data from the past 30 days to estimate how the rule impacts your ATS usage, providing this information:

      • The adjusted ATS usage after rule creation.
      • The percentage decrease in your ATS usage compared to your current usage.
      • The effect on ATS in a chart (see the image below for an example). The purple series shows the ATS matching your rule. The blue series shows your adjusted ATS usage after applying the rule.

      Estimated effects on ATS usage

      Note that the estimates use historical data. The actual changes may differ because they depend on future data ingestion.

  3. To save your rule, click Save. In the Save rule? dialog, type CONFIRM and click Yes, save.

Lightstep displays Rule saved and returns you to the Active rules tab. Your rule now appears in the table, and you can rename the rule by clicking ⋮ More > Edit name.

You can only edit rule names in Lightstep. To change a rule’s configuration, delete the rule and then create a new one.

Delete ingestion rules

When you delete an ingestion rule, Lightstep stops implementing the rule and ingests data matching the rule.

Deleting a rule can affect your ATS usage, so you may risk going over your limits. Lightstep can’t estimate how much your ATS usage may increase because it’s been blocking data matching the rule.

Follow these steps to delete an ingestion rule:

  1. In Lightstep, click Account management > Metric ingestion rules.
  2. Find the rule you want to delete and click ⋮ More > Delete.
  3. In the Delete rule? dialog, type CONFIRM and click Yes, delete.

Lightstep displays Successfully deleted <rule name> and returns you to the Active rules tab.

You now know how to use ingestion rules to help control Lightstep costs. To learn more about billing, visit the links below.