Control costs with ingestion rules

Create ingestion rules to control metric-ingestion costs.

With ingestion rules, you block low-value metrics from Cloud Observability based on metric names and other attributes. Ingestion rules can help you:

  • Ingest only the metrics you need.
  • Lower your monthly Active Time Series (ATS) usage.
  • Stay within your budget.

This page shows how to view, create, and delete ingestion rules. It also shows how Cloud Observability estimates a rule’s impact on your monthly ATS usage.

Before you begin

The sections below assume you’ve:

The sections are also primarily for users with the Organization Billing Admin role. Organization Billing Admin users can do everything with ingestion rules, including creating, renaming, and deleting rules. Other roles (Organization Admin, Organization Editor, and Organization Viewer) can only do the following:

  • View active ingestion rules.
  • View ingestion-rule activity.

View ingestion rules

To view ingestion rules in Cloud Observability, click Settings > Metric ingestion rules. The Metric ingestion rules page has two tabs: Active rules and Activity.

Active rules

The Active rules tab lists the ingestion rules working in Cloud Observability. It shows these details about each rule:

  • Name - The name of the ingestion rule.
  • Project - The Cloud Observability project the rule applies to.
  • Created on - The date the rule was first activated.

You can also click a rule to learn more about it, including its configuration and who created it.

Activity

The Activity tab shows all ingestion-rule user activity. The tab is organized by time, and you can use it to see when users create, delete, and update rules.

The table in the Activity tab includes this information:

  • Date - The date of the event.
  • Event - The action that happened: rule created, rule updated, or rule deleted.
  • Rule - The ingestion rule’s name.
  • User - The email of the user who did the action.

You can also click a rule to learn more about it, including its configuration and who created it.

Create ingestion rules

Ingestion rules block data from Cloud Observability, helping you lower your ATS usage. When creating ingestion rules, keep in mind the following:

  • Cloud Observability can’t recover blocked data.
  • Cloud Observability doesn’t delete any historical data.
  • Cloud Observability blocks future data, so you can create rules matching data you’re not sending yet.

Follow these steps to create an ingestion rule:

  1. In Cloud Observability, click Settings > Metric ingestion rules and then click Create new rule.
  2. On the Rule configuration page:
    1. Enter a name for your rule. Example: Block supernova-host requests.
    2. Select the relevant project in the project drop-down.

    3. Next, create a filter for your rule by selecting attribute keys and values. You can add several filters to your rule by clicking add filter. Example:

      The rule in the image below blocks all metrics named requests where the customer attribute isn’t sweetpines. Note that when you use !=, you block everything except that value.

      Filter one specifies metrics where `lightstep.metric_name` equals `requests`. Filter two specifies metrics where `customer` isn't equal to `sweetpines`.

    4. Preview the estimated effects of your rule on your ATS usage.

      Cloud Observability uses data from the past 30 days to estimate how the rule impacts your ATS usage, providing this information:

      • The adjusted ATS usage after rule creation.
      • The percentage decrease in your ATS usage compared to your current usage.
      • The effect on ATS in a chart (see the image below for an example). The purple series shows the ATS matching your rule. The blue series shows your adjusted ATS usage after applying the rule.

      Chart estimates the rule's impact using data from the past 30 days.

      Note that the estimates use historical data. The actual changes may differ because they depend on future data ingestion.

  3. To save your rule, click Save. In the Save rule? dialog, type CONFIRM and click Yes, save.

Cloud Observability displays Rule saved and returns you to the Active rules tab. Your rule now appears in the table, and you can rename the rule by clicking ⋮ More > Edit name.

You can only edit rule names in Cloud Observability. To change a rule’s configuration, delete the rule and then create a new one.

Delete ingestion rules

When you delete an ingestion rule, Cloud Observability stops implementing the rule and ingests data matching the rule.

Deleting a rule can affect your ATS usage, so you may risk going over your limits. Cloud Observability can’t estimate how much your ATS usage may increase because it’s been blocking data matching the rule.

Follow these steps to delete an ingestion rule:

  1. In Cloud Observability, click Settings > Metric ingestion rules.
  2. Find the rule you want to delete and click ⋮ More > Delete.
  3. In the Delete rule? dialog, type CONFIRM and click Yes, delete.

Cloud Observability displays Successfully deleted <rule name> and returns you to the Active rules tab.

You now know how to use ingestion rules to help control Cloud Observability costs. To learn more about billing, visit the links below.

See also

Billing

Monitor usage

Create billing dashboards

Updated Apr 24, 2023