Cloud Observability’s log ingestion pipelines have three new transformations: Format, Set, and Regex extract. The new transformations can help you manage, customize, and analyze your logs.
What’s new?
Log ingestion pipelines have the following new transformations:
Format
Aggregate multiple attributes into a single attribute.
Format can help you create more meaningful log attributes from existing attributes.
For example, combining the source and environment attributes can help you identify a log’s origin in different environments.
Set
Assign specific values to new or existing attributes.
Set helps you standardize logs or add context to logs.
For example, assign a critical priority to logs matching specific error conditions.
Regex extract
Parse attributes using regular expressions. Use Regex extract to apply a pattern and capture specific information from logs. For example, if logs have error messages with timestamps, error codes, and descriptions, Regex extract can parse and move those details into individual attributes.
Why use the new transformations?
The new transformations give you more control over your log data, helping you with the following:
- Simplifying data - Aggregate and parse attributes to reduce clutter and make logs easier to analyze.
- Standardizing information - Use Set to assign consistent values, helping you filter and interpret logs.
- Improving log clarity - Create more organized and meaningful logs.
Get started
Start using Format, Set, and Regex extract by navigating to Cloud Observability and selecting Settings > Log ingestion pipelines. For instructions and examples, visit the updated documentation.
Updated Sep 25, 2024