Log ingestion pipelines let you process and parse incoming logs in a unified interface. Use log ingestion pipelines to manage, secure, and optimize the logs you see in Cloud Observability.
What are log ingestion pipelines?
Log ingestion pipelines help you manage incoming logs. Within ingestion pipelines, you create transformations to process and parse logs to suit your specific needs.
Cloud Observability’s log ingestion pipelines come with these features:
- Customized pipelines: Create pipelines for specific data sources or unify log management with one pipeline.
- Supported tools: Use ingestion pipelines on logs from OpenTelemetry or Datadog.
- Transformations: Process and parse logs with transformations, including Drop matching, Keep matching, and Parse JSON.
Key benefits
Some benefits of using log ingestion pipelines include:
- Centralized log management: Process and manage logs from multiple sources in a unified interface.
- Cost optimization: Control storage costs by keeping only the logs that matter to your operations.
- Security: Safeguard sensitive information by removing it from incoming logs.
Get started
Explore the new documentation to start managing and transforming your logs to suit your needs:
- Create ingestion pipelines - Set up your first transformation and start optimizing your log management process.
- Ingestion pipeline reference - Explore every transformation type with detailed examples.
Updated May 1, 2024