Explore older logs by moving them from cold to hot storage.
Overview
To help you manage costs, Cloud Observability stores logs based on their timestamps:
- Hot storage - Stores logs in a queryable format for 3 days by default.
- Cold storage - Stores logs in a less expensive, non-queryable format for 30 days by default.
Rehydration – moving logs from cold to hot storage – lets you query and explore the older logs. The process can help you investigate issues or events that happened before the hot-storage retention period.
Cloud Observability bills you for rehydrated logs at hot-storage rates. You can remove the rehydrated logs from hot storage when you’re done exploring them.
Log retention periods are configurable. Reach out to your customer success representative to learn more.
Rehydrate logs
Follow these steps to rehydrate logs:
- In Cloud Observability, click Settings > Log rehydration.
- Select the relevant project in the For project drop-down.
-
Click Create a new rehydration event and fill out the form:
- Reason for rehydration - Describe why you’re moving these logs from cold to hot storage. Your description is searchable on the Log rehydration page.
-
Date range - Select the date and time.
You can only rehydrate logs for a one-hour interval, maximum. For example, you can rehydrate logs from 9:45 to 10:45 but not from 9:45 to 11. If you need to rehydrate logs for a longer interval, create several hour-long rehydration entries.
- Review the ingest size in the Statistics table and click Rehydrate logs to return to the Log rehydration page.
Your rehydration event appears in the table. In-progress rehydration events are yellow – hover over the spinner to track the progress.
You can query rehydrated logs in dashboards, notebooks, and the logs tab as soon as the rehydration event starts. All rehydrated logs are available when the event finishes and turns grey in the table.
Remove rehydrated logs
Follow these steps to remove rehydrated logs from hot storage. Cloud Observability still keeps the logs in cold storage for the 30-day or customized cold-storage retention period.
- In Cloud Observability, click Settings > Log rehydration.
- Find the relevant rehydration event and click ⋮ > Evict logs.
Your event turns red, indicating an in-progress removal. Hover over the spinner to track the progress.
The event is italicized and turns grey when Cloud Observability finishes removing the rehydrated logs from hot storage.
See also
Understand data retention in Cloud Observability
Updated Sep 22, 2023