Docs > Data Management > Data Retention
1. Getting Started with vuSmartMaps™
3. Console
5. Configuration
6. Data Management
9. Monitoring and Managing vuSmartMaps™
In vuSmartMaps, administrators can efficiently manage data retention settings through a user-friendly graphical interface. This comprehensive guide provides an overview of how to define retention durations for Hot, Warm, and Archived storage storages, optimizing system performance, compliance, and data management.
To manage data retention effectively, it is essential to have admin access.
vuSmartMaps supports a two-storageed data storage strategy, subdivided into three retention categories:
Hot Storage serves immediate and frequent access needs for recent and critical data. This storage offers the fastest access speeds, making it ideal for live queries, active analytics, and real-time operations.
Configurable Retention: Administrators can set the number of days data remains in Hot Storage before being transitioned to the next storage, ensuring optimal performance and availability.
Warm Storage provides an intermediate storage for data that is less frequently accessed than Hot Storage but remains relevant and needs moderate-speed access. This storage is suitable for data that is still in use but not as frequently queried or analyzed as Hot data.
Configurable Retention: You can specify how long data stays in Warm Storage before being moved to the Archive storage.
Archived Storage is used for long-term retention of historical or infrequently accessed data. Data in this storage is not immediately available for live queries but is retained for reference, compliance, and long-term storage needs. Administrators can bring archived data back to the Warm storage when necessary using the restore functionality.
Administrators have the flexibility to determine when data transitions from Live (Hot/Warm) storage to the Archive storage. This can be achieved through the “Configure Archival Strategy” option. Clicking on this button opens a modal that provides two options for defining the archival strategy:
In this strategy, data is archived daily starting from the previous day of ingestion into Live Storage (Hot/Warm). For example, if today is the 25th, data starting from the 24th will be archived. Data from the current day will not be archived. It remains in the Archive storage for a user-defined period before deletion, ensuring a continuous and incremental movement of data to the Archive for timely retention and deletion.
💡 Note: The retention period for this strategy begins from the day after the data is ingested into Hot or Warm storage. Data will be archived daily and retained in the Archive storage for the specified number of days before being permanently deleted.
This strategy archives data once its configured retention period in Live Storage (Hot/Warm) has ended. From that point, the data is retained in the Archive storage for a user-defined duration before deletion.
💡 Note: The retention period for this strategy is calculated from the day data is configured to be removed from Live Storage (Hot or Warm). To ensure data continuity, archiving starts a few days earlier, based on a configurable environment variable (DAYS_TO_SYNC_COLD_STORE). By default, this value is set to 5 days.
For example, if today is the 25th and the retention period for Hot/Warm storage is configured to end on the 20th, data from the 20th will start being archived on the 15th (5 days earlier). From the 20th, the data will be retained in the Archive storage for the configured duration before it is deleted.
💡 Note: Every day at 10:30 PM, data is automatically moved from Hot/Warm to Archived storage. Older data is then deleted from Archived storage based on the retention settings.
You can configure data retention periods for each storage to suit your organization’s needs:
Consider a configuration with the following retention periods:
Explanation:
Consider a configuration with the following retention periods:
And today is 21st, the system calculates:
By categorizing data into Hot, Warm, and Cold Storage, vuSmartMaps enables administrators to strategically align their data management with the nature and frequency of access, thus optimizing storage efficiency. Data retention settings are designed to effectively manage data across these states. Optimizing data storage is crucial, as it directly influences costs. The configuration of data retention settings is flexible, allowing configurations to suit user-specific requirements. Refer configuration of data retention settings to know the specific steps in detail to manage hot, warm, and cold retention periods.
The retention periods follow a hierarchical order:
For example, if Hot storage is set to 10 days, Warm to 20 days, and Cold to 30 days, data flows from Hot to Warm to Cold storage accordingly.
Yes, you can edit the default data retention policy by clicking on the Default policy row and modifying the retention periods for Hot, Warm, and Cold storage tiers. However, you cannot delete the default policy.
Data is automatically moved from Hot/Warm to Cold storage every day at 10:30 PM based on the retention settings configured.
If only the hot retention period is defined, data will remain in the hot state until it reaches the end of the hot retention period. After that, if no warm or cold retention periods are defined, the data that is beyond the retention will be deleted. If the retention period is 3 days, then the hot storage will have only the last 3 days of data.
In this scenario, data from the last 10 days is stored in Hot storage. Data from the preceding 10 days (days T-11 to T-20) is stored in Warm storage. Cold storage maintains a backup copy of Hot data, preserving the last 30 days of data.
To achieve this, set retention periods for Hot, Warm, and Cold storage as 3, 6, and 9 days, respectively.
In this case, data from the last 3 days would be stored in Hot storage. No data will be stored in Warm storage. A duplicate copy of the data from the last 3 days would be maintained in Cold storage.
No, the retention policies will take effect from the moment they are applied. Starting today, your data will be managed according to the new settings: data will be stored in the hot tier for 10 days, then moved to the warm tier for the next 10 days, and finally retained in the cold tier for a total of 30 days. Data from the past 30 days prior to applying these settings will not be retroactively moved to the cold tier.
Direct querying or visualization of data in the cold tier is not possible. To access and visualize this data, you must first restore it from the cold tier to the warm tier. Once restored, you can query and build visualizations on the data as needed.
If you reduce the retention period for any tier, existing data that exceeds the new retention period will be purged according to the updated policy. For example, if you reduce the hot tier from 10 days to 5 days, any data older than 5 days will be moved to the warm tier (or purged if it exceeds the warm tier’s retention as well).
Browse through our resources to learn how you can accelerate digital transformation within your organisation.
VuNet’s Business-Centric Observability platform, vuSmartMaps™ seamlessly links IT performance to business metrics and business journey performance. It empowers SRE and IT Ops teams to improve service success rates and transaction response times, while simultaneously providing business teams with critical, real-time insights. This enables faster incident detection and response.