Differences

This shows you the differences between two versions of the page.

--- alarm_analysis:data_retention [2019/11/15 08:29] – [Configuring a Data Retention Policy] su
+++ alarm_analysis:data_retention [2023/12/29 13:42] (current) – su
@@ Line 12: / Line 12: @@
 ==== 1. Collection ====
-Source Alarm & Event (A&E) data arrives on the import flow and is stored for later processing.
+Source Alarm & Event (A&E) data arrives on the import flow and is stored for later processing. We call this the **A&E Raw Data Cache**.
-Data is stored in monthly indices determined by date of arrival. This may be different to the date of the actual event record cotained in the raw A&E message (remember, the data is unprocessed at this stage so we don't know what it contains).
+Data is stored in monthly indices determined by date of arrival. This may be different to the date of the actual event record contained in the raw A&E message (remember, the data is unprocessed at this stage so we don't know what it contains).
 For an ongoing (near real-time) data flow, the monthly indices of raw data will be closely aligned with A&E event time. However, exceptions may occur. For example, importing a large historical backlog.
@@ Line 25: / Line 25: @@
 ==== 2. Processing ====
-The source data stored in stage 1 undergoes cleaning and transformation to generate an Alarm Analysis record optimised for analytics and aggregation.
+The source data stored in stage 1 undergoes cleaning and transformation to generate an Alarm Analysis record optimised for analytics and aggregation. This is the **Alarm Analysis database**.
 Data is stored in monthly indices determined by the A&E event date. The index name format is:
@@ Line 59: / Line 59: @@
 The following settings are available:
-  * Delete Expired Indices \\ Enable a data retention policy that deletes expired indices. \\ //Only enable when absolutely sure all other config is correct as this will attempt to permanently delete data.//
+^ Setting ^ Description ^ Example ^
+^ Delete Expired Indices | Enable a data retention policy that deletes expired indices. \\ \\ Only enable when absolutely sure all other config is correct as this will attempt to permanently delete data. \\ \\ | True or False |
+^ Index Names | Indices to be included in deletion checks. \\ \\  Wildcards should be employed with care and only used for date suffixes. \\ \\  Comma separate multiple names. \\ \\  Do not modify when Data Retention is enabled. \\ \\ | scadaopc.evt_* |
+^ Retention Period | Specify data retention period in months. \\ \\ Do not modify when Data Retention is enabled. \\ \\  | 6 |
+----
+===== Schedule =====
+The Data Retention Policy (once enabled) is executed daily at 1am.
-  * Index Names
-  * Retention Period