User Tools

Site Tools


alarm_analysis:data_retention

Differences

This shows you the differences between two versions of the page.

Link to this comparison view

Both sides previous revisionPrevious revision
Next revision
Previous revision
Last revisionBoth sides next revision
alarm_analysis:data_retention [2019/11/15 14:41] – [Configuring a Data Retention Policy] sualarm_analysis:data_retention [2019/11/18 15:55] su
Line 12: Line 12:
 ==== 1. Collection ====  ==== 1. Collection ==== 
  
-Source Alarm & Event (A&E) data arrives on the import flow and is stored for later processing.+Source Alarm & Event (A&E) data arrives on the import flow and is stored for later processing. We call this the **A&E Raw Data Cache**.
  
 Data is stored in monthly indices determined by date of arrival. This may be different to the date of the actual event record cotained in the raw A&E message (remember, the data is unprocessed at this stage so we don't know what it contains).  Data is stored in monthly indices determined by date of arrival. This may be different to the date of the actual event record cotained in the raw A&E message (remember, the data is unprocessed at this stage so we don't know what it contains). 
Line 25: Line 25:
 ==== 2. Processing ====  ==== 2. Processing ==== 
  
-The source data stored in stage 1 undergoes cleaning and transformation to generate an Alarm Analysis record optimised for analytics and aggregation.+The source data stored in stage 1 undergoes cleaning and transformation to generate an Alarm Analysis record optimised for analytics and aggregation. This is the **Alarm Analysis database**.
  
 Data is stored in monthly indices determined by the A&E event date. The index name format is: Data is stored in monthly indices determined by the A&E event date. The index name format is:
Line 63: Line 63:
 ^ Index Names | Indices to be included in deletion checks. \\ \\  Wildcards should be employed with care and only used for date suffixes. \\ \\  Comma separate multiple names. \\ \\  Do not modify when Data Retention is enabled. \\ \\ | scadaopc.evt_* | ^ Index Names | Indices to be included in deletion checks. \\ \\  Wildcards should be employed with care and only used for date suffixes. \\ \\  Comma separate multiple names. \\ \\  Do not modify when Data Retention is enabled. \\ \\ | scadaopc.evt_* |
 ^ Retention Period | Specify data retention period in months. \\ \\ Do not modify when Data Retention is enabled. \\ \\  | 6 | ^ Retention Period | Specify data retention period in months. \\ \\ Do not modify when Data Retention is enabled. \\ \\  | 6 |
 +
 +----
 +
 +===== Schedule =====
 +
 +The Data Retention Policy (once enabled) is executed daily at 1am.
 +
 +
 +
  
  
alarm_analysis/data_retention.txt · Last modified: 2023/12/29 13:42 by su