alarm_analysis:data_retention
Differences
This shows you the differences between two versions of the page.
| Both sides previous revisionPrevious revision | Last revisionBoth sides next revision | ||
| alarm_analysis:data_retention [2019/11/15 14:46] – su | alarm_analysis:data_retention [2019/11/18 15:55] – su | ||
|---|---|---|---|
| Line 12: | Line 12: | ||
| ==== 1. Collection ==== | ==== 1. Collection ==== | ||
| - | Source Alarm & Event (A&E) data arrives on the import flow and is stored for later processing. | + | Source Alarm & Event (A&E) data arrives on the import flow and is stored for later processing. We call this the **A&E Raw Data Cache**. |
| Data is stored in monthly indices determined by date of arrival. This may be different to the date of the actual event record cotained in the raw A&E message (remember, the data is unprocessed at this stage so we don't know what it contains). | Data is stored in monthly indices determined by date of arrival. This may be different to the date of the actual event record cotained in the raw A&E message (remember, the data is unprocessed at this stage so we don't know what it contains). | ||
| Line 25: | Line 25: | ||
| ==== 2. Processing ==== | ==== 2. Processing ==== | ||
| - | The source data stored in stage 1 undergoes cleaning and transformation to generate an Alarm Analysis record optimised for analytics and aggregation. | + | The source data stored in stage 1 undergoes cleaning and transformation to generate an Alarm Analysis record optimised for analytics and aggregation. This is the **Alarm Analysis database**. |
| Data is stored in monthly indices determined by the A&E event date. The index name format is: | Data is stored in monthly indices determined by the A&E event date. The index name format is: | ||
alarm_analysis/data_retention.txt · Last modified: 2023/12/29 13:42 by su