alarm_analysis:resilient_data_transfer
Differences
This shows you the differences between two versions of the page.
Both sides previous revisionPrevious revisionNext revision | Previous revisionLast revisionBoth sides next revision | ||
alarm_analysis:resilient_data_transfer [2018/01/18 11:30] – su | alarm_analysis:resilient_data_transfer [2022/02/28 14:02] – su | ||
---|---|---|---|
Line 1: | Line 1: | ||
- | ====== | + | ====== |
===== Overview ===== | ===== Overview ===== | ||
- | This document | + | Before reading this guide, you should be familiar with [[alarm_analysis: |
- | For example, relaying data from a Process Control Network | + | In these scenarios, it is necessary |
- | Points to consider are security and data resilience. | + | This guide focuses on such a scenario |
+ | Data is gathered from serial feed and relayed via a Data Core Node from the Process Information Network to an App Store Connect on the Business Network. The relay is configured for Resilient Data Transfer. | ||
- | ===== Terminology ===== | + | {{ : |
- | | Alarm Analysis | Intelligent Plant application that processes Alarm & Event data to produce interactive EEMUA (Engineering Equipment and Materials Users Association) grade reports and analysis. | | + | A two box installation is required, with the following |
- | | Data Core Node | Intelligent Plant' | + | |
- | | Industrial App Store | Cloud portal for industrial apps (including online editions of Intelligent Plant applications). | | + | |
- | | App Store Connect | An instance of Data Core Node pre-configured for secure connections with the Industrial App Store. | | + | |
- | | TCP | Transmission Control Protocol. A network communication protocol. | | + | |
+ | {{ : | ||
- | ===== Data Relay across 3-Zone Network Architecture ===== | + | * Data Core listens for events (TCP Printer Stream). Arriving events are parsed and sent... |
+ | | ||
+ | - To a local data repository (Big Data Sink) | ||
- | The following example illustrates relaying data across a 3 zone network architecture. In each zone a Data Core Node is installed. | + | * A second process on the Data Core Node polls the local data repository for stored events (Big Data Source) and sends to... |
+ | - a further data relay that awaits an Ack from the App Store Connect (TCP Out). If Ack is not received, data is resubitted. | ||
- | {{: | + | * App Store Connect listens for incoming events (TCP In) and stores to a local data repository (Big Data Sink). |
- | Alarm & Event data enters Data Core Node 1 via a serial printer-port cable connected to a Serial to Ethernet Converter. This physical architecture guarantees one-way data-flow from Process Controllers | + | While connectivity between the PIN and BN is good, all events will reach the destination twice. The fast stream means data arrives quickly, the second stream guarantees resilience. App Store Connect consolidates the two streams |
- | A **Fast Flow** relays data across the zones via a fire-and-forget process providing near-real-time data on the destination server. | ||
- | On a separate **resilient flow** the database on the source server is poled and new data relayed via a guaranteed delivery process. This provides a complete data record on the destination server. | ||
- | The database on the destination server consolidates data from both flows. | + | ====== Step by Step Install and Configuration ====== |
- | ===== Data Core Installation ===== | + | **1. Install |
- | Stand-Alone installations of Data Core are required on each domain traversed. | + | For detailed instructions on how to deploy a Data Core node, see: [[data_core: |
- | If the intention is to make data available to the App Store, the destination server could be an "App Store Connect" | + | **2. Data Core Node Configuration** |
- | < | + | Configure the following |
- | App Store Connect is an instance of a Data Core Node pre-configured for secure connection with the Industrial App | + | |
- | Store. | + | |
- | </ | + | |
- | + | ||
- | ===== Data Core Configuration ===== | + | |
- | Configure the following | + | For detailed instructions on how to create an Event Source |
- | + | ||
- | {{:alarm_analysis:resilientdatatransfer7.png|}} | + | |
^ //TCP Printer Stream// ^^ | ^ //TCP Printer Stream// ^^ | ||
^ Type | TCP Printer (Event Source) | | ^ Type | TCP Printer (Event Source) | | ||
+ | ^ Description | Listen and parse data arriving on TCP channel | | ||
+ | ^ Disabled | False | | ||
^ TCP Port | 9000 | | ^ TCP Port | 9000 | | ||
^ Message Delimiter | New Line {\n} | | ^ Message Delimiter | New Line {\n} | | ||
^ Maximum Characters per Scan | 4000 | | ^ Maximum Characters per Scan | 4000 | | ||
- | ^ //Fast TCP Out// ^^ | + | ^ // Fast TCP Out// ^^ |
^ Type | TCP Event Sink (Event Sink) | | ^ Type | TCP Event Sink (Event Sink) | | ||
+ | ^ Description | Immediate data transfer to the Business Network | | ||
+ | ^ Disabled | False | | ||
^ TCP Server Host | [ IP Address of server hosting "Fast TCP In" ] | | ^ TCP Server Host | [ IP Address of server hosting "Fast TCP In" ] | | ||
^ TCP Server Port| 11000 | | ^ TCP Server Port| 11000 | | ||
^ Username | [Service account with access to server hosting "Fast TCP In" ] | | ^ Username | [Service account with access to server hosting "Fast TCP In" ] | | ||
^ Password | [Service account password with access to server hosting "Fast TCP In" ] | | ^ Password | [Service account password with access to server hosting "Fast TCP In" ] | | ||
- | ^ Check Response | False | | + | ^ Check Response | No| |
+ | ^ // | ||
- | ^ //Retrieve | + | ^ //Big Data Sink// ^^ |
+ | ^ Type | Big Data Event Sink (Event Sink) | | ||
+ | ^ Description | Save data to the Big Data Store | | ||
+ | ^ Disabled | False | | ||
+ | ^ Big Data URL | http:// | ||
+ | ^ // | ||
+ | |||
+ | ^ //Big Data Source// ^^ | ||
^ Type | Big Data Event Source (Event Source) | | ^ Type | Big Data Event Source (Event Source) | | ||
+ | ^ Description | Retrieve collected data from Big Data Store | | ||
+ | ^ Disabled | False | | ||
+ | ^ Paused | False | | ||
^ Big Data URL | http:// | ^ Big Data URL | http:// | ||
^ Index Filter | tcpprinterstream.evt_* | | ^ Index Filter | tcpprinterstream.evt_* | | ||
^ Sleep Period | 30 | | ^ Sleep Period | 30 | | ||
^ Lag | 60| | ^ Lag | 60| | ||
+ | |||
+ | ^ // TCP Out// ^^ | ||
+ | ^ Type | TCP Event Sink (Event Sink) | | ||
+ | ^ Description | Resilient data transfer to the Business Network | | ||
+ | ^ Disabled | False | | ||
+ | ^ TCP Server Host | [ IP Address of server hosting "TCP In" ] | | ||
+ | ^ TCP Server Port| 11000 | | ||
+ | ^ Username | [Service account with access to server hosting "TCP In" ] | | ||
+ | ^ Password | [Service account password with access to server hosting "TCP In" ] | | ||
+ | ^ Check Response | Yes | | ||
+ | ^ // | ||
+ | |||
+ | **3. Install App Store Connect on BN** | ||
+ | |||
+ | For detailed instructions on how to deploy App Store Connect, see: [[data_core: | ||
+ | |||
+ | |||
+ | **4. App Store Connect Configuration** | ||
+ | |||
+ | Configure the following Data Core components (assume default settings unless otherwise stated). | ||
+ | |||
+ | For detailed instructions on how to create an Event Source to Sink subscription, | ||
^ //TCP In// ^^ | ^ //TCP In// ^^ | ||
^ Type | TCP Event Source (Event Source) | | ^ Type | TCP Event Source (Event Source) | | ||
+ | ^ Description | Receive data from Process Information Network | ||
+ | ^ Disabled | False | | ||
^ TCP Server Port| 11000 | | ^ TCP Server Port| 11000 | | ||
- | ^ //TCP Out// ^^ | + | ^ //Big Data Sink// ^^ |
- | ^ Type | TCP Event Sink (Event Sink) | | + | |
- | ^ TCP Server Host | [ IP Address of server hosting " | + | |
- | ^ TCP Server Port| 11000 | | + | |
- | ^ Username | [Service account with access to server hosting " | + | |
- | ^ Password | [Service account password with access to server hosting " | + | |
- | ^ Check Response | True | | + | |
- | + | ||
- | ^ //Store Data// ^^ | + | |
^ Type | Big Data Event Sink (Event Sink) | | ^ Type | Big Data Event Sink (Event Sink) | | ||
+ | ^ Description | Save data to the Big Data Store | | ||
+ | ^ Disabled | False | | ||
^ Big Data URL | http:// | ^ Big Data URL | http:// | ||
^ Big Data Refresh Interval | 5s | | ^ Big Data Refresh Interval | 5s | | ||
+ | ^ // | ||
+ | |||
+ | |||
+ | **5. Firewall Settings** | ||
+ | |||
+ | Firewalls will need to allow passage for the following protocols on ports: | ||
+ | |||
+ | ^ Firewall ^ Requirements ^ | ||
+ | | BN:Internet Network Firewall | TCP Port 443 open to outbound traffic from computer hosting App Store Connect and user machines to: \\ \\ https:// | ||
+ | | Computer hosting App Store Connect | Windows Firewall TCP Port 443 open to outbound traffic \\ TCP Port 11000 open to inbound traffic | | ||
+ | |PIN:BN Network Firewall | TCP Port 11000 open to outbound traffic | | ||
+ | | PCN:PIN Network Firewall | No inbound access required. | | ||
+ | |||
+ | ==== 5. Testing ==== | ||
+ | |||
+ | Assuming that the TCP Printer Stream configured above is listening to an active Alarm & Event stream, we should see evidence of data store to the Big Data repositories on the PIN and BN. | ||
+ | |||
+ | A quick test is to execute a URL search query. | ||
+ | |||
+ | 1. Log on to servers hosting Data Core Node | ||
+ | |||
+ | Open web browser and enter | ||
+ | < | ||
+ | |||
+ | A " | ||
+ | |||
+ | < | ||
+ | health status index pri rep docs.count docs.deleted store.size pri.store.size | ||
+ | green open | ||
+ | </ | ||
+ | |||
+ | A " | ||
+ | |||
+ | 2. Log on to servers hosting App Store Connect | ||
+ | |||
+ | Open web browser and enter | ||
+ | < | ||
+ | |||
+ | You should expect to see something like: | ||
+ | |||
+ | < | ||
+ | health status index pri rep docs.count docs.deleted store.size pri.store.size | ||
+ | green open | ||
+ | </ | ||
+ | |||
+ | A " | ||
+ | |||
+ | A " | ||
+ | |||
+ | |||
+ | |||
+ | |||
+ | |||
+ | |||
+ | |||
+ | |||
+ | |||
+ | |||
+ | ==== 6. Next Steps ==== | ||
+ | |||
+ | So far, we've moved Alarm & Event data across a network. We are now ready to configure Alarm Analysis processing. | ||
+ | For more info, see [[alarm_analysis: | ||
- | ===== Firewall Requirements ===== | ||
- | Network and server firewalls must be open for TCP traffic on the ports configured above. | ||
alarm_analysis/resilient_data_transfer.txt · Last modified: 2023/12/29 13:41 by su