App Store Wiki

User Tools

Site Tools

Trace:
alarm_analysis:resilient_data_transfer

Differences

This shows you the differences between two versions of the page.

Link to this comparison view

Both sides previous revisionPrevious revision
Next revision		Previous revision
Next revisionBoth sides next revision
alarm_analysis:resilient_data_transfer [2018/02/05 14:19] – [1. PIN: Data Core Node] sualarm_analysis:resilient_data_transfer [2018/02/21 10:26] – su
Line 3: Line 3:
 ===== Overview ===== ===== Overview =====
  
-Intelligent Plant produces highly configurable tools for capturing and analyzing data. We provide solutions that can be deployed in high-security/militarized network zones and reliably move data across networks, without compromising security.+Intelligent Plant produces highly configurable tools for capturing and analyzing data.
  
-This document describes best practice for a particular use-case:+We provide solutions that are deployed in high-security/militarized network zones and reliably move data across networks, without compromising security.
  
-//Collect **Process Control Alarm & Event** data for **Alarm Analysis** on the **Industrial App Store.**//+This document describes best practice for a particular use-case: 
  
 +//How to collect Alarm & Event data from a Process Control Network and make available to Alarm Analysis on the Industrial App Store.
 +//
  
 ===== Terminology ===== ===== Terminology =====
Line 31: Line 33:
     - TCP Printer Port listener (listening for input from the Serial to Ethernet Converter)     - TCP Printer Port listener (listening for input from the Serial to Ethernet Converter)
     - Alarm & Event Collector (collecting A&E data from the PCN)     - Alarm & Event Collector (collecting A&E data from the PCN)
-    - a TCP Out Channel for data transfer \\ \\+    - TCP Out Channel for data transfer \\ \\
   - The Data Core TCP Out Channel (on the PIN) initiates an authenticated TCP connection with App Store Connect (on the BN). Data transmitted on the TCP channel is signed and encrypted. \\ \\ Alarm & Event data is sent to App Store Connect. App Store Connect responds with an acknowledgement. If the Alarm Analysis server fails to receive a positive acknowledgement, data is resubmitted. \\ \\   - The Data Core TCP Out Channel (on the PIN) initiates an authenticated TCP connection with App Store Connect (on the BN). Data transmitted on the TCP channel is signed and encrypted. \\ \\ Alarm & Event data is sent to App Store Connect. App Store Connect responds with an acknowledgement. If the Alarm Analysis server fails to receive a positive acknowledgement, data is resubmitted. \\ \\
-  - App Store Connect acts a local data processor and supports connections to the Industrial App Store. \\ \\ App Store Connect incudes:+  - App Store Connect acts as local data processor and supports connections to the Industrial App Store. \\ \\ App Store Connect incudes:
     - TCP Listener (listening for input from the Alarm Analysis server)     - TCP Listener (listening for input from the Alarm Analysis server)
     - Alarm & Event Collector (collecting A&E data from the Alarm Analysis server)     - Alarm & Event Collector (collecting A&E data from the Alarm Analysis server)
Line 42: Line 44:
  
 ===== Deployment and Configuration Guide ===== ===== Deployment and Configuration Guide =====
 +
 +The network topology above assumes data crosses into the PIN via a serial printer feed.
 +
 +In order to collect data on the printer stream and transfer to the BN we need to undertake the following:
 +
 +  - Install a Data Core node on the PCN
 +  - Install App Store Connect on the PIN
 +
 +The configuration of these //Nodes// is as follows:
 +
 +{{ :data_core:resilienttransfer_02.png |}}
 +
  
 ==== 1. PIN: Data Core Node ==== ==== 1. PIN: Data Core Node ====
  
-The network topology above assumes data crosses into the PIN via a serial printer porter feed.+For detailed instructions on how to deploy a Data Core node, see: [[data_core:stand-alone_installation]]
  
-Our first task is to install and configure a Data Core node on the PIN.+For detailed instructions on how to create an Event Source to Sink subscription, see: [[data_core:Event Subscription]]
  
-  * [[data_core:stand-alone_installation | Install Stand-Alone instance of Data Core ]] +Configure the following Data Core components (assume default settings unless otherwise stated):
- +
-Configure the following Data Core components (assume default settings unless explicitly specified):+
  
 ^ //TCP Printer Stream// ^^ ^ //TCP Printer Stream// ^^
 +^ Type | TCP Printer (Event Source) |
 ^ Description | Listen and parse data arriving on TCP channel | ^ Description | Listen and parse data arriving on TCP channel |
-^ Type | TCP Printer (Event Source) |+^ Disabled | False |
 ^ TCP Port | 9000 | ^ TCP Port | 9000 |
 ^ Message Delimiter | New Line {\n} | ^ Message Delimiter | New Line {\n} |
Line 61: Line 74:
  
 ^ // Fast TCP Out// ^^ ^ // Fast TCP Out// ^^
-^ Description | Immediate data transfer to the Business Network | 
 ^ Type | TCP Event Sink (Event Sink) | ^ Type | TCP Event Sink (Event Sink) |
 +^ Description | Immediate data transfer to the Business Network |
 +^ Disabled | False |
 ^ TCP Server Host | [ IP Address of server hosting "Fast TCP In" ] | ^ TCP Server Host | [ IP Address of server hosting "Fast TCP In" ] |
 ^ TCP Server Port| 11000 | ^ TCP Server Port| 11000 |
Line 71: Line 85:
  
 ^ //Big Data Sink// ^^ ^ //Big Data Sink// ^^
-^ Description | Save data to the Big Data Store | 
 ^ Type | Big Data Event Sink (Event Sink) | ^ Type | Big Data Event Sink (Event Sink) |
 +^ Description | Save data to the Big Data Store |
 +^ Disabled | False |
 ^ Big Data URL | http://localhost:9200 | ^ Big Data URL | http://localhost:9200 |
 ^ //Subscribes to:// | TCP Printer Stream | ^ //Subscribes to:// | TCP Printer Stream |
  
 ^ //Big Data Source// ^^ ^ //Big Data Source// ^^
-^ Description | Retrieve collected data from Big Data Store | 
 ^ Type | Big Data Event Source (Event Source) | ^ Type | Big Data Event Source (Event Source) |
 +^ Description | Retrieve collected data from Big Data Store |
 +^ Disabled | False |
 +^ Paused | False |
 ^ Big Data URL | http://localhost:9200 | ^ Big Data URL | http://localhost:9200 |
 ^ Index Filter | tcpprinterstream.evt_* | ^ Index Filter | tcpprinterstream.evt_* |
Line 85: Line 102:
  
 ^ // TCP Out// ^^ ^ // TCP Out// ^^
 +^ Type | TCP Event Sink (Event Sink) |
 ^ Description | Resilient data transfer to the Business Network | ^ Description | Resilient data transfer to the Business Network |
-^ Type | TCP Event Sink (Event Sink) | +^ Disabled | False | 
-^ TCP Server Host | [ IP Address of server hosting "Fast TCP In" ] |+^ TCP Server Host | [ IP Address of server hosting "TCP In" ] |
 ^ TCP Server Port| 11000 | ^ TCP Server Port| 11000 |
-^ Username | [Service account with access to server hosting "Fast TCP In" ] | +^ Username | [Service account with access to server hosting "TCP In" ] | 
-^ Password | [Service account password with access to server hosting "Fast TCP In" ]   |+^ Password | [Service account password with access to server hosting "TCP In" ]   |
 ^ Check Response | Yes | ^ Check Response | Yes |
 ^ //Subscribes to:// | Big Data Source | ^ //Subscribes to:// | Big Data Source |
  
  
-===== Data Relay across 3-Zone Network Architecture =====+==== 2. BN: App Store Connect ====
  
-The following example illustrates relaying data across a 3 zone network architecture. In each zone a Data Core Node is installed.+For detailed instructions on how to deploy App Store Connect, see: [[data_core:how_to_connect_your_data_to_the_app_store]]
  
-{{:alarm_analysis:resilientdatatransfer2.png|}}+Configure the following Data Core components (assume default settings unless otherwise stated):
  
-Alarm & Event data enters Data Core Node 1 via a serial printer-port cable connected to a Serial to Ethernet Converter. This physical architecture guarantees one-way data-flow from Process Controllers to Data Core.+^ //TCP In// ^^ 
 +^ Type | TCP Event Source (Event Source) | 
 +^ Description | Receive data from Process Information Network  | 
 +^ Disabled | False | 
 +^ TCP Server Port| 11000 |
  
-A **Fast Flow** relays data across the zones via a fire-and-forget process providing near-real-time data on the destination server.+^ //Big Data Sink// ^^ 
 +^ Type | Big Data Event Sink (Event Sink) | 
 +^ Description | Save data to the Big Data Store | 
 +^ Disabled | False | 
 +^ Big Data URL | http://localhost:9200 | 
 +^ Big Data Refresh Interval | 5s | 
 +^ //Subscribes to:// | TCP In |
  
-On a separate **resilient flow** the database on the source server is poled and new data relayed via a guaranteed delivery process. This provides a complete data record on the destination server. 
  
-The database on the destination server consolidates data from both flows.+==== 3. Firewall Requirements ====
  
-===== Data Core Installation ===== +Intelligent Plant software is firewall friendly and designed to operate across firewall protected network zones.
  
-Stand-Alone installations of Data Core are required on each domain traversed.+^ Firewall ^ Requirements ^ 
 +| BN:Internet Network Firewall | TCP Port 443 open to outbound traffic from computer hosting App Store Connect and user machines to: \\ \\ https://appstore.intelligentplant.com \\ https://login.microsoftonline.com * \\ \\ * Required for Azure Active Directory log-in. \\ \\ For instructions on enabling log-in to the App Store with business accounts, refer to: App Store Registration for Organisations. | 
 +| Computer hosting App Store Connect | Windows Firewall TCP Port 443 open to outbound traffic \\ TCP Port 11000 open to inbound traffic | 
 +|PIN:BN Network Firewall | TCP Port 11000 open to outbound traffic | 
 +| PCN:PIN Network Firewall | No inbound access required. |
  
-If the intention is to make data available to the App Store, the destination server could be an "App Store Connect" Data Core instance.+==== 4. Alarm Analysis Processing ====
  
-<code> +So far, we've moved Alarm & Event data across a network. We are now ready to configure Alarm Analysis processing.
-App Store Connect is an instance of a Data Core Node pre-configured for secure connection with the Industrial App +
-Store. +
-</code> +
- +
-===== Data Core Configuration =====  +
- +
-Configure the following Event Source and Event Sink components and subscriptions. +
- +
-{{:alarm_analysis:resilientdatatransfer7.png|}} +
- +
-^ //TCP Printer Stream// ^^ +
-^ Type | TCP Printer (Event Source) | +
-^ TCP Port | 9000 | +
-^ Message Delimiter | New Line {\n} | +
-^ Maximum Characters per Scan | 4000 | +
- +
-^ //Fast TCP Out// ^^ +
-^ Type | TCP Event Sink (Event Sink) | +
-^ TCP Server Host | [ IP Address of server hosting "Fast TCP In" ] | +
-^ TCP Server Port| 11000 | +
-^ Username | [Service account with access to server hosting "Fast TCP In" ] | +
-^ Password | [Service account password with access to server hosting "Fast TCP In" ]   | +
-^ Check Response | False | +
- +
-^ //Retrieve Data// ^^ +
-^ Type | Big Data Event Source (Event Source) | +
-^ Big Data URL | http://localhost:9200 | +
-^ Index Filter | tcpprinterstream.evt_* | +
-^ Sleep Period | 30 | +
-^ Lag | 60| +
- +
-^ //TCP In// ^^ +
-^ Type | TCP Event Source (Event Source) | +
-^ TCP Server Port| 11000 | +
- +
-^ //TCP Out// ^^ +
-^ Type | TCP Event Sink (Event Sink) | +
-^ TCP Server Host | [ IP Address of server hosting "Resilient1TCP In" ] | +
-^ TCP Server Port| 11000 | +
-^ Username | [Service account with access to server hosting "Resilient TCP In" ] | +
-^ Password | [Service account password with access to server hosting "Resilient TCP In" ]   | +
-^ Check Response | True | +
- +
-^ //Store Data// ^^ +
-^ Type | Big Data Event Sink (Event Sink) | +
-^ Big Data URL | http://localhost:9200 | +
-^ Big Data Refresh Interval | 5s |+
  
 +For more info, see [[alarm_analysis:how_to_configure_an_alarm_event_import_stream|How to configure an Alarm & Event Import Stream]].
  
-===== Firewall Requirements =====  
  
-Network and server firewalls must be open for TCP traffic on the ports configured above. 
  
alarm_analysis/resilient_data_transfer.txt · Last modified: 2023/12/29 13:41 by su