Building Open Scalable Multi Site Enterprise Architectures Travis

Building Open & Scalable Multi. Site Enterprise Architectures Travis Cox Co-Director of Sales Engineering, Inductive Automation

Agenda • What does an Enterprise architecture look like? • • • Site/plant • Remote locations • Corporate, DMZ • Cloud Enterprise challenges Goals & key factors Understanding your requirements, objectives, and network Building an enterprise architecture • Configuration, best practices, security

Enterprise Architecture

Enterprise Challenges • • • Amount of devices & data Faster rates Loss of communication, slow communication, high latency Maintaining local control Centralizing all data (real-time & historical) Security Management Scalability Business demands (data, machine learning, analytics, cloud) and more…

Today’s Goals Unlimited Possibilities • • Understand Ignition’s products, modules, and features Provide examples Provide tools and best practices Provide tuning tips

Key Factors • • Requirements Configuration & design Data flow Bandwidth Network latency Security Administration

Understanding Requirements, Objectives, and Network • Understand requirements at all levels (machine, site, corporate, cloud) • What functionality do I need locally? Centrally? Cloud? • Do I need redundancy? • Understand minimum requirements for Ignition • CPU, Memory, Disk, NIC • Physical vs. Virtual • Understand network (architecture, bandwidth, latency, firewalls) • Purdue model, DMZ • Understand all connections and data flow • Outbound/inbound, firewalls, ports, protocols

Building an Enterprise Architecture

Site / Plant Components 5 Critical Components • • • Ignition Edge Ignition’s Gateway Network MQTT Critical Asset Redundancy

What is Ignition Edge? Easily Extend Ignition to the Edge of Your Network

What is Ignition Edge? Edge Panel Edge Enterprise Edge MQTT

Ignition Edge Features & Benefits • Access data from PLCs & OPC-UA servers • Features unlimited tags (as of 7. 9. 9) • Equipped with OPC-UA, including Modbus, Siemens, and Allen-Bradley drivers (Other Ignition-supported drivers, such as DNP 3, can be added onto Ignition Edge for an additional cost) • Work on Windows (any version), and OSX, Linux & more • Work seamlessly with Ignition systems

Site / Plant Components Ignition Edge • Ignition Edge Enterprise • Ignition Edge MQTT • Ignition Edge Panel

What is Ignition’s Gateway Network? The Gateway Network allows you to connect multiple Gateways together over a wide area network, and opens up many distributed features between gateways. The Gateway Network provides the following features: • Web sockets provide fast, firewall-friendly 2 -way communication over a single configured connection • Setup proxy node • Security and SSL • Remote tags, history, alarming, and EAM

Gateway Network Setup Gateway Network • • • Just for Ignition Outbound connection Bi-directional Web sockets RBE Secure (port 8060)

Gateway Network Setup

Tip #1: Name Ignition Servers • Name each server uniquely and properly • Used to identify servers for tag history and Gateway Network services • Important for remote services & EAM • Configure names before setting up tag history or Gateway Network

Tip #2: Connect Up • Connect local servers to central servers • Easier to open firewalls on central servers vs. local firewalls

Tip #3: Connect Only to Master • Only connect to master node of redundant pair • Connection is aware of both servers • Don’t make 2 outgoing connections from the local server

Gateway Network Services • • Remote tags Remote alarm notification Remote history Enterprise Administration Module (EAM)

Gateway Network Services: Remote Tags • Tags exist on local Gateway • Setup remote tag provider on higher level server • Real-time status and control • Alarm status & acknowledgement • Query historical data • Only subscribes to tags needed • Remote tag management

Gateway Network Services: Remote Tags

Tip #4: Name Real-time Tag Providers Properly • Never use “default” • Give proper names for each Ignition server • Make sure names are unique across all Ignition servers in the enterprise • Make sure the remote tag provider has the same name edge 1 (local) edge 1 (remote)

Tip #5: Use Fully Qualified Tag Paths Real-time Tag Binding: [edge 1]path/to/my/tag History Tag Path: [splitter/ignition-systemname: edge 1]path/to/my/tag edge 1 (local) edge 1 (remote)

Gateway Network Setup edge 1 (local) edge 1 (remote)

Tip #6: Use Subscribed Mode for Alarms • • Alarms held in memory Better performance Heavier on memory Lighter on bandwidth (WAN) • Configured on remote tag provider

Remote Tag History Querying Gateway Network • Queries through Gateway Network • Heavier on bandwidth (WAN) • No need to mirror data

“Gateway Network” History Access Mode

Tip #6: Remote Tag History Querying Database • Queries from local database • No bandwidth (WAN) • Requires mirroring or replication • Specify remote driver and provider

Tag History Splitter • Mirrors tag historian data to 2 databases at the same time • Both connections go through store & forward • Local database should be specified first • Ability to query local database first for specific amount of time • Keep local database small

Gateway Network Services: Tag History Splitter

Tip #7: Use “Database” History Access Mode

Gateway Network Services: Remote Alarm Notification

Gateway Network Services: Remote Alarm Notification A single remote alarm notification profile unlocks 2 features • Local pipeline, remote alarm notification profile on notification block • Send alarm to remote pipeline directly • All remote pipelines visible in alarm configuration

Gateway Network Services: Remote History • Store history on central database • No local database required • Store & Forward • Compresses data over Gateway Network • Ignition Edge Enterprise = 1 week history buffer

Gateway Network Services: Remote History

Remote Tag History Bandwidth & Latency Concerns

Tip #8: Remote Tag History Bandwidth & Latency Concerns • If latency is high increase write size and write time • Slower connections = send more data slower • Don’t send data faster than latency time • Configured on store & forward connection

Gateway Network Services: EAM Manage multiple Gateways from one Gateway. Use the Controller Gateway to coordinate and automate many administrative tasks for Agent Gateways, including: • • • Monitor Agent health and performance Automate Gateway backup and recovery Synchronization projects and resources Deploy modules Central licensing Remote upgrades

Gateway Network Services: EAM

Gateway Network Services: EAM Agents Controller Agents Proxy through Gateway Agents

Tip #9: Best Practices for Security • Use HTTPS/TLS for everything • Gateway Network (use SSL, Approved. Only connection policy) • Security Zones (lock down access by IP or hostname) • Security Policies (tag access, alarm acknowledgement, tag history) • Gateway/Project Role-based Policies

Gateway Network Security

Gateway Network Service Security Lock down: • Tag Access / Management • History Access / Storage • Alarm Notification • Alarm Status (ack, shelve)

Non Gateway Network Services • Alarm history (journal) • Audit logs • Transaction groups Requires direct database access from remote site (highly requested feature)

What is MQTT? Message Queuing Telemetry Transport MQTT is a machine-to-machine (M 2 M) data transfer protocol that is quickly becoming the leading messaging protocol for the Industrial Internet of Things (IIo. T)

MQTT Architecture

Why MQTT? • • • Decouples devices from applications Low bandwidth Report by Exception (RBE) TLS security (port 8883) Access Control Lists (ACLs) Outbound connection only (no inbound firewall rules) Stateful awareness Single source of truth Plug and play functionality Eliminates cutovers (parallel applications)

Leading Protocol

MQTT = Future Proofing Vibrant ecosystem • Gateways • Sensors • Applications • RTUs, PLCs • Remote I/O

MQTT Sparkplug Specification Sparkplug is a specification that defines how to use MQTT in a mission critical, real time environment. • Eclipse Tahu Project • Defines MQTT Topic Namespace • sp. Bv 1. 0/group/DDATA/edgenode/device • Defines MQTT Payload Definition • Defines MQTT State Management • High Availability / Redundancy / Scale

Why MQTT & Gateway Network? Gateway Network MQTT • Ignition only • Management (projects & tags) • EAM • Remote history (select tags) • Remote alarm ack & notification • Open standard • Decouple devices from applications • Future proofing (ecosystem) • Access to 3 rd party • Tag exists centrally • Get data to cloud

Building an Enterprise Architecture

Frontend Gateways & Load Balancer • Hardware or software (F 5 Load Balancer) • Turn on sticky sessions • No state (memory tags, alarms, SFC engines, timer scripts, etc. ). Requires dedicated server for that. • Get data from I/O servers through Gateway Network and SQL databases • Handle authentication through shared authorization such as Active Directory or federated identity.

Building an Enterprise Architecture

Building an Enterprise Architecture

Questions?

Thank You