Enterprise Service Bus Introduction to Safe Softwares ESB

Enterprise Service Bus Introduction to Safe Software’s (ESB) FME® Prepared by Darren Fergus

Why FME – What are the issues? Organisational Problems 101 ￮ P 2 P integrations: ￮ Absolute Nightmare to manage – typically bespoke solutions consisting of custom script (different languages C#, Python and whatever else!), are hard to maintain integration, have version control issues and or changes in apis, or lack of documentation both api and scripts, reliance on skilled workers and or held ransom by contractors ￮ By product - Lack of full integration tends to lead towards work arounds = Data Silos, a generally a rather messy and unreliable IT infrastructure that provides inaccurate/poor data, is costly and time inefficient

FME - What is an ESB? ￮ ESB - Allows different applications (CRMs, Asset Management Systems etc. ) to pass information bi-directionally with each other by acting as a transport system of data between applications within an enterprise environment and or worldwide web. This process is commonly known as middleware ￮ Required FME Software Elements ￮ FME Desktop (Workbench) ￮ FME Server FME provides the interoperability, integration, automation, synchronisation and data orchestration that provides the ESB (Enterprise Service Bus) functionality which is a key requirement in provisioning a Publish and Subscribe enterprise model.

Quick Recap - FME Desktop & Server FME Desktop – For Authoring ￮ ￮ ￮ Capable of multiple Integrations, Transformations and Translations or combinations of such Capable of reading a multitude of data sources at anyone time – Cad, esri formats, tables, Rest, Soap – Everything is ultimately resolved to Tabular level ￮ The POWER of the TRANSFORMER (Content and or Structural Transformations) ￮ Transform GIS, Tables, Json, XML, Imagery, 3 D, Exposing Attributions, Lists……. ￮ Data Integrity – Duplication Evaluation – Update, Inserts, Deletes – Multiple sources evaluated to a single source of truth ￮ Web Connections, Database Connections – Out the box functionality, One. Drive, AWS, Sharepoint, Google, Esri AGOL/Portal and so on, > 400 + Integrations out the box. FME Desktop a Client of FME Server or Cloud

Quick Summary Cont. - FME Server or Cloud ￮ FME Server – Enterprise Enabled Technology ￮ Enterprise solution for taking authored workspaces to the next level with Enterprise type Capability ￮ Workspace - Integrations and or tools for transforming/translating data can now be FULLY AUTOMATED ￮ Automations with Triggers – by a schedule, FTP, Email, folder watcher, topics, other protocols ￮ Full Orchestration, in series or parallel or both ￮ Server Apps (Secure Custom Apps) ￮ Security and fully accessible REST API! ￮ Versioning

FME Server Cont. ￮ Enterprise Level Solution, either: ￮ On Premise ￮ Virtualisation - Azure, AWS etc. ￮ FME Cloud, SAAS (Safe Software) ￮ Provides the necessary Job Scheduling, Job Management, Notifications, Repository Management, Load Balancing, Engine Management, Data Streaming etc. for those Workspaces that have been published to FME Server.

Demo – Dial Before You Dig Demonstration will include the following FME Server ￮ Dial Before You Dig - A working demo showing a real world scenario, “Dial Before You Dig” Email & Triggers -> Workspace & FME Server Automations (+ Auto Writer) –> E 2 E automated process -> Output Email + Dynamic Map ￮ Disseminating the functionality of the demo Automation: ￮ The Automation – Breaking down the relationship of internal elements of the automation ￮ FME Workspace power – Dynamic vision, building a map on the fly! ￮ Persistent Data –How automations has the ability to pass data upstream, allowing it to persist until it’s not required

FME – So what’s required? Key Workspace ESB Components (Desktop - Integration) Transformers ￮ HTTPCaller - The HTTPCaller is used for any system integration and is easy to use and configure. This transformer now incorporates the ability to configure and choose the authentication method that is required. Both REST and SOAP protocols are supported, please see additional references for more information (API documentation would be required for key systems). ￮ FTPCaller - Supports ftp, ftps and sftp protocols. Reader/Writers (100’s of supported formats) ￮ Reader/Writers – Web Connection based, S 3, Google, Onderive, Share. Point, and or Database Connections - see Safe documentation for supported formats

FME – So what’s required? Cont. Monitoring (Files and Paths – FME Server) ￮ (FME Server) Directory and/or FTP Watch Publishers (Tools for Server) These modules are available for Server from SAFE and are used to monitor changes to files and folders either locally (Directory Watch) or remotely (FTP Watch). Topic Specific Notifications ￮ FME Server provides the ability to monitor specific Topics which can be used to: ￮ Email – Send status information to users or other systems; success, failure or other custom information based on specific Topics ￮ Topics can be used to activate triggers which launch other actions, see FME Server “Automations” https: //knowledge. safe. com/articles/87453/getting-started-withautomations. html

FME – What’s required? Cont. Communication Other ￮ Emailer – Uses SMTP to send emails that contain either HTML or Plain text and can also send multiple attachments ￮ Twilio - Text Messages/sms can be setup to be sent out as notifications as per requirement. There are other methods of providing sms within FME for example using email to text third party. Locus can also provide an SMS gateway service (at 25 c per message).

Extending FME - Other Languages ￮ Programming Languages Support ￮ Python. Caller – Calls any Python script and runs it within the Workspace (Used as a replacement for where a transformer does not exist) ￮. Net – Can be used however it will require the SDK and additional plugins, see additional references ￮ TCLCaller - Utilises TCL (Tool Command Language). FME will wait for Parameters to be returned ￮ System. Caller – Will run a program or operating system command wait for that to close before continuation of translation ￮ Rcaller - Executes an R script that has the ability to access feature data from a temporary R data frame. ￮ See FME Documentation for full list.

Additional Resources & Reading Safe Software Presentations https: //www. safe. com/presentation/making-fme-your-esb/ https: //www. safe. com/webinars/integrating-enterprise-event-driven-messaging-using-fmeserver-esb/ User Stories (2019 World Tour) https: //www. safe. com/presentation/fme-esb-township-langley/