AWIPS Continuous Technology Refresh CTR AWIPS Software CTR

AWIPS Continuous Technology Refresh (CTR) AWIPS Software CTR ADE System Architecture September 4, 2008 This document includes data that shall not be duplicated, used, or disclosed – in whole or in part – outside the Government for any purpose other than to the extent provided in contract DG 133 W-05 -CQ-1067. However, the Government shall have the right to duplicate, use, or disclose the data to the extent provided in the contract. This restriction does not limit the Government’s right to use information contained in this data if it is obtained from another source without restriction. The data subject to this restriction are contained in all sheets.

AWIPS-II: Architecture Topics n n n n n Architecture Introduction Architecture Overview Architecture of SOA Service Architecture Patterns – Plug In pattern – Build pattern, Localization Pattern – u. Engine pattern, GIS capability, Adapter pattern – Notification/Subscription pattern Data Model Visualization Server Execution Hardening Security 1/5/2022 Page 2

System Concept: AWIPS-II Architecture Data Fusion and Visualization SOA Framework n Architecture Introduction Requirements Vision Drives Architecture – Focus on “ilities” drives new architecture – Features and capabilities get generalized into reusable patterns – Customer TIMs give priority to capabilities n Architecture Framework Vision: – Create a new low cost framework for hosting a full range of environmental services including thick client visualization – The framework will scale down to a small laptop and up to clusters of enterprise servers without software change – The framework will be based on highly reusable design patterns that maximize reuse, has datatype independence, and fast adaptability – Open source is leveraged to maximize reuse 1/5/2022 Page 3

AWIPS-II: Architecture Definitions n n n n Architecture Introduction ADE: AWIPS Development Environment, source code to execution framework enterprise development kit including tools SOA, End points, I/O Routing, Transforms: Service Orientated Architecture where system capability is available at stateless endpoints Canonical XML: Well formed XML that follows high level rules Patterns: Implements a design solution that solves a problem that occurs many times Technical Reference Architecture: A physical software execution framework JMS, JMX: Java Messaging System (API), Java Management Extensions CAVE: Common AWIPS Visualization Environment using Eclipse RCP SEDA: Serial Event Driven Architecture 1/5/2022 Page 4

AWIPS-II Architecture Concept – Architecture Framework Implementation n Architecture Introduction Framework Implementation: Integrated several best of breed open source projects with a set of advanced enterprise patterns to create a highly extendable framework. – Patterns Implemented in Pure Java code (Reuse Example: Product. Srv and Auto. Bld. Srv use u. Engine) – Open Source is primary for re-use: l ~15 major open source projects integrated l Version controlled with CM baseline, libraries part of run env. l Leverage Internet community for core infrastructure l Standards compliant, rapid evolution l Free, large body of public expertise l Open source libraries controlled by putting them in the CM compile library and deploying them to the runtime environment – Packaged together into the ADE which contains everything from the Source Code repository to the execution environment including operator Clients 1/5/2022 Page 5

AWIPS-II: Problem Definition & User Needs Architecture Introduction Increasing Data Interactions Drives Architecture Towards Advanced Solutions First Person Shooter Video Game Weather Edge User Visualization Totally Non-Propriety Solution (Open Source) Quickly Adaptable New Science & Data Types Accelerate Innovation Speed of Deployment Sat Com Google Earth Situational Awareness Visualization CAD Systems Con-Ops Flexibility 164 Unique Op Centers Increasing Data Rates High Data Rates With Gaming Style Interaction 1/5/2022 Page 6

AWIPS-II: Development Sequence, T 09 is GFE+ 3 to 6 month fixed price task orders 2007 2006 2008 Today 2009 Architecture Introduction 2010 AWIPS II Development T 03 T 04 T 05 T 06 R&D Clean Sheet Start Architecture Development 2 Week Cycles Stability Testing Full Data Load T 08 Wx Workstation T 09 Forecast Editor T 010 Hydrology AWIPS-II Release 1 T 011 Comm Deployment Spiral Development with Interim Deliveries 1/5/2022 Page 7

AWIPS-II: Architecture Topics n n n n n Architecture Introduction Architecture Overview Architecture of SOA Service Architecture Patterns – Plug In pattern – Build pattern, Localization Pattern – u. Engine pattern, GIS capability, Adapter pattern – Notification/Subscription pattern Data Model Visualization Server Execution Hardening Security 1/5/2022 Page 8

SOA Architecture – Logical Layered View Layers Separated By Simple APIs Architecture Overview Client/Presentation Services JMX Ingest. Srv Notify. Srv Product. Srv Archive. Srv Auto. Bld. Srv Adapter. Srv Index. Srv Subscribe. Srv Utility. Srv Staging. Srv Purge. Srv Data Access Layer <<Java>> HDF 5 Data. Store <<Java>> Data. Layer HDF 5 API <<abstract>> Base. Dao Data Persistence Store External Programs Hydro Models LAPS FORTRAN/C/C++ Command Line Programs Hibernate Platform Layer HDF 5 Security Services /Demilitarized Zone (DMZ) Vtec. Srv Mbean Enterprise Service Bus - Communication Mission Services Layer CAVE Localization Store Spatial Index Postgre. SQL Metadata Index Synchronous Interface 1/5/2022 Page 9

AWIPS-II: ESB SOA Framework Architecture Overview Generic Core Services Adapt Through Plug Ins Core Base of Services l Micro Engine l Plug-in Framework l Core libraries l <<EDEX Framework Architecture>> Services Structure Extend to a specific set of services l Plug in specific libraries l Plug in data types, transforms & scripts l + libraries +core plug-ins +Ingest Data +Disseminate +Index Data +Store Data +Transform Data +Build Products +Collaboration +Fusion Engine <<Technical Implementation>> Concrete Services based on Data Types +weather libraries +mapping libraries Meta Data Automatic Metadata Indexing l High Performance HDF 5 blobs l + libraries +core plug-ins <<Visualization Extension >> Data Types Transforms Data <<CAVE Visualization Framework >> Scripts +Visualize Data +Mapping Vis Plugins Vis Scripts Vis Localize Service Interface To Data l Clean separation between data and visualization l AWIPS-II Designed for Extension to New Domains 1/5/2022 Page 10

AWIPS-II: Plug In adaptable data fusion and data transformation framework with Visualization Plug In Data Adaptable Auto Metadata & Data Persistence Gaming Visualization Architecture Overview Configuration via XML CAVE u. Engine Data Transformer Machine to Machine Plug In Extendible Enterprise Service Bus Based on Advanced Design Patterns 1/5/2022 Page 11

AWIPS-II: Designed for reuse across multiple domains AWIPS-II Architecture Overview Extending to new domains • Add Data Type Plug Ins • Add Libraries • Add Scripts • Add CAVE Plug Ins Domain 1 Domain 2 Domain 3 *** Low Cost Extension to New Domains 1/5/2022 Page 12

AWIPS-II Dependencies: Open Source Based Free of all fee based COTS dependencies COTS Flexibility Architecture Overview Mule ESB + Spring Enterprise service bus and dependency injection container for SOA services Eclipse RCP Plug In driven visualization framework Geo. Tools Enables GIS capabilities and map projection framework JOGL Java API to Open. GL enables Gaming level visualization performance HDF 5 High performance file persistence of large data sets such as satellite and radar Java. Script + Python + num. PY Data transform scripting languages with high performance math library JAVA + ANT Primary programming language and software build framework active. MQ Java messaging provider with clustering Postgre. SQL Relational database for storing Metadata from Data plug ins and spatially enables ingested data 1/5/2022 Page 13

AWIPS-II High Level System Services Architecture Overview SOA Services Running in an ESB Container CORE SOA Services WAN Visible ADE 1. 0 Mbean Ingest. Srv Staging. Srv TAF Plug In METAR Plug In Radar Plug In Satellite Plug In GRIB Plug In Mbean Archive. Srv Index. Srv Purge. Srv File. System RDBMS via JDBC HDF 5 persistence Meta Data Index Spatial Data Base JMX Remote Service Management Client Mbean Product. Srv u. Engine Manage Subscription GFE IFPServer Mbean Adapter. Srv Exec Adapter JNI Adapter Mbean Auto. Bld. Srv Notify. Srv Subscription Notify u. Engine Utility. Srv Collaboration Eclipse config XML Localization Data Color Tables Maps and Topo CAVE Visualization Client Data Rendering CAVE Bundles Data Interrogation CAVE Procedures Wx Drawing Wx Warning Radar All Tilts Hydro Visualization Shapefile Visualization Mbean Vtec. Srv XMPP Enterprise Service Bus – HTTP, JMS, Virtual Memory, File Endpoints Services Independent of End Points 1/5/2022 Page 14

Requests Data for Display: as a GIS Layer Architecture Overview Layered View of SOA Service with Data Flow Client/Presentation Layer Native Services Layer Enterprise Service Bus – HTTP, JMS, VM, … Endpoints CAVE SEDA Service Scaling Within Container Product. Srv Input Message Canonical XML With body containing action commands Output Message Canonical XML Platform/Infrastructure Services Layer u. Engine Pattern make. Response file. In term. Query Data Access Objects Response to Client Containing URI references to data generated Platform/Infrastructure Resources Layer Persistence Repository HDF 5 Meta. Data Index 1/5/2022 Page 15

Ingest Data Flow: Layered View of SOA Service Architecture Overview Ingest at a Clustered End Point Platform/Infrastructure Resources Layer Enterprise Service Bus – HTTP, JMS, VM, … Endpoints /awips/opt/data/sbn/… /awips/opt/data/processing Native Services Layer JMS Broker SOA ESB Services Queue Clustering JMS: //cp/… File Name vm: //*Index. VMQueue Index. Srv Staging. Srv Ingest. Srv Platform/Infrastructure Services Layer File End Point Plugin(plugin. Type) Data. Layer +save. Data(record) +save. Metadata(record) HDF 5 Dao Data Access Objects Platform/Infrastructure Resources Layer Persistence Repository HDF 5 Meta. Data Index 1/5/2022 Page 16

AWIPS-II Alerting Pattern Update Architecture Overview Added Observer Interface to enable multiple users CAVE Decision. Tree CAVE Startup EDEX Quinlan ID 3 Algorithm Ingest. Srv Alert. Job. Starter Menu data. URIs Data Item Build Data URICatalog Message Received Index. Srv Menu. Updater Data URI List ingested data. URIs Met. Watch. Monitor IAlert. Observer Avn. FPS quartz Timer Data URI Aggregator topic: edex. alerts 1/5/2022 Page 17

AWIPS-II Data Cube Enhancements Redesigned data cube to enhance performance & flexibility HDF 5 Grid Repository Volume Browser Plan. View. Display Architecture Overview Request. Job Loader Time. Matcher Browser. Dialog Browser. Data. Catalog <Job> Data. Cube. Manager Data. Cube. Container purge. Data. Cube refresh. Derived. Parameters examine. Attributes get. Data. Record perform. Time. Query List Grid. Tree set|get. Grid. Tree Dynamically loaded Python Derived. Parameter. Generator Derived Parameter Deriv. Param. Scripts. Loader 1/5/2022 Page 18

AWIPS-II: Architecture Topics n n n n n Architecture Introduction Architecture Overview Architecture of SOA Service Architecture Patterns – Plug In pattern – Build pattern, Localization Pattern – u. Engine pattern, GIS capability, Adapter pattern – Notification/Subscription pattern Data Model Visualization Server Execution Hardening Security 1/5/2022 Page 19

SOA Service Pattern SOA Service Design Pattern n n SOA Service Vision: All server side capabilities are available as a standard service which communicate using standard network protocols and formats. Implementation: All services inherit from a single base class that encapsulates life cycle and communicate details. Services are Dependency Injection components that are configured to communicate with any supported endpoint. Abstract. Srv. MBean Abstract. Message. Srv +on. Call() +register() Hides Interface Details to Container Extracts message Calls +process() on return Reports exceptions Registers with JMX for Remote monitoring Ascii. Ingest. Srv. MBean Ascii. Ingest. Srv Does the work of the Service Return puts message on outbound endpoint +process() 1/5/2022 Page 20

SOA Service Pattern Dependency Injection - Minimizes Coupling n SOA Service Dependency Injection: All AWIPS SOA services get there interfaces defined by XML configuration which maximizes service reuse and minimizes hardcoded coupling. Service interface and configuration Files Generally One file for each service Service output endpoint Message sent when +process() executes return Service class file reference File endpoint wakes up Services’ +process() method when data is picked up 1/5/2022 Page 21

SOA Service Pattern SOA Service Container and ESB startup n Startup: ADE 1. 0 has all SOA services running in one container. The start script automatically builds the classpath from jars in lib directory. SOA Service Container/ESB start scripts start – linux start. bat -- Windows Sets clustering configuration Builds list of ESB End-point and Wiring Files Starts Mule ESB Service Configuration Files, Adding a service Includes adding a new service config file to the startup. Dependency Injection Container 1/5/2022 Page 22

Canonical XML Interface Pattern SOA Service SOA service interfaces n Vision: A simplified XML based message format that is independent of the endpoints. The message schema is flexible and is orientated towards the client. Example Response Message From Product. Srv Top Message Tag Message Header Tag With properties Message Body Tag Response by URI Reference Returned product in “png” image format 1/5/2022 Page 23

Canonical XML Interface Pattern SOA Service SOA service interfaces n Reponse: The body of the response is a island of XML, a base 64 encoded binary image, or a URI reference for large data sets. Example XML Response Abstract. Response. Message binding. xml -file. Type -data. URI -valid. Time Make. Response. Msg Response. Message. Error Response. Message. URI -product. URI Response. Message. ASCII -filename -contents -type -station -time Response. Message. Inline -file. Name -zipped. Data[ ] -error. Source -error. Msg -error. Cause -error. Chain 1/5/2022 Page 24

AWIPS-II: Architecture Topics n n n n n Architecture Introduction Architecture Overview Architecture of SOA Service Architecture Patterns – Plug In pattern – Build pattern, Localization Pattern – u. Engine pattern, GIS capability, Adapter pattern – Notification/Subscription pattern Data Model Visualization Server Execution Hardening Security 1/5/2022 Page 25

AWIPS-II: SOA Data Type Plug In Pattern Enables New Data Types and Transforms EDEX Services Ingest Index SOA Data Type Plug In Meta. Data Interface Decoder Interface HDF 5 Alert Topic Meta. Data Notify Plug In Pattern Storage Interface Schema Creator URI Generator XML Meta. Data Def Data URI Def Meta. Data Decoder Message Separator Data Decoder u. Engine Tasks u. Engine Interface Product Extreme Adaptability Through Plug In Pattern 1/5/2022 Page 26

SOA Data Plug In: Enables Extensibility Plug In Pattern Meta Data, Decode, Storage, Transforms Enterprise Service Bus Dependency Injection Container Enterprise Service Bus XML Msg <<SOA Service>> Product. Srv Jibx XML Msg Response Class Loader u. Engine <Task> Hibernate DAL JEBB Python Interface SOA Data Type Plug In <Task> Enables Extending the u. Engine Language Dynamic Meta Data Creation HDF 5 Persistence Dynamically Extends System to New Problems 1/5/2022 Page 27

Plug Ins Enable Adapting to New Data Plug In Pattern Automatically Creates Meta Data Persistance Reference Architecture • IProduct{Extract Metadata} • I{*}Record • IDecode • IStore • Task • Discovery Plug In Package (JAR) plugin. xml metadata. xml attributes. xml Implementation Classes Binding Classes u. Engine Task Implementations u. Engine Enables new science through new Tasks Enables new data to Be queried and visualized <<Base Table>> Meta. Data{Plug. ID} <<Table>> Spatial. Data <<Table>> Plug. In. Registry <<HDF 5 File>> Raw/Decoded[1] Table Cycle Trigger <<Java>> Data. URI <<Inherited Table>> Meta. Data{Plug. ID}1 <<Inherited Table>> Meta. Data{Plug. ID}n <<HDF 5 File>> Raw/Decoded[n] +create() Dynamically Extendible to New Data & Transforms 1/5/2022 Page 28

Plug Ins Enable Adapting to New Data Dynamically Extends to New Data & Transforms Plug In (i. e)plugin-airep ESB Endpoint Config Plug In Pattern Auto Generated HDF 5 schema for Blob data Metadata DB HDF 5 Lighting Radar Satellite Grid Auto Generates Schema Creates Data URI Specific Data Type Decoding Custom Data Transforms if necessary Custom Data Access Plug In Registry Table 1/5/2022 Page 29

SOA Plug In CM File structure SOA Data type Plug Ins Plug In Pattern Plug In XML and Configuration Meta data definition files Plug In Implementation Classes u. Engine Task Scripts Extends Script. Task 1/5/2022 Page 30

AWIPS-II T 09 Services and Plug Ins ESB SOA Plug In Adaptable T 010 Improve Gfe. Purge. Srv Smart. Init Staging Ingest Hdf. Http Purging Index VTEC Archive Notify Adapter Utility Auto. Bld Product GFE py Proxy AWIPS-II Services Airep (22 total) Binlightening Bufrmos 801 Bufrmos Bufrua ccfp 268 Gfe 10808 Goessounding 547 Grib modelsounding 846 Obs Pirep poessounding 598 profiler 560 Radar Recco redbook 730 Satellite Sfcobs shef 3973 Taf Text 678 warning 1115 T 09 Plug In Pattern Future Eclipse RCP Visualization Plug Ins 14 Core 8 User Interface 22 Library 14 Application GFE 26944 Redbook 619 Python Editor 122 Hydro 10039 aviation(avn. FPS) 11539 CAVE Thick /Light Client Mode Enterprise Service Bus SOA Data Type Plug Ins 1/5/2022 Page 31

SOA Plug In Tool Added to ADE Adding using the tool to a training module n Plug In Pattern New with ADE 1. 0: a Plug-in Creation tool. – The Plug-in Creation tool is implemented as an Eclipse plug-in. – Once installed, the Plug-in Creation tool may be used like any other element of the Eclipse IDE. n The Plug-in Creation tool automates most of the work involved in generating the initial files for a data-type plug-in. – Automatically creates the required directory structure for the plug-in. – Generates class stubs for the required Java classes. – Generates initial configuration and build files. 1/5/2022 Page 32

Software CM/Build/Deploy Pattern Design pattern Build Patterns Build Vision: Create a simple layered build system that manages component coupling and supports partial deployment. n Build Implementation: Implemented in ANT as a series of macros and ANT extensions. Jar Files for n Compiling -Open Source Jars deploy. xml test. xml build. xml config. xml build-global. properties compile. xml build-local. properties jibx. xml mortar. xml javadoc. xml 1/5/2022 Page 33

Software CM/Build/Deploy Pattern Design pattern (Server Side) Build Patterns CM Controlled Baseline (Subversion) 1) Build services & components ant build -Dcompfile=deployments/deployment. properties 2) Deploy to run time environment ant deploy Services and Components + Open Source Dependencies Special Jars Plugin Jar Files 1/5/2022 Page 34

Build Patterns Deployment Simplification Created a new GUI based installer n Standard Linux Server Install for Server Side pieces n Standard Linux Workstation Install for CAVE n Windows Workstation for CAVE n Procedure for clustering (Clustering Con-ops? ) n Linux Workstation • CAVE • Basemaps • Jdk • Topo Remote Mount Linux Server 1 • Edex server • Jdk • Postgre. SQL • Eclipse 3. 3 • ANT 1. 7 • Edex source code • Edex Javadoc Win Workstation • CAVE • Basemaps • Jdk • Topo Data Partition 1/5/2022 Page 35

AWIPS-II Deployment Simplification n Build Patterns CAVE application and EDEX services installed using two separate installers – Each installer can install files to the user’s home directory. 1/5/2022 Page 36

AWIPS-II: Localization Pattern Enables SOA customization of configuration n Localization Pattern Localization procedure occurs at startup – CAVE localization is controlled through Eclipse Preferences – Extended Eclipse Preferences to use a XML data store Different approach to data since CAVE can work with large data sets less subsetting is required n Localization is provided through two simple, unified interfaces n – Configuration for the server – Localization for the client, with server synchronization capability n Localization provides a multi-tiered configuration – Base (standard national configuration values), Site for Server (any values the site chooses to override) – Base, Site and User for Client (values that the user overrides) 1/5/2022 Page 37

Workstation (CAVE) Localization Pattern Uses Utility. Srv with a HTTP interface ESB Sync Protocal Log In User Name Determines User Context Site Through Perferences Localization Pattern <<SOA Service>> Utility. Srv CAVE Config At CAVE Start Up CAVE Menus Localization Hierarchy Base Local Persistence Of Current Active Localization Color Maps Site User 1/5/2022 Page 38

Localization Data – Hierarchical Preferences, Menus, Data … Localization Pattern Base: <configuration> <texture. Card. Preference>128</texture. Card. Preference> <texture. Memory. Preference>384</texture. Memory. Preference> <frames. Per. Second. Preference >25</frames. Per. Second. Preference > <tile. Boundaries>false</tile. Boundaries> <connection. Method>jms</connection. Method> <jms. Server. Address>tcp: //localhost: 61616</ jms. Server. Address> <data. Directory>/awips/opt/data/hdf 5</data. Directory> <font. Magnification>1. 0</font. Magnification> </configuration> Site: <configuration> <site. Name>KOAX</site. Name> <site. Full. Name>Omaha</site. Full. Name> <site. Type>WFO</site. Type> <data. Directory>/oax-awips/opt/data/hdf 5</ data. Directory> </configuration> User: <configuration> <font. Magnification>1. 25</font. Magnification> </configuration> 1/5/2022 Page 39

u. Engine Task Execution Pattern Breaks up execution into small reusable tasks u. Engine Pattern u. Engine Vision: Create an execution framework for generating custom SOA requests on-demand. Customer systems can request products by script requests over a network. The script performs small general units of work that get chained together to produce a customer product. n u. Engine Implementation: Runs out of a SOA service “Product. Srv” that is attached for I/O to ESB endpoints. The u. Engine executes scripts in a standard scripting language that has been extended. Current scripting language is Java. Script and transitioning to Python with Num. Py for improved algorithm performance. n 1/5/2022 Page 40

AWIPS-II: u. Engine Design Pattern u. Engine Pattern Executes Commands to Transform/Fuse Data Service Container Enterprise Service Bus Inbound Endpoint XML Msg <SOA Service> Product. Svc Meta Data u. Engine term. Query Outbound Endpoint XML Msg Response HDF 5 Persistence query. Field query. Text F 01 DE 29344 D 222234343… decode SOA Plug. Ins Contains Extensions reproject color. Image image. Out IR Color Geo. Tiff <<abstract>> Script. Task +init() +object execute() Service. Response u. Engine Enables Flexible User Requests/Response 1/5/2022 Page 41

u. Engine Scripting Changed to Python Enables common scripting language, and numpy benefits u. Engine Automatic Script Generation Users IFPClient Plot. Model. Generator u. Engine Pattern (i. e) Volume Browser Text Workstation Warn. Gen u. Engine Script Catalog. Query Script. Creator Py. Product. Srv Jep. Pool Script. Factory Jep execute. Script Velocity Template Engine Velocity Template Build Response 1/5/2022 Page 42

SOA Plug In: u. Engine Tasks Extensions Eclipse IDE Views of Plug In Implementations u. Engine Pattern Satellite Plug Ins Each gets built & Packaged in a separately Deployable Jar Note: Each Plug In Has a u. Engine Task Package Grib Plug In Radar Plug In 1/5/2022 Page 43

u. Engine Task Execution Pattern Enables highly flexible SOA request/response u. Engine Pattern Metadata Fields Defined by SOA Plug In <<SOA Service>> Product. Srv +receive() <<Java Class>> UEngine. Script +run() +setup. Subscription(script) <<Java Class>> UEngine. Util +convert. From. Js. To. Java() Provides Interface To Scripting Language Output Product in Response by Reference 1/5/2022 Page 44

u. Engine Scripting Overview u. Engine Pattern Enables SOA scripting of scientific algorithms Goal: Provide u. Engine scripting in a language suited for scientific algorithms while enabling high performance n Uses standard Python with JEBB for a bridge to Java n – Uses “num. Py” with it’s numerical data types and functions – Includes interface to Fortran library “Linpack” for linear algebra – Includes interface to “BLAS” for taking advantage of multiple processing chains in modern CPUs for improved performance Uengine. Script class, simply call set. Script. Text(String java. Script. Code) and run() n Extend Script. Task, Basic POJOs with an execute() method n Tasks only operate on one object (loops at the script level) n 1/5/2022 Page 45

Product. Srv Changes Java. Script u. Engine Language & DB for Subscribe u. Engine Pattern <<SOA Service>> Product. Srv u. Engine. Script(script) . run() Subscribe Y subscribe(sub. Script, data. URI, script. ID) N Subscribe. DAO postgre. SQL Meta. Data script subscription End 1/5/2022 Page 46

AWIPS-II GIS Capabilities – Geo. Tools GIS Data Types, Transforms, and Display n GIS Capability Data. Store and Data Formats – ESRI Shapefile (Read/ Write) – WFS Web Feature Server (OGC) (Read / Write) – Post. GIS geometric objects for Postgre. SQL – GML Geography Markup Language – Geo. TIFF geo-referenced TIFF image n Coordinate Transformation (Open. GIS CTS) – Map Projections (Mercator, Transverse Mercator, Lambert Conformal Conic, Albers Equal Area Conic, Stereographic, Orthographic, …) – Math Transform 1/5/2022 Page 47

Map Library Capability – Map reprojection Transform Ingested Data to Display Projection GIS Capability • Geo. Tools builds a transformation Matrix that gets reused in transform operations. • The Transformation matrix can be build up through a series of transforms. • Used In Barnes Analysis Pattern to calculate distances. • Degrib Task uses Geo. Tools to dynamically define projection data (I. e. NCEP hur. ) <<Java abstract Class>> Task +execute() <<Java abstract Class>> Shapefile. Query +execute() <<Java abstract Class>> Shapefile +execute() <<Java Class>> Reproject. Image Coordinate. Reference. System Grid. Geometry 2 D Buffered. Image Geo. Tools +execute() {Math. Transform …} Reprojected. Image 1/5/2022 Page 48

Map Libraries Through Geo. Tools + JTS Topology Suite Spatial Operations Java API GIS Capability Computing Spatial Relationships • Shape File Intersect Demonstration • Distance Tool Demonstration Overlay Operations Buffer Operations Polygonization & Quantization 1/5/2022 Page 49

Spatial Table Refactor – Geo Spatial Postgis extension to Postgre. SQL n GIS Capability Geo Spatial Enabling Data – Chosen Approach: Create Static Spatial Tables in Postgre. SQL l Post. GIS extension: Free, simple, high performance – Input / Output: l l All ingested data is spatially indexed and can be spatially queried Can create Shape and Geo. Tiff output Postgre. SQL Data Base radar_spatial_grids spatial_obs_stations spatial_satellite 1/5/2022 Page 50

Legacy Adapter Pattern SOA enable legacy application functions n Legacy Adapter Vision: Enable command line functionality to be brought in the SOA service flow Command Line Application Adapter. Srv Stream. Handler +start() stream output exec with arguments -process() Program. Output Canonical XML Message Command & Arguments Adapter Srv Response Enterprise Service Bus – HTTP, JMS, Virtual Memory, File Endpoints 1/5/2022 Page 51

Legacy Adapters Pattern Adapter. Srv Receives XML messages of commands Xml Msg Command Name Arguments[ ] JIBX unmarshall XML <<Java>> Command +name +arguments[ ] Adapter. Srv <<Java>> Abstract. Msg. Srv +on. Call() +process() <<Java>> Adapter. Srv +process() Adapter Pattern Xml Msg Command output <<Java>> Program. Output +to. XML() JIBX <<Java>> Process. Builder +start() +get. Input. Stream() Command Line Process Enables Command Line Process to run as SOA Service 1/5/2022 Page 52

Legacy Adapters Pattern u. Engine Runs Command Within Action Script Execute. Command Adapter Pattern Name arguments <<abstract>> Task +on. Call() +process() <<Java>> Execute. Command Adapter. Srv +execute() Xml Msg Command JIBX marshall XML <<Java>> Command +name +arguments[ ] u. Engine Can Run Command As If It Was Part of the Script 1/5/2022 Page 53

Data Notification / subscription Enable Automation of Products and Clients Subscription Pattern Notification / Subscription Vision: Automate product generation and client product display based on new data ingested. n Implementation: Any u. Engine script can be made into a subscription based on ingest of new data or a time schedule. n – New data triggers the execution of the cached u. Engine script – The resulting product response is placed on a JMS Topic 1/5/2022 Page 54

CAVE Data Notification + Subscription Data Matched to Menus by Optimized Decision Tree CAVE Decision. Tree CAVE Startup Quinlan ID 3 Algorithm Alert. Job. Starter Subscription Pattern EDEX Data Item Ingest. Srv Build Data URI Menu data. URIs URICatalog Message Received Index. Srv Alert. Manager. Job Data URI List ingested data. URIs EDEX Auto. Bld. Srv Script Ids post. Gre. SQL topic: edex. subscription Table: Subscription Table: scripts Notify. Srv quartz Timer Data URI Aggregator topic: edex. alerts Check scripts for URIs 1/5/2022 Page 55

Alerting Pattern Based on Ingest Warning GIS located on Map and Displayed Tree Subscription Pattern ascii File: //. . /data/sbn/ascii ESB Reg. Ex. Filter Fan Out Routing Threaded Listener Alert. Manager. Job warning jms: //cp/tstormwarning Locate Zooms In To Warning Area on Map 1/5/2022 Page 56

Repository Refinement n Subscription Pattern Subscription added to Postgre. SQL data base – Server state data simplifies clustering Configurable Purge Retention Period n Performance enhancements with indexes on metadata n Postgre. SQL Data Base scripts subscription 1/5/2022 Page 57

AWIPS-II: Architecture Topics n n n n n Architecture Introduction Architecture Overview Architecture of SOA Service Architecture Patterns – Plug In pattern – Build pattern, Localization Pattern – u. Engine pattern, GIS capability, Adapter pattern – Notification/Subscription pattern Data Model Visualization Server Execution Hardening Security 1/5/2022 Page 58

ADE Data Model Introduction Canonical XML SOA Interfaces Excluded Data Model Data Access Layer Implementation Using Hibernate n Data Access Object (DAO) Concept Leverages Hibernate n Data Persistence Through HDF 5 Why? n – High performance gaming level interactions supported – Chucking of data records supports visualization tiling – Flexible retrieval supports 4 D rendering – Streaming compression n Meta Data Implemented in Postgis. SQL through Inheritance – Defined only in plug ins, drives Data URI Base Object Model is Extended in Plug Ins n Data URI Concept Ties Everything Together n Purging Concept of Circularly Repository Structures n 1/5/2022 Page 59

ADE Conceptual Data Model Design I/O Formats Follows Existing Standards Data Model Output Formats Input Formats • Vector: ERSI shape file, SVG, Redbook • Raster: Geo. TIFF, png, jpg • Text: Canonical XML, WMO Bulletins… • VTEC warnings • grib 1/2 • GINI • ASCII (WMO, shef, …) • Radar Level II Data Flow CAVE Internal Object Model Request Formats • Canonical XML Ingested Persistance Respository • Flat Files • HDF 5 Raw Storage Model • Transmission Formats • HDF 5 model Decoded Persistance Respository • Flat Files • HDF 5 • RDBMS Decoded Storage Model • RDBMS Schema • HDF 5 model • Static data model T 05 Data Access Layer Metadata Index • RDBMS Meta. Data Model • RDBMS Schema 1/5/2022 Page 60

Data Access Layer API Hibernate Leading Object To Relational Approach Data Model Solves Fundamental Problem of Impedance Mismatch n Maps between Object Model and Relational Data Model n Provides Object based Query Facilities n Improves Performance over JDBC, Designed for Clustering n Reduces Code Count Improves Productivity n Built in Support in SPRING n Internal Object Model Hibernate Enables Meta Data Performance and Adaptability 1/5/2022 Page 61

Hibernate XML Object/ Relational Mapping Defined in SOA Plug In: Enables Adaptability Satellite Plug In satellite. hbm. xml Satellite. Record. java Data Model Data Object Model <<abstract>> Abstract. Blob. Data. Record <<Java>> Satellite. Record mapping Plug In Defined Object to Relational Mapping by Hibernate XML -product_type -datatype *** Data Object Model Extended By Plug In Follows Base Model Plug In Enables Adapting To New Data Types 1/5/2022 Page 62

AWIPS-II: Data Access Pattern Designed to Support Clustering Data Model Mule ESB Enables Thread Safe Access DALconfig. xml • data. Source • hibernate. Props • session. Factory • DAOs SPRING Dependency Injection Container SPRING’s Hibernate Support Session. Factory <<Java>> Data. Layer +save. Metadata() +find. Plugin. Version(plugin) +execute. Query(query) +execute. Update(query) <<abstract>> Hibernate. Dao. Support +get. Hibernate. Template() +get. Session. Factory() C 3 P 0 JDBC mapping. Jar. Locations Connection Pooling +persist. Metadata() +find. Plugin. Version(plugin) +execute. Query(query) +execute. Update(query) +generate. Table(table. Name) plugins/**/*. jar *. hbm. xml <<Java>> Text. Dao <<Java>> Blob. Dao <<Java>> Direct. Query. Dao + <<abstract>> Base. Dao + + 1/5/2022 Page 63

SOA Plug In Defines a Meta Data Table Set Each Plug In Also Defines a HDF 5 set RDBM Meta Data Circular Series [name 1] Plug In <<table>> name 1 <<table>> Name 1_1…n [name. N] Plug In <<table>> Plugin. Version Data Model HDF 5 Repository Circular Series Data URI Creates HDF 5 Record Structure name 1 name. N <<table>> Name. N_1…n Dynamic Meta Data Schema Follows Rules Flexible Data Model is Plug In Extendable To New Data Types 1/5/2022 Page 64

Plug In Creates New Meta. Data in RDBMS Uses Postgre. SQL Table Inheritance and Rules Plug In <<Java>> Table. DDLGenerator +set. Table. Definition() +generate. DDL() +create. Partition. Tables() +create. Rules() +get. Purge. DDL() *. db. xml <<table>> Plugin. Version Name table. Name Version hib. Class Data Model Auto generates Meta Data Schema At Plug In Load Time <<table>> satellite Insert Rule Chooses Sub Table -product_type As Function of Time -datatype *** <<table>> satellite_1 … N Circular Series of Sub Tables Enables A Self Maintaining Schema Sub Tables 1/5/2022 Page 65

Meta Data Demo Using CAVE’s Volume Browser Data Model Dynamically Populates Select Boxes Through Catalog Queries SOA Service Product. Srv u. Engine Canonical XML Message Query Meta Data For Catalog <<Java>> Data. Layer T 05 All New Hibernate Meta. Data Store <<table>> satellite 1_1…n 1/5/2022 Page 66

Refines Data. URI Concept Key for System Adaptability to New Data Types Data Model Data. URI is a reference to data in the data store (i. e. D 2 D Data KEY) n Enables Automatic Subscriptions For all Ingested Data n Automatically ties data persistence to meta data n Enables Plug In Extendibility to new data types with changing any base code n T 05 implemented a design for automatic generation of Data. URIs n 1/5/2022 Page 67

Meta Data Model Drives Data. URI Auto generated Data. URI couples HDF 5 to Meta. Data Model Satellite Plug In satellite. db. xml Satellite. Writer. java Auto Generates Data URI references from XML Definition And Meta Decode of Ingested Record Ingest. Srv <<Java>> Plugin +get. Data. URI() +get. Match. URI() Example Data URI From a LOG File <<Java>> Satellite. Writer HDF 5 Repository XML Meta Data Definition File Tag Specifies If Element is Part of Data URI RDBM Meta Data <<table>> satellite Plug In Enables Adapting To New Data Types 1/5/2022 Page 68

Data Persistence Using HDF 5 Files In Time Ordered Bins Like Meta. Data Circular Time Bins Data Model /awips/opt/data/hdf 5 i. e. Autogenerated data. URI ties Metadata to HDF 5 Record Plug In 1 Plug In 2 Plug In 3 1/5/2022 Page 69

Data Persistence Using HDF 5 Application Code Interfaces Through API Plug In Code Using HDF 5 api <<Java>> Satellite. Writer +write(record) { data. Store = (HDF 5 Data. Store) Data. Store. Factory} <<Java>> Data. Store. Factory +get. Data. Store(file) <<Java>> Short. Data. Record +get|set. Short. Record() <<interface>> IData. Store Data Model <<interface>> IData. Record +add. Data. Record() +store() +retrieve() +get|set. Dimension() +get|set. Name() +get|set. Sizes() +get. Data. Object() <<Java>> Storage. Properties <<Java>> HDF 5 Data. Store +add. Data. Record() +store() +retrieve() -write. HDF(group, record) -create. Group() jhdf 5. jar -un|lock. File() <<Java>> Integer. Data. Record +get|set. Integer. Record() <<abstract>> Abstract. Data. Record +get|set. Dimension() +get|set. Name() +get|set. Sizes() +get. Data. Object() <<Java>> Float. Data. Record +get|set. Float. Record() <<Java>> Byte. Data. Record +get|set. Byte. Record() 1/5/2022 Page 70

u. Engine Using the Data Access Layer Single API enables u. Engine to access all data Data Model u. Engine <<Task>> Spatial. Query +execute() <<Java>> Data. Layer +save. Metadata() +find. Plugin. Version(plugin) +execute. Query(query) +execute. Update(query) <<Task>> Term. Query. Index +execute() Results = data. Layer. find. Meta. Data (plugin, fields[], operands[], values[], sort, count) create. Meta. Data. Map() HDF 5 Repository Dynamically Creates URI Reference to Data Record 1/5/2022 Page 71

Data Repository Maintance Based on a circular cycle of tables & HDF 5 files 1 hr Wake Up Table holds retention period /2 of data (I. e. 12 hr of data for 24 hr of retention time) Data Model Purge. Srv Db. Manager HDF 5 directory tree Plugin Metadata delet e 30 1 Metadata_2 Metadata_3 new day & hour 2 Delete Referenced HDF 5 files Metadata_4 Purge creates new table and deletes oldest 1/5/2022 Page 72

AWIPS-II: Architecture Topics n n n n n Architecture Introduction Architecture Overview Architecture of SOA Service Architecture Patterns – Plug In pattern – Build pattern, Localization Pattern – u. Engine pattern, GIS capability, Adapter pattern – Notification/Subscription pattern Data Model Visualization Server Execution Hardening Security 1/5/2022 Page 73

AWIPS-II: Visualization Overview (CAVE) Built from Eclipse RCP Plugins + is Extensible Visualization Eclipse RCP 3. 3 XML User Localized Menus & Tool bar GIS Display All data layers map projected, scriptable, and subscribable High Performance Image Rendering GPU Shader Language Controls of (Projection, Color, …) QUAD Tiling of Raster Data World Wide Google style zoom/pan Data Sampling/Analysis Through raw data load GIS Whiteboard Collaboration Projection independent whiteboard, Vo. IP, Chat High Performance Vectors Through Vertex Array Interface Animation of All Data All displayed data automatically under subscription Thick/Thin Client Through ESB Endpoint Enterprise Service Bus Data Connections Advanced GIS Visualization Enables Gaming Interactions 1/5/2022 Page 74

CAVE Enhancements Most Significant Update Since Project Began n n n Visualization Eclipse RCP 3. 3 based, Geo. Tools 2. 4, JOGL 1. 1, Velocity, and batik Entirely new raster rendering implementation based on raw float data New raster tiling concept using HDF 5 New color map rendering concept New GPU mechanism for map reprojection Implemented Display Bundles – JIBX extension for Resource to Bundle mapping enables quick adaption to new visualization resource n n n High Performance radar rendering similar to D 2 D all-tilts – On demand rendering Active raster pixel data integration Warning generation with “Velocity” parsing of templates 1/5/2022 Page 75

AWIPS-II: Example Data Fusion in CAVE GIS display of raster, point, and vector data Visualization Weather Satellite QUAD tiled display raster data Data Spatially and Temporally Correlates Lighting Strikes High frequency point data converted to vector display Point Weather Observations Text base observation converted to vector display Geo Political Maps Shape File rendering for ERSI Map data Complete Layer Controls Order, Color, transparency, visibility, Active Layer Automatic Subscriptions All displayed data automatically under subscription and animates Automatic Data Fusion for All Ingested Data 1/5/2022 Page 76

AWIPS-II: Leverages Eclipse Perspectives Visualization Enables Multiple User Interfaces Weather Workstation Hydrology Workstation Graphical Forecast Editing Python Editor ESB Localization Utility. SRV • Perspectives leverages entire set of CAVE eclipse plug ins • Each perspective can have unique menus, toolbars, and dialogs • User can actively switch back and forth between perspectives XML Driven Menus Color Tables Scripts + Config Maps… 1/5/2022 Page 77

CAVE Menu Extensibility – Eclipse 3. 3 Configuration Preferences, Menus, Data … Visualization plugin. xml Menus defined in plug in xml CAVE satellite plug In bundle. xml references bundle Menu fields update Automatically through the Alert. Manager. Job as data is ingested Embedded u. Engine Script Retreives and transforms data for display 1/5/2022 Page 78

CAVE Menu Extensibility Pattern Leverages Capability in Eclipse 3. 3 Visualization CAVE satellite Plug In plugin. xml Bundle IRWindow. xml u. Engine Script to Retrieve loop of data 1/5/2022 Page 79

Workstation Menus / Toolbars / Rendering XML Configurable Menus + XML Bundles Visualization . ui. personalitles. awips plugin. xml. site. (from localization) plugin. xml XML Display Resource Bundles Commands plugin. xml Commands Parameters Menu labels ~19 Map set. Scale ~11 Plots Radar. Display. Controls menu. Retrieval product. URI product. Name variable. List 1 Skew-T ~30 Display Resources 8 Radar 11 Models 1/5/2022 Page 80

CAVE Display Bundles Enables Adding Procedures and Display History Visualization XML Document Bundle Definition +layers +color maps +line widths *** 1/5/2022 Page 81

CAVE Visualization – Service Endpoint Enables Gaming Style Data Interactions Visualization Eclipse RCP 3. 2 – Plug In Extendable Plug In for Warn Generation Added Radar Rendering Uses Dynamic raster tiling Added Bundles with Save / Retrieve GPU Shader Language Rendering Controls (Color, …), Animation New Quad tiling of large raster Sets leverages HDF 5 chunking Active Raster Data Interrogation Dynamic map reprojection using Geo. Tools Transforms& GPU Warping All Tilts Keyboard Controls 1/5/2022 Page 82

CAVE Radar Rendering Radial Data Gets Tiled Dynamically Visualization <<interfaces>> ICAVEResource IColormappable. Resource IImaging. Resource ITime. Seq. Resource IVert. Seq. Resource IToggleable. Resource IInspectable. Resource IBlendable. Resource <<java>> Radar. Tile. Set +create. Tile() +dispose() <<Job>> Radar. Loader. Job +request. Load() +run() HDF 5 Radar Data <<java>> Radar. Resource +paint() +init() +dispose() +*** <<java plug in>> Radar. Tiler +create. Tile() +create. Full. Image() +calculate. Pixels() +set. Pixel() 1/5/2022 Page 83

CAVE Raster Data Interrogation All raster and shape files are inspectable Visualization Mouse X, Y Events are Coordinate Converted to Lat/Lon n The Lat/Lon location is transformed to retrieve the data out of the correct displayed tile at the x, y location n The data is transformed into displayable units n <<java>> Radar. Resource +inspect(lat. Lon) <<java>> Abstract. Tile. Set +interrogate(coord) <<java>> Radar. Type. Record +data. URI; +Radial. Container; +Abstract. Tile. Set; +Timestamp; +type. Map; +tilt. Map; 1/5/2022 Page 84

Workstation Plot Model and Maintenance Plot Vector Library Converted to SVG (XML) Visualization Metars Obs Meta. Data & Parsed data fields {…} Plug In Job Buoys Plot. Resource Plot. Model Generator Progressive Disclosure SVGPlot SVG Plot Models UA plot 925 MB Entire present Wx symbol set converted to SVG 1/5/2022 Page 85

Workstation Warning Generation T 09 Delivered T 08 Delivered Warn. Gen GUI Visualization T 010+ Delivered Life Cycle Rules Dissemination VTEC Database tornado. vm severethunderstorm. vm Templates 3. . 19 GIS Capabilities Warning by Poly. Gon Text. DB UPDATE LIST Severe Weather Statement Store Set VTEC Generate Text Send to Text. WS Restart Text Workstation Editor 1/5/2022 Page 86

Workstation Text WS and Wx Editor Implemented as 2 RCP plug ins in CAVE Visualization . texteditor. textworkstation EDEX Text. DB Workstation Interacts with Text. DB Hibernate Access Objects {tablename}Dao text Plug In . . /opt/data/sbn/test Text Ingest Endpoint 1/5/2022 Page 87

GIS Map Reprojection: By Texture Warping GPU Program 50 times Faster than CPU Visualization Analysis/Sampling enabled by the use of raw data CAVE Visualization HDF 5 Repository Chunked Float Array Mapping Vertex Array GIS Fused Layer GPU Program Color Map To 1 D Graphic Texture Open. GL >= 2. 0 Graphical Processing Unit (GPU) Open. GL API JOGL Decoded Blobs Grids Satellite Rasters Radar Lighting XML Color Tables RGBA Quads Unlimited Warp CPU-Card Boundary Enables Gaming Style GIS Data Fusion Lambert conformal + Color Map 1/5/2022 Page 88

GPU Fragment Shader Color Mapping Enables Interactive Color Controls <<JAVA>> GLTarget Visualization Grib Data Decoded to float array texture 1 D: RGB color texture 2 D: float Image data <<texture shader>> IMAGING_GLSL_PROGRAM +main() { loaded into GPU during Initialization} User’s. RGB color map (RGBAs in XML format) index gl_Frag. Color Final Image Interactive Color Maps 20 to 50 Times Faster Than CPU 1/5/2022 Page 89

GIS Collaboration Capability: Two Laptops Based on XMPP protocol standards AWIPS Laptop 1 Visualization AWIPS Laptop 2 Connecting to a Specific Collaboration wildfire Collaboration Wide Area Network xmpp Concurrent CAVE Perspective has Chat Area CAVE Menu Interface To Collaboration Jabber Standard Collaboration Protocol Supports Vo. IP through plugin 1/5/2022 Page 90

AWIPS-II: Architecture Topics n n n n n Architecture Introduction Architecture Overview Architecture of SOA Service Architecture Patterns – Plug In pattern – Build pattern, Localization Pattern – u. Engine pattern, GIS capability, Adapter pattern – Notification/Subscription pattern Data Model Visualization Server Execution Hardening Security 1/5/2022 Page 91

AWIPS-II: Server Execution Structure Marrying ESB with Dependency Injection n n Server Execution ESB’s evolved out of the expense of getting services to communicate across multiple protocols and languages Ross Mason identified these problems back around 2001 and developed the Mule Project which now has a large and active community A parallel development by Progress Software created Sonic ESB and they coined the term Mule ESB is based on “Enterprise Integration Patterns” and is the most elegant extensible solution The high decoupling of Spring’s Dependency Injection Container with Mule’s endpoint abstract creates a high value approach Author: Hohpe, Gregor Author: Woolf, Bobby • Solving Integration Problems Using Patterns • Messaging Systems, Channels • Message Construction • Message Routing • Message Transformation • Messaging Endpoints • System Management Mule ESB + Spring Becomes a Discriminator 1/5/2022 Page 92

Mule ESB + SPRING Container Services decoupled from transport mechanisms MULE ESB Mule End Point Provider Data Message Receiver Inbound Router Server Execution SEDA Load Balanced Queue Connector Transformers Mule Interceptor Pattern Logger Mule End Point Provider Transformers Connector Data • JMS • Files • HTTP • FTP • E-Mail • *** Outbound Router Profiler Message Receiver Security Check EDEXSOA Service EDEX SOA services are Dependency Injected Components • Configured in the XML Mule files 1/5/2022 Page 93

AWIPS-II: Data Ingest Endpoints Example Warnings Routed to Their Own Endpoint SBN Satellite File: //. . /data/sbn/sat jms: //cp/sat Grib File: //. . /data/sbn/grib jms: //cp/grib Radar File: //. . /data/sbn/radar jms: //cp/radar ascii File: //. . /data/sbn/ascii ESB Reg. Ex. Filter Fan Out Routing plugin. Type SATELLITE Ingest. Srv plugin. Type GRIB plugin. Type RADAR Server Execution plugin. Type MESOWEST mesowest jms: //cp/mesowest SEDA Scaled SOA Service warning jms: //cp/tstormwarning metar jms: //cp/metar shef jms: //cp/shef alert jms: //cp/alert 1/5/2022 Page 94

SOA – Capability Available as Services Server Execution Scales Up with SEDA Clustering on two Levels • Within Container • Within Data Server Cluster SBN CP. 1 SBN Srv JMS Broker JMS: //cp Automated Client Enterprise Service Bus Data Server. 1 JMS Broker Data Server. 2 JMS Broker JMS: //product. Srv . N ESB Container Product. Srv. 1 Ingest. Srv. 1 Product. Srv. 2 Ingest. Srv. 2 Product. Srv. n State 1/5/2022 Page 95

Server Deployments Clustered JMS Broker Enables SEDA load balancing Deployment Data Server. 1 Server Execution Deployment Data Server. 2 JMS Broker activemq_clustered postgis. SQL Server ESB Container HDF 5 Lock State activemq_clustered ESB Container JDBC Driver HDF 5 Lock State HDF 5 Files u. Engine processing postgis. SQL Tables 1/5/2022 Page 96

Server State Synchronization of Locks Ensures Data Integrity of HDF 5 Writes Server Execution Local synchronized method for thread level locking within JVM Data Server. 1 ESB Container <<Java>> HDF 5 Data. Store <<Java>> Clustered. Lock. Manager +store() -write. HDF(group, record) -un|lock. File() -local. Synchronization. Map +get. Lock(Lockname) +release. Lock(Lockname) JGroups HDF 5 Files From JBOSS J 2 EE Clustering HDF 5 Lock State Multicast Voting Algorithm Clustered synchronized method uses a multicast voting algorithm File Level HDF 5 Locking For Cluster 1/5/2022 Page 97

Remote Management Through JMX Gives Detailed Insight Into Remote Java JVMs Server Execution Local Area Network (LAN) – ie Demo is using a Cross Over Cable Server 1 + CAVE Server 2 Running JMX Console Remotely Monitoring Server 1 active. MQ Mule ESB SOA Services 1/5/2022 Page 98

AWIPS-II: Architecture Topics n n n n n Architecture Introduction Architecture Overview Architecture of SOA Service Architecture Patterns – Plug In pattern – Build pattern, Localization Pattern – u. Engine pattern, GIS capability, Adapter pattern – Notification/Subscription pattern Data Model Visualization Server Execution Hardening Security 1/5/2022 Page 99

Hardening – JPROBE dynamic code analysis Jprobe Execution Time Breakout For Grib Ingest Hardening Satellite Ingest 108 ms 46 ms/msg 64 ms/msg • CAVE memory leak when running loops overnight, fixed • active. MQ dropping Mule connections (time outs), fixed • Jgroups configuration for clustering when clustering Con-ops determined • Message reference memory leak in AWIPS services fixed • Message file caching for idle messages eviction policy added to prevent memory growth 1/5/2022 Page 100

Omaha Test Bed Configuration Development and Test Hardware Resources Hardening Omaha Raytheon Server Room CP Processor Dell 2550 Raytheon Firewall awipscm Dell Server Unidata LDM-6. 6. 5 Auto Nightly 2 Week Stability Testing Fiber Line DVB-S awips-int 1 2950 Server awips-int 2 2950 Server LDM-6. 6. 5 filters EDEX Cluster EDEX Bld. XX awips-dev 1 2950 Server LDM-6. 6. 5 filters EDEX Nightly Orion Network awips-db 2950 Server postgre. Sql awips-nas 2950 Server Shared NFS 1/5/2022 Page 101

AWIPS-II Cumulative Memory Profiles JMX memory and cpu monitoring + Your. Kit Hardening JMX cpu Useage 5. 0% Day Average Your. Kit Java Profiler 1/5/2022 Page 102

Notify. Srv Changes Scripts and Subscriptions are in RDBMS <<SOA Service>> Notify. Srv Hardening <<SOA Service>> Auto. Bld. Srv Script Ids get. Data. URI() from New Ingested Data get. Modified. Script (subscribe. Key, script. ID, data. URI) get. Subscription. Key(data. URI) Run Script get. Subscription () Build Response JMS Topic Subscription Return Output Send list of script Ids that Depend on the Ingested Data Compiled Java. Script u. Engine script Postgre. SQL Data Base subscription scripts CAVE 1/5/2022 Page 103

AWIPS-II: Architecture Topics n n n n n Architecture Introduction Architecture Overview Architecture of SOA Service Architecture Patterns – Plug In pattern – Build pattern, Localization Pattern – u. Engine pattern, GIS capability, Adapter pattern – Notification/Subscription pattern Data Model Visualization Server Execution Hardening Security 1/5/2022 Page 104

Security Architecture Concepts Do not change anything significantly Security Each installation has perimeter security similar to what exists now we are not changing the security model. n Government is responsible for C&A package n Goal is to be able to use a remote service as if it is local n Access Authentication n Service Authorization n Data Ingest Through Data Diode Type concept n SOA Endpoint Security Through ESB & JMS provider n 1/5/2022 Page 105

Proxy Gateway Enforces SOA Polices Encapsulates Customer Connection Polices Commercial Packages - Jax. View, Amber. Point SOA PROXY Authentication WS Security… En/Decrypt Message Content Security Service Mediation -Protocol Conversions -Throttling -Load Balancing -Service Level Agreements Blocking Non-compliant Requests Registry j. UDDI… Opening Up The Enterprise 1/5/2022 Page 106

Security Concepts Continued SOA infrastructure has built in capabilities n Security JMS provider: active. MQ – Support JAAS as a provider for authentication – Default points to a “login. config” on classpath that in turn points to: l l user. properties {has: user=password info} group. properties {has: group=user 1, user 2… } – User Roles on Queues and Topics: l Read, Write, Admin (create) – Configurable message authorization policy allows each message to be content based authorized by a custom policy n Mule Security: Acegi; JAAS; PGP – Built in configurable security manager is responsible for authenticating requests – Configurable encryption strategy includes transports such as SSL and HTTPS – JAAS security provider (Java Authentication & Authorization Service) – API in since Java 1. 4 – Mule is extendable to PGP signed encryption on endpoints for end-to-end communications 1/5/2022 Page 107

Security Architecture Data Ingest Concept of Data Diode for Firewall Low Side LDAD Custom Socket Listener Has No Clue About Other Side Firewall Security High Side DX Cluster Custom Socket Client Initiates All Request And Data Transfers /clean-inbound/… /inbound/… One Way Interface HTTPS CAVE ESB Endpoint Security Filter ESB Endpoint 1/5/2022 Page 108