Enterprise Service Bus Innovation Project for FY 15

Enterprise Service Bus Innovation Project for FY 15 Presented by: Mike Thomas – Sr. Architect Administrative Technology Services Enterprise Applications HUIT Sr. Leadership Team Meeting - June 18, 2015

ESB Innovation Project Timeline: POC: April 2014 - June 2014 Pilot: July 2014 - present Pilot Goals: • Get experience with ESBs in general and with 2 or 3 specific ESB products Team Members: • Bill Brickman (Data Warehouse) • Anita Cutting (Fin. Proc) • Leo Dai (Grants) • Sreeni Gunnala (Fin. Proc) • Lisa Justiniano (Sponsor) • John Marks (Grants) • Determine if an ESB would benefit ATS • Shanti Muppirala (Peoplesoft) • Evaluate ESBs to identify one that best fits our needs • Joe Rickert (Fin. Proc) • John Shen (Peoplesoft) • Get our feet with Amazon Web Services • Brian Sullivan (Data Warehouse) • Curt Springer (Student Financials) • Karen Stelle (Proj Mgr) • Mike Thomas (Sr. Architect) Pilot Budget: $200 per month 2

Enterprise Service Bus (ESB) - Wikipedia Definition An enterprise service bus (ESB) is a software architecture model used for designing and implementing communication between mutually interacting software applications in a service-oriented architecture (SOA). As a software architectural model for distributed computing, an ESB promotes agility and flexibility with regard to communication between applications. Its primary use is in enterprise application integration (EAI) of heterogeneous and complex landscapes. 3

Problem Statement • Data integration / data Interoperability between our applications has become a quagmire of technology and methodology. • Each integration is hand-crafted and designed for a single purpose. • Each development team selects the technology and approach that fits best with their products and skill sets, without consideration for standardization, consistency or scalability. • The same pieces of data are being included in a variety of different integrations because there is no catalog or directory of available data integrations. 4

Enterprise Service Bus Software Components Two Major Components: 1. ESB – provides web services – provides integrations using common protocols like SSH, JDBC, SMTP 2. Message Broker – pass arbitrary data – two models: • point-to-point queues • publish and subscribe topics 5

Advantages of an ESB • Standards-based integration mechanism instead of one-off solutions • Flexible integration options: anything that listens on a TCP port • Faster development: configuration instead of code • Improved discovery and transparency • Centralized security • Data / protocol transformations • Data fan-out VS • Decoupling systems • Batch processing can become transactional 6

Software Stack We chose: • Apache Service. Mix & Active. MQ • Free, active community We built a HA stack: • 2 x AZ • 2 x Service. Mix ESB • 2 x Active. MQ Msg Broker • 2 x (or 4 x) ELB • Not shown: • 1 x VPC • 4 x network subnets • 3 x security groups • 1 x direct connect 7

We built four integrations 1. General Ledger Chart Of Account Validator: • Provide standards-based mechanism to validate chart of accounts values • Existing validator PL/SQL JDBC Web Service 2. Eureka (Online Training System) to People. Soft Feed: • Push course completion data to People. Soft • Standalone Service. Mix Queue People. Soft Web Service 3. FRAP Data Load: • Move sponsored financial reporting data from Windows laptop to GMAS • FRAP control laptop Queue JDBC GMAS DB 4. Staff Terminations: • Make terminations feed available to interested parties • DB JDBC Queue, using another Queue to maintain batch state 8

What Challenges Did We Encounter? • Amazon Open. Stack Amazon • Service. Mix Fabric 8 Service. Mix • Amazon Direct Connect, asymmetric routing, and Elastic Load Balancers • Active. MQ authentication is all or nothing What did the developers learn? 100% Service. Mix and its components: Camel, CXF 90% Active. MQ 80% Amazon Web Services (in many cases their first experience) Every developer who hadn't already taken the 1 day AWS course took it. 9

Selected Developer Survey Comments I learned a good amount about ESB and how to jump out to custom code, and what a message broker is all about, good hands-on experience. Hosting/consuming web services in Servicemix, using OSGi blueprint with Camel for routing data and XSL. . . Active. MQ for data transportation, AWS. Very easy to learn and start developing bundles, fewer lines of code to create an integration, more configuration rather than coding, loosely coupled systems I learned about redundancy and failover in the cloud. For me it was an intro to the new products/tools, infrastructure and terminology. This technology is in its infancy. Learning about the different services in Service. Mix and their functionality was a very positive experience. I feel it could come in handy when prototyping integration projects in the future. I learned to create Camel routes using blueprint to work with Active. MQ. . . I created a Java bundle. . . performed the transformation using XSL. I was assigned to work on installing SSL keys for the load balancers. . . this is a great introduction to AWS for me. I had no prior experience with AWS. 10

Developer Conclusions 1. Service. Mix is suitable when embedded for the exclusive use of a single application, but what developers want is an ESB deployed as a centralized service. Service. Mix lacks: • security flexible enough to truly isolate integrations from one and other • code isolation to avoid naming collisions and version collisions • GUI for admin actions like adding users/groups, deploying code, browsing logs • good debugging support • registry / discovery of services • rate limiting (to guarantee SLAs) • analytics 2. Given choice of a single product, 2/3 rds preferred the message broker. 3. These are powerful and useful tools and we want one. 11

What is Next? • Wind down our infrastructure in AWS • Continue to use the products: • We can use standalone Service. Mix to ease integrations, e. g. COA Validator, maybe FRAP • We can embed Camel directly in Java applications to get the nice terse integration language without the overhead of a service • Data Interoperability Initiative led by Jon Saperia • Jon joined ESB project at the tail end • ATS Participants: Bill Brickman, Sreeni Gunnala, Curt Springer • Much larger scope: includes data governance, master data mgt. , etc. • Becomes a request for an ITCRB Capital Project? 12

Thank You! 13