Enterprise Integration Erik Doernenburg Thought Works Inc edoernenthoughtworks

Enterprise Integration Erik Doernenburg Thought. Works, Inc. edoernen@thoughtworks. com EMEA 2

Agenda v Challenges in enterprise application development v Design patterns v Enterprise Integration patterns v Interop technologies v Conclusion & Resources EMEA 3 © Copyright Thought. Works, Inc. ® 2005

The challenge v Enterprises depend on a growing number of IT systems v The systems must provide an integrated solution for the enterprise v The systems must interoperate with each other v Architectural trends and “waves” of technologies v Changing business needs and requirements EMEA 4 © Copyright Thought. Works, Inc. ® 2005

Finance Operator Application fax/telex app web email New System Existing Systems document recognition ERP ref. data v Workflow items are stored on Tibco RV/CM queue v Tibco Integration Manager w organises workflow w handles imports from document recognition w handles interaction with existing ERP systems v Reference data imported using linked servers EMEA 5 © Copyright Thought. Works, Inc. ® 2005

Portfolio Management Tool v Front-end uses Office 2003 code-behind technology v communicates with server using Web. Sphere MQ queues New application Excel front-end application logic v Message format defined with XSD schema technology adaptor analytics library Existing deal manager Existing messaging library Existing v Application uses existing services v accesses analytics library with COM bridge v accesses stored deals through deal manager service v accesses service bus through messaging abstraction library EMEA 6 © Copyright Thought. Works, Inc. ® 2005

Design Patterns EMEA 7

Why design patterns? v Code reuse remains difficult but… v Knowledge reuse can be very valuable v Patterns encapsulate knowledge of successful designs EMEA 8 © Copyright Thought. Works, Inc. ® 2005

Using technology successfully v Syntax w Basic language mechanism w A necessity but not a guarantee v Constructs w Classes, Interfaces, Inheritance, Polymorphism, etc. w Helpful but not sufficient for good design v Principles w Separation of concerns, Dependency Injection, etc. w Steer us towards a better solution v Patterns w Model-View-Controller, Observer, Decorator w Concrete design guidance EMEA 9 © Copyright Thought. Works, Inc. ® 2005

Patterns v “Mind sized” chunk of information (Ward Cunningham) v Shows a good solution to a common problem within a specific context v Observed from actual experience v Has a distinct name v Not copy-paste v Thought. Works collaborates with Microsoft to document design patterns for the. NET platform EMEA 10 © Copyright Thought. Works, Inc. ® 2005

Enterprise Integration Patterns Message Routing Message Construction Endpoint Application A Messaging Endpoints Message Channel Messaging Channels Message Transformation Router Translator Endpoint Application B Monitoring System Management v Gregor Hohpe defined a visual pattern language describing message-based enterprise integration solutions v Pattern language comprises 65 patterns in 6 categories EMEA 12 © Copyright Thought. Works, Inc. ® 2005

Pattern: Request-Response Consumer Request Provider Request Channel Reply Channel Response v Service Consumer and Provider (similar to RPC) v Channels are unidirectional v Two asynchronous point-to-point channels v Separate request and response messages EMEA 14 © Copyright Thought. Works, Inc. ® 2005

Multiple consumers Consumer 1 Requests Provider Request Channel Reply Channel 1 Reply Channel 2 ? Responses Consumer 2 v Each consumer has its own reply queue v But how does the provider send the response? w Could send to all consumers (very inefficient) w Hard code (violates principle of context-free service) EMEA 15 © Copyright Thought. Works, Inc. ® 2005

Pattern: Return Address Consumer 1 Reply Channel 2 Provider Request Channel Reply Channel 1 Reply Channel 2 Responses Consumer 2 v Consumer specifies Return Address (the reply channel) in the request message v Service provider sends response message to specified channel EMEA 16 © Copyright Thought. Works, Inc. ® 2005

Load-balanced service providers Provider 1 Consumer Request Channel Provider 2 Reply Channel v Request message can be handled by multiple providers v Point-to-point channel supports competing services v Only one service receives each request message v But what if the response messages are out of order? EMEA 17 © Copyright Thought. Works, Inc. ® 2005

Pattern: Correlation Identifier Consumer 1 2 2 1 1 2 Message Identifier Request Channel Provider 1 1 2 Provider 2 2 1 Reply Channel 2 1 Correlation Identifier v Consumer assigns a unique identifier to each message w Identifier can be an arbitrary ID, a GUID, a business key v Provider copies the ID to the response message v Consumer can match request and response EMEA 18 © Copyright Thought. Works, Inc. ® 2005

Multiple specialised providers Order Messages Order Entry Widget Inv. ? Gadget Inv. v Each provider can only handle a specific type of message v Route the request to the "appropriate" provider. But how? w Do not want to burden sender with decision w Letting providers "pick out" messages requires coordination EMEA 20 © Copyright Thought. Works, Inc. ® 2005

Pattern: Content-Based Router Order Messages Widget Inv. Order Entry Content Based Router Gadget Inv. v Insert a content-based router v Routers forward incoming messages to different channels v Message content not changed v Mostly stateless, but can be stateful, e. g. de-duper EMEA 21 © Copyright Thought. Works, Inc. ® 2005

Composite messages Order Message Order Entry Widget Inv. ? Gadget Inv. v How can we process a message that contains multiple elements? EMEA 22 © Copyright Thought. Works, Inc. ® 2005

Pattern: Splitter & Router Order Message Order Item 1 Order Items Widget Inv. Order Entry Splitter Gadget Inv. Router Order Item 2 v Use a splitter to break out the composite message into a series of individual messages v Then use a router to route the individual messages as before v Note that two patterns are composed EMEA 23 © Copyright Thought. Works, Inc. ® 2005

Producing a single response Order Item 1 Response 1 Confirmed Order Widget Inv. ? Billing Gadget Inv. Order Item 2 Response 2 v How to combine the results of individual but related messages? w Messages can be out-of-order, delayed w Multiple conversations can be intermixed EMEA 24 © Copyright Thought. Works, Inc. ® 2005

Pattern: Aggregator Order Item 1 Response 1 Confirmed Order Widget Inv. Billing Gadget Inv. Order Item 2 Aggregator Response 2 v Use a stateful filter, an Aggregator v Collects and stores messages until a complete set has been received (completeness condition) v Publishes a single message created from the individual messages (aggregation algorithm) EMEA 25 © Copyright Thought. Works, Inc. ® 2005

Communicating with multiple parties Vendor A Request For Quote Best Quotes Vendor B ? Vendor C Aggregator v How to send a message to a dynamic set of recipients? v And return a single response message? EMEA 26 © Copyright Thought. Works, Inc. ® 2005

Pattern: Scatter-Gather Pub-Sub channel Quotes Vendor B Request For Quote Best Quote Vendor A Vendor C Aggregator v Send message to a pub-sub channel v Interested recipients subscribe to a "topic" v Aggregator collects individual response messages w may not wait for all quotes, only returns one quote EMEA 27 © Copyright Thought. Works, Inc. ® 2005

Complex composition Pub-Sub channel Vendor A Quotes Vendor B Splitter Aggregator Quote request for each item Best Quote for each item Vendor C Aggregator v Receive an order message v Use splitter to create one message per item v Send to scatter/gather which returns "best quote" message v Aggregate to create quoted order message EMEA 28 © Copyright Thought. Works, Inc. ® 2005

Interop technologies EMEA 29

Major interop technologies v Resource based DB files v RPC style / distributed objects RMI-IIOP (CORBA) Remoting (DCOM) in-process bridge WS v Message based / document-oriented MOM WS WS-* EMEA 30 © Copyright Thought. Works, Inc. ® 2005

Interop technology characteristics platform neutral DB/files MOM WS-* WS J 2 EE server . NET server in-proc RMI-IIOP (CORBA) Remoting (DCOM) bridge point-to-point routable transient messages durable message clustering server lifetimemanagement EMEA 31 © Copyright Thought. Works, Inc. ® 2005

Messaging v Channels are separate from applications Ø Removes location dependencies v Channels are asynchronous & reliable Ø Removes temporal dependencies v Data is exchanged in self-contained messages Ø Removes data format dependencies v Loosely coupled integration enables independent variation EMEA 32 © Copyright Thought. Works, Inc. ® 2005

Why Web Services? v Web services provide all benefits of messaging solution w loose-coupling w service oriented w reliable communication v Why web services? w composable protocol w vendor neutral w suitable for access through Internet EMEA 33 © Copyright Thought. Works, Inc. ® 2005

Web Services development v Web Services are expected to become the default Messaging solution in the future v WS-* standards are evolving rapidly, most important w WS-Security w WS-Policy w WS-Addressing v WS-I defines profiles, sample applications and testing tools w Profiles are guidelines for using WS specifications w Use Basic profile 1. 1 to build interoperable solutions v Further research on WS is being carried out EMEA 34 © Copyright Thought. Works, Inc. ® 2005

Conclusion v Enterprise systems are becoming more complex v We have patterns to help us with the design v We have technologies to implement our designs v Building for interoperability and integration is key EMEA 36 © Copyright Thought. Works, Inc. ® 2005

Recommended books Enterprise Integration Patterns Gregor Hohpe, Bobby Woolf Addison-Wesley, 2004 Integration Patterns Microsoft Patterns & Practices 2004 Patterns of Enterprise Application Architecture Martin Fowler Addison-Wesley, 2003 Enterprise Solution Patterns using Microsoft. NET Microsoft Patterns & Practices 2003 Enterprise SOA Dirk Krafzig, Karl Banke, Dirk Slama Prentice Hall, 2004 EMEA 37 © Copyright Thought. Works, Inc. ® 2005

Please complete your session feedback form THANK YOU