Brief introduction to Messaging Systems Daniel Rodrigues CERN

Brief introduction to Messaging Systems Daniel Rodrigues CERN IT Department CH-1211 Genève 23 Switzerland www. cern. ch/it

Summary • Messaging Systems Overview – – What Why How Where • Monitoring context in the Grid • The MSG – Messaging System for Grids • Fast Forward Internet Services CERN IT Department CH-1211 Genève 23 Switzerland www. cern. ch/it

Messaging Systems • (Grid Maps image) Internet Services CERN IT Department CH-1211 Genève 23 Switzerland www. cern. ch/it

Messaging Systems • What is Messaging? – Human characterized Uhooo… that communication sounds complicatedprocess, and off the scope : S for being asynchronous “Messaging is a technology that enables high-speed, – Examples: • Internet Services CERN IT Department CH-1211 Genève 23 Switzerland www. cern. ch/it asynchronous, program-to-program communication • The medieval with reliable delivery. messenger; Programs communicate by sending • Post; packets of data called messages to each other. • Channels logical pathways that connect the Message(. . . ) in a are bottle. programs and convey messages. Sender or producer is a program that sends a message (. . . ). A Abstraction of the process receiver or consumer is a communication program that receives a message being (. . . )” sent from one or more as messages producers to one Integration or more Patterns consumers; Hohpe et al, Enterprise

Messaging Systems • Why use messaging? – For communicating we could use: • • • Internet Services CERN IT Department CH-1211 Genève 23 Switzerland www. cern. ch/it – – File transfer Shared Databases Remote Procedure Calls Web Services Mail CORBA They do exist; They have common ideas; They share implementations; You might be using more than one to achieve a result that suits your needs!

Messaging Systems • Why use messaging? – Key ideas and benefits: • • • Loosely coupled distributed communication; Exceptional interoperability; Asynchronous; Reliable; Configurable Persistence – Drawbacks: • More complex programming model • Harder to do sequenced and synchronous model • Performance? (maybe FTP could do the trick) Internet Services CERN IT Department CH-1211 Genève 23 Switzerland www. cern. ch/it

Messaging Systems How? (Or, “Ok, may we finally see a picture? ”) • How? Publisher Internet Services CERN IT Department CH-1211 Genève 23 Switzerland www. cern. ch/it Publisher Consumer

Messaging Systems • Patterns: Message – Header • Routing information • Description – Body • Data • Ignored by the messaging system Internet Services CERN IT Department CH-1211 Genève 23 Switzerland www. cern. ch/it – Event. Message, Command. Message, Document. Message, Request. Reply – Could be SOAP, JMS, Stomp, etc.

Messaging Systems • Patterns: Message Channel – Point-to-Point • Snail Mail • Queues – Publish-Subscribe • Television/radio Broadcast • Topics Internet Services CERN IT Department CH-1211 Genève 23 Switzerland www. cern. ch/it – Data. Type. Channel, Invalid. Message. Channel, Dead. Letter. Channel, Channel. Adapter, Message. Bus, Messaging. Bridge

Messaging Systems • Patterns: Message Endpoint – Publisher • Gets data from application and creates a message. – Consumer • Extracts data from a message and passes it on to the application. Internet Services CERN IT Department CH-1211 Genève 23 Switzerland www. cern. ch/it – Selective. Consumer, Competing. Consumer, Durable. Subscriber, Message. Dispatcher, Transactional. Clients, Event. Driven. Consumer. – Endpoints either sends or receives messages, and are channel specific. (Ears mouth, eyes are not the same thing)

Messaging Systems • Isn’t it too complex to implement all this? – Indeed. • But someone has already done most of the work for you: – Commercial solutions: • Tibco Rendezvous, IBM Web. Sphere MQ, SUN Java Message Service, Microsoft MSMQ, BEA Message. Q, Sonic. MQ, 29 West UME/LBM. – Open. Source providers: • Apache Active. MQ, Object. Web JORAM, Open JMS. • Each are adequate to different problems. Internet Services CERN IT Department CH-1211 Genève 23 Switzerland www. cern. ch/it – Integration on different platforms; – Latency concerns; – High throughputs;

Messaging Systems • Where is it used? – Financial Services • exchanges, brokerages, hedge funds; – Insurance Companies – Banking Industry – Telecoms • Usually embedded in integrated solutions – Enterprise Backbones; • Websphere. MQ example (March 2007): Internet Services CERN IT Department CH-1211 Genève 23 Switzerland www. cern. ch/it – 10. 000 customers – 10 billion messages carrying US$1 quadrillion (US$ 1 000 000 000) worth of business transactions.

Messaging Systems • Where is it used? – MSG! Messaging System for the Grid; • • Used in Monitoring; Based on an Active. MQ backbone and python; Simple way to send records across… …and have multiple consumers consuming into an Oracle DB; • SAM: metric. Output, metric. Descriptions; • Grid. FTP transfers : Gridview; • Job Status, Job Details from various sources. Internet Services CERN IT Department CH-1211 Genève 23 Switzerland www. cern. ch/it

Messaging Systems • And now, for something completely different Internet Services CERN IT Department CH-1211 Genève 23 Switzerland www. cern. ch/it

Messaging Systems • Other Patterns – Message Routers • Message may be routed to different channels depending on its characteristics; • Simple Example: use a wild card topic! – grid. usage. transfer. *, where it will be forwarded to grid. usage. transfer. <INFRASTRUCTURE> – Message. Translators • Translation at different layers (data structure, types, representation, or transport). • e. g. transport protocols: TCP => HTTP => SOAP => JMS – Pipes and Filters Internet Services CERN IT Department CH-1211 Genève 23 Switzerland www. cern. ch/it • Message may need processing in different steps. • A Message goes through filtering and pipes that perform different functions (e. g, auth. N, auth. Z)

Messaging Systems • And now, for something completely different Internet Services CERN IT Department CH-1211 Genève 23 Switzerland www. cern. ch/it