Event Processing in Business Applications Roy Schulte 15

Event Processing in Business Applications Roy Schulte 15 March 2006 Notes accompany this presentation. Please select Notes Page view. These materials can be reproduced only with Gartner's official approval. Such approvals may be requested via e-mail — quote. requests@gartner. com.

Event Processing is a Bounded, Definable Discipline We assume these definitions: 1. First definition: An “event” (“ordinary event” or real-world event) is a state change (something that happened) which is significant to someone for some purpose. Implication: Not everything that happens is an event. A state change is an event only if you treat it as one. Many state changes are too fine grained, unimportant or lacking in identity to be events. 2. Second definition: An “event” (“software event” or “event object, ”) is an object, usually in the form of a message, that reports on a ordinary (type 1) event. 3. “Event processing” means computing that uses events (definition 2, not definition 1). Note 1: Ordinary events (type 1) may be (i) Simple – or “atomic, ” if they contain no meaningful (type 1) member events (ii) Complex – if they reflect the significance of two or more member events, each of which are meaningful (type 1 events) in their own right Note 2: An event object (type 2) may report either a simple event or a complex event. A complex-event object (type 2) may be • Raw or original, if it directly reports a complex event from the real world or • Derived (“synthesized”) if it is generated by processing two or more type 2 events

Event Processing Brings Two Kinds Of Business Benefits 1. Event-Driven Architecture (EDA) for Flexibility and Maintainability Order Entry Manufacturing Shipping 2. Complex-Event Processing (CEP) for Earlier and Better Insight

Agenda 1. Four Ways To Process Events 1. 1. Event passing 1. 2. Basic event mediation 1. 3. Complex-Event Processing (CEP) 1. 4. Combinations (e. g. , events with BPM) 2. Commercial Adoption of Event Processing 2. 1. Market drivers – is there a killer application? 2. 2 Market inhibitors – why hasn’t this already happened? 2. 3 Industry standards – critical to expanding the market

Event Passing in Event-Driven Applications Asynchronous “staged” business processes may be called • “Event-driven, ” • “Message-driven” or • “Document-driven” Event Passing MOM, Web Services, ESB, E-Mail Sink Event Source Publish Subscribe Sink

Characteristics of Simple Event-Driven Applications § § § § Event handler (sink) is triggered by the arrival of one event in one event stream No real pattern matching – the rule always evaluates to “true” (i. e. , begin execution upon arrival of any one well formed message in the event stream) Benefit is experienced mostly in the IT department Benefit comes through increased flexibility of software engineering. The absence of control coupling (source and sink are not interdependent) makes it easy to add new sinks, move a sink, or even eliminate a sink, all without touching the event source (of course, the source and sink must still share a common understanding of the event semantics) Relevant middleware here is usually some form of MOM Event messages usually don’t carry any genetic information that would indicate causality, or if there is such information, it is indirect (inserted by the application as a data element in the payload rather than deliberately planned and inserted on the header by an event processing agent (EPA) or manager) These events are raw (original), they are not explicitly synthesized directly from other events, although an application might generate an event which is seen as raw (original) but which actually reflects the significance of some other event(s) which were also seen as raw

The Popular, Modern (Loose) Definition of “SOA” May Encompass EDA § If “SOA” means a modular, distributable, loosely-coupled, sharable application, then EDA is a type of SOA (because EDA sources and sinks are also modular, distributable, sharable, encapsulated and uncoupled) § But if “SOA” means that one module provides a service to another (consumer) module, then EDA is not a type of SOA • A ‘service’ is the performance of duties or responsibilities for the benefit of others, but an event sink does not provide a service to a consumer module An SOA consumer delegates work to a subprocedure (the service provider), in contrast to an event source which does not delegate work or pass control Be careful of the word “consumer. ” In EDA, an event sink is said to be the “consumer” (of the event), while in SOA, the service “consumer” initiates the relationship and consumes a function supported by the service provider • • § The IT industry has acronym fatigue and hype fatigue, so wider adoption of EDA may be slow for another 5 years unless it is seen as wave 2 of “SOA”

Real Applications Leverage Both EDA and Request/reply SOA Relationships Order Entry Manufacturing Shipping Source Event Sink (event handler) Request/reply SOA within an application Sales Admin Use EDA when: § Application stages should run even when another adjacent stage is not running. § There will be a need to frequently add, drop or modify processing stages without affecting any previous or subsequent stage. § Multiple stages may execute simultaneously. § Two or more sinks (stages) need the same input event data.

Business Events Are Used in Four Ways: 2. Mediated Events Integration Broker Basic Event Mediation Event Passing Source Event Broker Event’ Service Provider Transform, Route on Contents MOM, Web Services, ESB, E-Mail Sink Event Source Publish Subscribe Sink

Characteristics of Basic Event Mediation n n n One event stream goes into a broker/mediator, one or more streams come out (message splitting but not message combining). Broker/mediator is stateless; it acts on each input event separately (it has temporary state only while it is operating and it does not retain state after it has acted upon an incoming event) Filtering, content-based routing, and transformation (“conditioning, ” or “enrichment”) may be performed by the intermediary Benefits directly accrue to IT people, indirectly to business end users The main benefit is insulating source and sink applications, effectively offloading application integration work from end point applications The relevant technology generally includes MOM and an integration broker Events sent out by the mediator generally lack explicit causality information

Business Events Are Used in Four Ways: 3. Complex-Event Processing CEP Tools Complex-Event Processing (CEP) Event Source Complex Event EPA BAM EPA Filter, Aggregate Integration Broker Basic Event Mediation Event Passing Source Event Broker Event Service Provider Transform, Route on Contents MOM, Web Services, ESB, E-Mail Sink Event Source Publish Subscribe Sink

Characteristics of CEP Two or more events come in from one or more event streams n Pattern detection requires rules, usually involving sophisticated algorithms n Non-IT people often see direct benefits through a BAM dashboard or similar mechanism - the most common benefit is business insight rather than faster and easier software engineering of a business application n The relevant technology generally includes MOM and a stateful eventprocessing agent (EPA) or other event manager (custom 3 GL coding would be onerous in many CEP situations) n Complex events are often consciously synthesized from simple events, and genetic information is sometimes inserted into the events Ø BPM is not required for CEP Ø A business process model is not required for CEP Ø But BPM may use BAM and CEP for monitoring of business processes n

CEP and BAM - Manipulating Multiple Event Streams 1. Capture simple events (e. g. , adapters) BAM Tool 2. Transport events (e. g. , MOM) Event EPA Dashboard Complex Event EPA Complex Event 3. Apply EPL rules: filter, correlate, apply constraints, aggregate, update event logs 4. Notify people, trigger BPM processes or invoke applications and SOA services, optionally enriching events with context from event logs and other databases Complex Subprocess Event BPM SOA Service

The Majority of Business Activity Monitoring (BAM) Applications Use CEP Goals n Monitor key objectives n Anticipate operational risks or incidents n Reduce latency between material events and action Orders processed Manufacturing defects Web channel volume IT capacity Supply chain status Support call times Approach n Update real-time dashboard n Link to BI historical reporting and analysis n Issue context-rich alerts n Integrate with business process management n Manage incident life cycles ?

Business Events Are Used in Four Ways: 4. Combinations Event-Based BPM Combinations: Source e. g. , Event-Enabled BPM Event BPM Sink Service Provider Process Management CEP Tools Complex-Event Processing (CEP) Event Source Complex Event EPA BAM EPA Filter, Aggregate Integration Broker Basic Event Mediation Event Passing Source Event Broker Event Service Provider Transform, Route on Contents MOM, Web Services, ESB, E-Mail Sink Event Source Publish Subscribe Sink

Characteristics of BPM-enabled Events One or more event streams are presented to a stateful BPM engine/mediator, one or more streams are generated as output (message splitting and message combining) or one or more services are invoked n Filtering, content-based routing, and transformation may be performed by the intermediary, but more importantly, the flow of control is governed by a stateaware BPM type mechanism that has a pre-specified model of the business process n Benefits mostly business end users n Benefits include making the process flow explicit, making the process flows easier to change and making it possible to monitor the integrity of a multistep process through multiple stages in its life cycle n The relevant technology includes MOM and a purpose-built stateful BPM engine. n Events are consciously synthesized from other events Ø BPM almost always uses message passing for some aspects of processing Ø BPM may also use event mediation and CEP for monitoring of business processes n

Business Process Management Tools Use Events to Trigger Activities and to Track Status Order Entry Shipping BPM Manufacturing Notify BPM engine that activity has completed Billing Invoke a SOA Service Trigger an EDA event handler

Adoption of EDA Application Types in Business Applications 100 Theoretical Usefulness 80 Actual Usage, 2005 Percentage Penetration 60 in New, Large-Scale Application 40 Systems 20 0 Event Passing Event Mediation Complex Events BPM Events

Agenda 1. Four Ways To Process Events 1. 1. Event passing 1. 2. Basic event mediation 1. 3. Complex-Event Processing (CEP) 1. 4. Combinations (e. g. , events with BPM) 2. Commercial Adoption of Event Processing 2. 1. Market drivers – is there a killer application? 2. 2 Market inhibitors – why hasn’t this already happened? 2. 3 Industry standards – critical to expanding the market

Market Drivers 1. Business strategies are enhanced by EDA. The real world is mostly event driven, and event-driven situations are best addressed by event-driven business applications (to eliminate the impedance mismatch) 2. Pressures for sense-and-respond behavior, fast action, predictionbased actions and agility are growing because of globalization, compliance demands and other forces (zero-latency enterprise, real -time enterprise and similar goals) 3. SOA is teaching developers about business components and will make event-capable middleware (ESBs) commonplace 4. Vendors are developing better event processing software tools 5. Event-oriented standards are expanding, especially within Web services and Java 6. Ongoing improvements in CPU and network price/performance and scalability 7. Candidate killer applications: Trading, defense, BAM, RFID, CRM, fraud detection, security, SCM, compliance

Market Inhibitors: Why this has not happened already 1. Gaps in software development methodologies even for simple staged event-passing EDA 2. Mainstream business application developers lack knowledge of eventcapable tools: MOM, integration suites, BPM, CEP and BAM 3. Business analysts do not understand EDA (design issues, technology or applicability), and EDA specialists don’t understand the business situations that could use EDA. Potential projects are not even identified (lack of ROI information is not the real problem) 4. The lack of standards breeds fear of high costs, lock-in and failure

New Analysis and Design Approaches for SOA and EDA Are Emerging OO Analysis and Design Enterprise Architecture Frameworks Data Modeling Event-oriented Analysis and Design Service-oriented Development of Applications (SODA) or Service-oriented Analysis and Design (SOAD) Next-generation Component-based Analysis and Design Business Process Modeling 1998 -2008 2003 -2013 Ø Most current work on developing SOA analysis and design methodologies still fails to make events first class citizens

Standards Relevant to Events (I) n Simple events – WS- RX, WS- Reliable Messaging and WS-Reliability – WS- Addressing – WS- Eventing, WS- Notification – getting back on track – WSDL v. 2 is better than WSDL v. 1 for EDA (MEP concept) – Java Message Service (JMS) – Message-Driven Beans (MDB) n Mediated events – XSLT, XPath, XQuery, JBI, SCA/SDO n BPM events – BPEL, BPMN

Standards Relevant to Events (II) n BAM/CEP Standards – Common Business Event (CBE) schema – first pass was pre-WS -Notification, is now seen as a part of the Oasis WSDM (Web services Distributed Management) event framework but it must be extended to handle business events – Event Processing Language (EPL) – No proposals currently on the table – Need standard APIs for handling events, event metadata and event databases (CEI is one possible starting point) n JAIN (Java for Advanced Intelligent Networks) – Includes SIP – SLEE (Service Logic Execution Environment)

Recommendations for User Companies ü Use EDA on the business process and application architecture levels. ü Design all new enterprise-class applications as sets of assignable modules, rather than with monolithic architectures. ü Use request/response SOA and the simpler forms of EDA in mainstream projects now. ü Use CEP in BAM applications and for leading-edge projects where real-time insight must be maximized and you are willing to invest in emerging technology.

Event Processing in Business Applications Roy Schulte 15 March 2006 Notes accompany this presentation. Please select Notes Page view. These materials can be reproduced only with Gartner's official approval. Such approvals may be requested via e-mail — quote. requests@gartner. com.