Event Based Systems Architectures Dr Emanuel Onica Faculty

Event Based Systems Architectures Dr. Emanuel Onica Faculty of Computer Science, Alexandru Ioan Cuza University of Iaşi

Contents 1. The publish/subscribe (pub/sub) architecture 2. Formal specification for event based systems 3. Data dissemination and filtering 4. Topic based pub/sub system solutions 5. Storm for pub/sub? 2/46

The pub/sub architecture Publish/subscribe (pub/sub): Probably the most common paradigm used in distributed event-based systems. The actors: Producers = Publishers Consumers = Subscribers Communication Middleware = Brokers overlay (typically) Where are the events? 3/46

The pub/sub architecture The (simple description of the) way this works: The event data is contained in publications produced and emitted by publishers. Subscribers register subscriptions that contain their interests by sending these to brokers. Brokers match publications with subscriptions and route these towards interested subscribers. 4/46

The pub/sub architecture Let’s look again at some use cases: • general model • stock exchange • e-health 5/46

The pub/sub architecture Some variations: • The messages sent by brokers to subscribers following a positive matching are sometimes called notifications. • Notifications may contain just a confirmation of the match and not the complete publication data. • This alternative can be used to optimize communication overhead in cases when subscribers are just interested to know that an event happened (e. g. , a new pacient was received) and not in the event data (e. g. , the pacient’s personal data). 6/46

The pub/sub architecture Some variations: • In some cases advertisements are sent by publishers before publications. • Advertisements are sent periodically and describe the types of publications that will follow. • Subscribers are notified therefore on new potential categories of events that might be of interest (e. g. , a new company being listed at a stock exchange) 7/46

The pub/sub architecture Some variations: • Subscribers can send unsubscriptions to brokers when they want to unregister a previous subscription. • A typical implicit approach is setting an expiration time at subscriptions registration (e. g. , subscribers pay for a limited time service) 8/46

The pub/sub architecture Some variations: • Peer-to-Peer architectures do not contain separate broker overlays. • Peers can be altogether subscribers, publishers and brokers. 9/46

Formal specification for event based systems • 10/46

Formal specification for event based systems • 11/46

Formal specification for event based systems Some examples: • ◊□F – from a certain state onwards F holds forever • □[F=>□F] – once F is true at any of the states it will hold forever • □[F=>◊G] – once F is true at any of the states G will hold starting at some state afterwards • F=>◊□G – if F is true initially, there is a state afterwards from where G will hold forever 12/46

Formal specification for event based systems Let us define the system state using three variables: • SX – the set of active subscriptions of subscriber X • PY – the set of notifications published by publisher Y • DX – the set of notifications delivered to subscriber X Note that notifications (publications) are unique (each can be published only once). Producers (publishers) and consumers (subscribers) execute operations using the service providing the functionality of the system (i. e. , by brokers) that affect the state of the system. 13/46

Formal specification for event based systems • 14/46

Formal specification for event based systems • 15/46

Formal specification for event based systems • 16/46

Formal specification for event based systems • 17/46

Data dissemination and filtering Two major paradigms for pub/sub systems: • Topic Based 18/46

Data dissemination and filtering Two major paradigms for pub/sub systems: • Content Based 19/46

Topic based pub/sub systems Probably the most known topic based pub/sub: JMS – Java Message Service • messages include event (publication) data • offers centralized functionalities through a JMS provider for creating, sending, receiving and reading messages • loose coupling: a message is addressed to a destination and can be retrieved from it without the sender and receiver being available at the same time or knowing of each other • asynchronicity: a JMS provider can deliver messages to clients as these arrive without specific request action from them • reliability: a JMS provider can ensure that a message is delivered once and only once 20/46

Topic based pub/sub systems The JMS API: • introduced in 1998 • part of the Java EE platform starting with version 1. 3 JMS API architecture: • JMS provider – the core messaging service, offering administrative and control features • JMS clients – programs that produce and consume messages • Messages – objects encapsulating data exchanged between JMS clients • Administered objects: • Connection Factories – objects used by clients to create connections with providers • Destinations – objects used by clients to specify the target of produced messages and the source of consumed messages 21/46

Topic based pub/sub systems JMS messaging paradigms (Figure source: Java EE 6 documentation) Point-to-Point: • Messages are addressed to specific queues • Queues retain all messages until consumed or expired • Each message has one consumer • Sender and receiver clients have no timing dependencies • Receivers aknowledge processing of messages 22/46

Topic based pub/sub systems JMS messaging paradigms (Figure source: Java EE 6 documentation) Publish/Subscribe: • Messages are addressed to specific topics • Topics retain messages until distributed to all subscribers • Each message can have multiple consumers • Publishers and subscribers have timing dependencies: subscribers can consume messages published only after their subscription, and should be active by default • Durable subscriptions: subscribers can be allowed to create subscriptions that permit them to receive publications while subscribers are not active (with late delivery) 23/46

Topic based pub/sub systems JMS API permits using the same code for sending and receiving messages when using either of the two paradigms: P-to-P or Pub/Sub Mesaging consumption can be done in two ways: • Synchronously: subscribers explicitly fetch messages from destination by calling the blocking receive method • Asynchronously: subscribers register a message listener, and the JMS provider calls the on. Message method in the listener when a message arrives 24/46

Topic based pub/sub systems Apache Hedwig: open source topic based pub/sub system Features: • Guaranteed delivery – messages are not lost when subscribers go offline • Scalability – claims of supporting up to 1 million topics with 10 subscribers per topic, and incrementally scaling by adding servers on the fly • Availability – system should remain functional when a single server fails 25/46

Topic based pub/sub systems Apache Hedwig architecture (figure source: Apache Hedwig documentation) Top level hierarchy: collection of regions Any region may publish on a topic and messages are delivered to any subscriber in any region subscribed to that topic. 26/46

Topic based pub/sub systems Apache Hedwig architecture Regions: collections of hub servers • Hub servers aggregate and store publications in their region • Hub servers store subscriptions generated by subscribers in their region • Hub servers subscribe to hubs in other regions to listen for subscriptions registered at their site • Subscription topics are randomly assigned to hubs in a region (one topic is assigned to one hub) 27/46

Topic based pub/sub systems Apache Hedwig architecture Components of a hub • Network I/O message communication module: uses Netty (Java NIO framework) • Topic Manager: coordinates topics ownership among hubs and is responsible for automatic failover at a hub fail (uses Zookeeper for coordination) • Subscription Manager: administrates registered subscriptions using consume points – a marker on a topic until which a subscriber has received and acknowledged publications, based on which the next publication can be delivered 28/46

Topic based pub/sub systems Apache Hedwig architecture Components of a hub • Persistence Manager: stores publications in a reliable way, such that these can be retrieved in order • Remote Subscriber: acts as a client that subscribes to hubs in other regions for obtaining publications wanted by the local region clients • Delivery Manager: delivers publications to subscriber clients or to hubs subscribed from other regions 29/46

Topic based pub/sub systems Apache Hedwig topic management Topic creation: • Lazy process – topics are created and assigned to a hub on demand: • when a new subscription for the topic is received • when a publication on the topic is received • when a subscriber to a failed hub that handled the topic reconnects Topic redistribution: • Done occasionally by shuffling topics between hubs for load balancing purposes: • when a new hub joins • when a hub is overloaded (hubs store load statistics) 30/46

Topic based pub/sub systems Probably the most used topic based pub/sub solution in the Internet-of-Things area: MQTT – MQ Telemetry Transport – open protocol developed in collaboration with IBM • lightweight publish/subscribe messaging • scaling up to thousands clients on a server • targeted to restricted resource environments: • • low network bandwidth limited processing power reduced memory capabilities high latencies 31/46

Topic based pub/sub systems MQTT is integrated with IBM’s Web. Sphere MQ (proprietary solution): a messaging framework offering functionalities similar to JMS • besides its own API, IBM MQ has direct compatibility with JMS API • offers reliable delivery of messages without duplicates • architecture based on a message queue abstraction Figure source: V. Lampkin – What is MQTT and how does it rwork with Websphere MQ? 32/46

Topic based pub/sub systems MQTT offers several degrees of Quality-of-Service in message delivery: • Qo. S 0 = at most once delivery – best network effort, fastest, no response, no retry • Qo. S 1 = at least once delivery – message is guaranteed to be delivered, duplicates can exist • Qo. S 2 = exactly once delivery – message is delivered uniquely 33/46

Topic based pub/sub systems MQTT communication use cases: • RFID chips in various cards • health senzors monitoring • energy meters • Facebook Messenger notices on mobile devices • Intelligent house applications • Warm. Dirt (controlling soil temperature for plants) • etc. . . 34/46 Picture source: Craig Hollabaugh Picture of the Day

Topic based pub/sub systems Rappel • Academic approach – J. A. Pattel et al. (Journal of Computer Networks 53, 2009) • Focused on improving fairness in topic based pub/sub • Advantages: • Low overhead at publisher & subscriber nodes • Fast reception of updates at subscribers • Noiselesness update dissemination • Potential area of application – specifically feeds: RSS, Twitter, etc 35/46

Topic based pub/sub systems Rappel takes advantage of: • Interest locality - two nodes subscribed to the same feed are considered having covering subscriptions • Network locality - the stretch factor is a measure counting number of hops a message travels Design components: 1. Dissemination tree per feed 2. Friendship overlay 36/46 Figure source: Rappel: Exploiting interest and network locality to improve fairness in publish-subscribe systems (J. A. Patel et al. , Computer Networks 2009)

Topic based pub/sub systems Rappel’s dissemination tree: • Uses network distance between node coordinates • Distance estimations are computed by mapping nodes in an n-dimensional Euclidean space • Could a node join the dissemination tree at the top publisher? • Yes, but it would lead to high load traffic at the tree top • Periodical re-joins to locate new parents that might improve the stretch factor 37/46

Topic based pub/sub systems Rappel’s friendship overlay: • Target: find nodes close in network and have also good subscription coverage • For maintaining the overlay a state per node is kept including three data structures: • Friends set – Node ni maintains a limited set of friend nodes in close proximity based on network coordinate and coverage • Fans set – Node ni maintains a limited set of fan nodes who have ni in their friends list • Candidates set – Node ni maintains a set of candidate nodes that might be audited and included in the friends set • A utility function is used to maintain the value of friends based on network distance and subscription coverage • A gossip protocol approach is used to maintain the candidates set based on ping-ack availability measurements (we will discuss gossip protocols in a future lecture) 38/46

Topic based pub/sub systems Sprinkler - Reliable Broadcast for Geographically Dispersed Datacenters (H. Geng, R. van Renesse – Middleware 2013) • Target: pub/sub approach for broadcasting event updates to geographically dispersed datacenters • Particular focus: specific reliability issues – caching inconsistencies • Potential area of application – updates on large scale web applications (e. g. , Facebook, Amazon, Google+) that rely on caching optimizations 39/46

Topic based pub/sub systems Sprinkler – Problem context – Large scale web app What is needed: low latency responses How it’s done: query results are cached opportunistically on thousands of datacenters for faster response times The issue: database update ⇒ query results change ⇒ cache invalidation needed ⇒ multicast invalidation notification. . . to thousands of datacenters ⇒ inconsistencies (when an update is confirmed at the client, the database already changed again) And that’s not all: what’s the database update rate? . . . 40/46

Topic based pub/sub systems Sprinkler – architecture Two layers: • for reliable multihop broadcasts between datacenters (PLP – proxy level protocol) • for reliable broadcasts within datacenters (RLP – region level protocol) 41/46

Topic based pub/sub systems Sprinkler – API • client. get. Event() → event • client. publish(event) An event e belongs to a stream: e. stream. Three event types: • data – byte arrays • garbage collection – also contains a predicate P(e) which renders the event as obsolete if true • tombstone – a placeholder for a sequence of garbage collected events 42/46

Topic based pub/sub systems Sprinkler – semantics: • An event e is published once client. publish(e) returns • An event e is delivered once client. get. Event() → e’ returns where e’=e or e’ is a tombstone event for e • Sprinkler maintains the event delivery order per stream: if e is published before e’, e is delivered before e’ • Each data event e is followed by a matching garbage collection event g • For each garbage collected event , a matching tombstone event is generated • Events are published via a proxy servicing a region for a number of streams 43/46

Topic based pub/sub systems Sprinkler fault tolerance: • chain replication for proxies (for f failures, f + 1 replicas) • clients submit events by sending to the chain head, which send them along the chain • failed replicas are removed from the chain • impact is minimal for intermediary nodes • head failure requires announcing every peer proxy that wants to publish to the current proxy • tail failure requires setting new connections with lead proxies in other chains 44/46

Topic based pub/sub systems Sprinkler garbage collection: • What’s the use? • Two updates in a database for the same key ⇒ two invalidations needed ⇒ the first will be obsolete. . . • Ways to do it: • Keep only events that meet an „age” criteria – each data event has an index i and is also a garbage collection event that collects all events with indices in the i – N range • Just rely on the key matching – garbage collect the events with a similar key 45/46

Storm for Pub/Sub Open Discussion Can we use Storm? What would be the pros/cons/needs/etc. ? 46/46