UNIT II 1 System Models Architectural model defines

  • Slides: 129
Download presentation
UNIT II 1

UNIT II 1

System Models • Architectural model defines the way in which the components of the

System Models • Architectural model defines the way in which the components of the system are placed and how they interact with one another and the way in which they are mapped onto the underlying network of computers. • Fundamental models: – Interaction model deals with communication details among the components and their timing and performance details. – Failure model gives specification of faults and defines reliable communication and correct processes. – Security model specifies possible threats and defines the concept of secure channels. • We will discuss the various models at a high level in this discussion and will elaborate on each of these as we discuss other systems. 2

Architectural Model • Concerned with placement of its parts and relationship among them. •

Architectural Model • Concerned with placement of its parts and relationship among them. • Example: client-server model, peer-to-peer model • Abstracts the functions of the individual components. • Defines patterns for distribution of data and workload. • Defines patterns of communication among the components. • Example: Definition of server process, client process and peer process and protocols for communication among processes; definition client/server model and its variations. 3

Software and hardware service layers in distributed systems 4

Software and hardware service layers in distributed systems 4

Clients invoke individual servers EX: 1. File server, 2. Web crawler EX: Web server

Clients invoke individual servers EX: 1. File server, 2. Web crawler EX: Web server EX: browser, web client 5

A service provided by multiple servers EX: akamai (data duplication), now amazon aws (zones)

A service provided by multiple servers EX: akamai (data duplication), now amazon aws (zones) 6

Web proxy server and caches Proxy servers + cache are used to provide increased

Web proxy server and caches Proxy servers + cache are used to provide increased Availability and performance. They also play a major role Firewall based security. http: //www. interhack. net/pubs/fwfaq/ 7

A distributed application based on peer processes Ex: distributed Whiteboard Application; Music sharing 8

A distributed application based on peer processes Ex: distributed Whiteboard Application; Music sharing 8

Web applets EX: Code streaming; mobile code 9

Web applets EX: Code streaming; mobile code 9

Interaction Models • Within address space (using path as addresses) • Socket based communication:

Interaction Models • Within address space (using path as addresses) • Socket based communication: connectionoriented, connection-less – Socket is an end-point of communication – Lets look at some code + details 10

Socket based communication int sockfd; struct sockaddr_in addr; addr. sin_family = AF_INET; addr. sin_addr.

Socket based communication int sockfd; struct sockaddr_in addr; addr. sin_family = AF_INET; addr. sin_addr. s_addr = inet_addr(SERV_HOST_ADDR); addr. sin_port = htons(SERV_TCP_PORT); sockfd = socket(AF_INET, SOCK_STREAM, 0); connect(sockfd, (struct sockaddr *) &addr, sizeof(serv_addr)); do_stuff(stdin, sockfd); 11

Classic view of network API • Start with host name • Get an IP

Classic view of network API • Start with host name • Get an IP address • Make a socket (protocol, address) • Send byte stream (TCP) or packets (UDP) foo. bar. com gethostbyname() 10. 5. 4. 3 socket(); connect(); … sock_id … 1, 2, 3, 4, 5, 6, 7, 8, 9. . . TCP sock UDP sock Network Eventually arrive in order May or may not arrive 12

Protocol layering • Communications stack consists of a set of services, each providing a

Protocol layering • Communications stack consists of a set of services, each providing a service to the layer above, and using services of the layer below – Each service has a programming API, just like any software module • Each service has to convey information one or more peers across the network • This information is contained in a header – The headers are transmitted in the same order as the layered services 13

Protocol layering example Browser process Web server process HTTP TCP Router IP IP Link

Protocol layering example Browser process Web server process HTTP TCP Router IP IP Link 1 Physical Link 1 IP Link 2 Link 1 Physical Link 2 14

Basic elements of any protocol header • Demuxing field – Indicates which is the

Basic elements of any protocol header • Demuxing field – Indicates which is the next higher layer (or process, or context, etc. ) • Length field or header delimiter – For the header, optionally for the whole packet • Header format may be text (HTTP, SMTP (email)) or binary (IP, TCP, Ethernet) 15

Demuxing fields • Ethernet: Protocol Number – Indicates IPv 4, IPv 6, (old: Appletalk,

Demuxing fields • Ethernet: Protocol Number – Indicates IPv 4, IPv 6, (old: Appletalk, SNA, Decnet, etc. ) • IP: Protocol Number – Indicates TCP, UDP, SCTP • TCP and UDP: Port Number – Well known ports indicate FTP, SMTP, HTTP, SIP, many others – Dynamically negotiated ports indicate specific processes (for these and other protocols) • HTTP: Host field – Indicates “virtual web server” within a physical web server 16

 • IP (Internet Protocol) Three services: – Unicast: transmits a packet to a

• IP (Internet Protocol) Three services: – Unicast: transmits a packet to a specific host – Multicast: transmits a packet to a group of hosts – Anycast: transmits a packet to one of a group of hosts (typically nearest) • • Destination and source identified by the IP address (32 bits for IPv 4, 128 bits for IPv 6) All services are unreliable – Packet may be dropped, duplicated, and received in a different order • • In binary, a 32 -bit integer In text, this: “ 128. 52. 7. 243” – Each decimal digit represents 8 bits (0 – 255) • “Private” addresses are not globally unique: – Used behind NAT boxes – 10. 0/8, 172. 16. 0. 0/12, 192. 168. 0. 0/16 • Multicast addresses start with 1110 as the first 4 bits (Class D address) – 224. 0. 0. 0/4 • Unicast and anycast addresses come from the same space 17

UDP (User Datagram Protocol) • Runs above IP • Same unreliable service as IP

UDP (User Datagram Protocol) • Runs above IP • Same unreliable service as IP – Packets can get lost anywhere: • Outgoing buffer at source • Router or link • Incoming buffer at destination • But adds port numbers – Used to identify “application layer” protocols or processes • Also a checksum, optional 18

TCP (Transmission Control Protocol) • Runs above IP – Port number and checksum like

TCP (Transmission Control Protocol) • Runs above IP – Port number and checksum like UDP • Service is in-order byte stream – Application does not absolutely know how the bytes are packaged in packets • Flow control and congestion control • Connection setup and teardown phases • Can be considerable delay between bytes in at source and bytes out at destination – Because of timeouts and retransmissions • Works only with unicast (not multicast or anycast) 19

Figure 2. 8 Real-time ordering of events 20

Figure 2. 8 Real-time ordering of events 20

Figure 2. 9 Processes and channels 21

Figure 2. 9 Processes and channels 21

Figure 2. 10 Omission and arbitrary failures Class of failure Affects Description Fail-stop Process

Figure 2. 10 Omission and arbitrary failures Class of failure Affects Description Fail-stop Process halts and remains halted. Other processes may detect this state. Crash Process halts and remains halted. Other processes may not be able to detect this state. Omission Channel A message inserted in an outgoing message buffer neve arrives at the other end’s incoming message buffer. Send-omission Process A process completes send, a but the message is not put in its outgoing message buffer. Receive-omission Process A message is put in a process’s incoming message buffer, but that process does not receive it. Arbitrary Process or Process/channel exhibits arbitrary behaviour: it may (Byzantine) channel send/transmit arbitrary messages at arbitrary times, commit omissions; a process may stop or take an incorrect step. 22

Figure 2. 11 Timing failures Class of Failure Affects Clock Process Performance Channel Description

Figure 2. 11 Timing failures Class of Failure Affects Clock Process Performance Channel Description Process’s local clock exceeds the bounds on its rate of drift from real time. Process exceeds the bounds on the interval between two steps. A message’s transmission takes longer than the stated bound. 23

Figure 2. 12 Objects and principals 24

Figure 2. 12 Objects and principals 24

Figure 2. 13 The enemy Copy ofm The enemy Processp m’ m Processq Communication

Figure 2. 13 The enemy Copy ofm The enemy Processp m’ m Processq Communication channel Principal Process Secure channel Process 25

Interprocess Communications 26

Interprocess Communications 26

Interprocess Communications - Operating systems provide facilities for interprocess communications (IPC), such as message

Interprocess Communications - Operating systems provide facilities for interprocess communications (IPC), such as message queues, semaphores, and shared memory. - Distributed computing systems make use of these facilities to provide application programming interface which allows IPC to be programmed at a higher level of abstraction. - Distributed computing requires information to be exchanged among independent processes. 27

IPC – unicast and multicast • In distributed computing, two or more processes engage

IPC – unicast and multicast • In distributed computing, two or more processes engage in IPC in a protocol agreed upon by the processes. A process may be a sender at some points during a protocol, a receiver at other points. • When communication is from one process to a single other process, the IPC is said to be a unicast. When communication is from one process to a group of processes, the IPC is said to be a multicast, a topic that we will explore in a later chapter. 28

Unicast vs. Multicast 29

Unicast vs. Multicast 29

Interprocess Communications in Distributed Computing 30

Interprocess Communications in Distributed Computing 30

Operations provided in an archetypal Interprocess Communications API • Receive ( [sender], message storage

Operations provided in an archetypal Interprocess Communications API • Receive ( [sender], message storage object) • Connect (sender address, receiver address), for connection-oriented communication. • Send ( [receiver], message) • Disconnect (connection identifier), for connection-oriented communication. 31

Interprocess Communication in basic HTTP 32

Interprocess Communication in basic HTTP 32

Event Synchronization • Interprocess communication requires that the two processes synchronize their operations: one

Event Synchronization • Interprocess communication requires that the two processes synchronize their operations: one side sends, then the other receives until all data has been sent and received. • Ideally, the send operation starts before the receive operation commences. • In practice, the synchronization requires system support. 33

Synchronous vs. Asynchronous Communication • The IPC operations may provide the synchronization necessary using

Synchronous vs. Asynchronous Communication • The IPC operations may provide the synchronization necessary using blocking. A blocking operation issued by a process will block further processing of the process until the operation is fulfilled. • Alternatively, IPC operations may be asynchronous or nonblocking. An asynchronous operation issued by a process will not block further processing of the process. Instead, the process is free to proceed with its processing, and may optionally be notified by the system when the operation is fulfilled. 34

Synchronous send and receive 35

Synchronous send and receive 35

Asynchronous send and synchronous receive 36

Asynchronous send and synchronous receive 36

Asynchronous send and Asynchronous receive 37

Asynchronous send and Asynchronous receive 37

Event diagram 38

Event diagram 38

Blocking, deadlock, and timeouts • Blocking operations issued in the wrong sequence can cause

Blocking, deadlock, and timeouts • Blocking operations issued in the wrong sequence can cause deadlocks. • Deadlocks should be avoided. Alternatively, timeout can be used to detect deadlocks. 39

Using threads for asynchronous IPC • When using an IPC programming interface, it is

Using threads for asynchronous IPC • When using an IPC programming interface, it is important to note whether the operations are synchronous or asynchronous. • If only blocking operation is provided for send and/or receive, then it is the programmer’s responsibility to using child processes or threads if asynchronous operations are desired. 40

Deadlocks and Timeouts • Connect and receive operations can result in indefinite blocking •

Deadlocks and Timeouts • Connect and receive operations can result in indefinite blocking • For example, a blocking connect request can result in the requesting process to be suspended indefinitely if the connection is unfulfilled or cannot be fulfilled, perhaps as a result of a breakdown in the network. • It is generally unacceptable for a requesting process to “hang” indefinitely. Indefinite blocking can be avoided by using timeout. • Indefinite blocking may also be caused by a deadlock 41

Indefinite blocking due to a deadlock 42

Indefinite blocking due to a deadlock 42

Data Representation • Data transmitted on the network is a binary stream. • An

Data Representation • Data transmitted on the network is a binary stream. • An interprocess communication system may provide the capability to allow data representation to be imposed on the raw data. • Because different computers may have different internal storage format for the same data type, an external representation of data may be necessary. • Data marshalling is the process of (I) flatterning a data structure, and (ii) converting the data to an external representation. • Some well known external data representation schemes are: Sun XDR ASN. 1 (Abstract Syntax Notation) XML (Extensible Markup Language) 43

Data Encoding Protocols 44

Data Encoding Protocols 44

Sample XML file http: //java. sun. com/xml/docs/tutorial/overview/1_xml. html#intro • XML is a text-based markup

Sample XML file http: //java. sun. com/xml/docs/tutorial/overview/1_xml. html#intro • XML is a text-based markup language that is fast becoming the standard for data interchange on the Web. • XML has syntax analogus to HTML. • Unlike HTML, XML tags tell you what the data means, rather than how to display it. • Example: <message> <to>you@your. Address. com</to> <from>me@my. Address. com</from> <subject>XML Is Really Cool</subject> <text> How many ways is XML cool? Let me count the ways. . . </text> </message> 45

Data Marshalling 46

Data Marshalling 46

Text-based protocols • Data marshalling is at its simplest when the data exchanged is

Text-based protocols • Data marshalling is at its simplest when the data exchanged is a stream of characters, or text. • Exchanging data in text has the additional advantage that the data can be easily parsed in a program and displayed for human perusal. Hence it is a popular practice for protocols to exchange requests and responses in the form of character-strings. Such protocols are said to be textbased. • Many popular network protocols, including FTP (File Transfer Protocol), HTTP, and SMTP (Simple Mail Transfer Protocol), are text-based. 47

Event diagram 48

Event diagram 48

Protocol • In a distributed application, two processes perform interprocess communication in a mutually

Protocol • In a distributed application, two processes perform interprocess communication in a mutually agreed upon protocol. • The specification of a protocol should include (i) the sequence of data exchange, which can be described using a time event diagram. (ii) the specification of the format of the data exchanged at each step. 49

HTTP: A sample protocol • The Hypertext Transfer Protocol is a protocol for a

HTTP: A sample protocol • The Hypertext Transfer Protocol is a protocol for a process (the browser) to obtain a document from a web server process. • It is a request/response protocol: a browser sends a request to a web server process, which replies with a response. 50

The Basic HTTP protocol 51

The Basic HTTP protocol 51

A sample HTTP session 52

A sample HTTP session 52

IPC paradigms and implementations Paradigms of IPC of different levels of abstraction have evolved,

IPC paradigms and implementations Paradigms of IPC of different levels of abstraction have evolved, with corresponding implementations. 53

Group Communication 54

Group Communication 54

Coordination in distributed systems • Coordination is needed by distributed systems but hard to

Coordination in distributed systems • Coordination is needed by distributed systems but hard to achieve: – – – Events happen concurrently Communication links are not reliable Computers can crash New nodes can join the systems Asynchronous environments Need of an efficient way to coordinate a group of processes 55

A Distributed System in war: Synchronous Example Friendly fighter Radar Enemy missile Friendly fighter

A Distributed System in war: Synchronous Example Friendly fighter Radar Enemy missile Friendly fighter Missile Ready to Fire Cancel Commander Ask Prepare 56

A Distributed System in war: Reality Friendly fighter Radar Enemy missile Friendly fighter Missile

A Distributed System in war: Reality Friendly fighter Radar Enemy missile Friendly fighter Missile Ready to Fire Commander Ask Prepare 57

Group communication • What is a group? – A number of processes which cooperate

Group communication • What is a group? – A number of processes which cooperate to provide a service. – An abstract identity to name a collection of processes. • Group Communication – For coordination among processes of a group. 58

Who Needs Group Communication? • • • Highly available servers (client-server) Database Replication Multimedia

Who Needs Group Communication? • • • Highly available servers (client-server) Database Replication Multimedia Conferencing Online Games Cluster management … 59

Distributed Web Server • High availability 60

Distributed Web Server • High availability 60

Online Game • Fault-tolerance, Order 61

Online Game • Fault-tolerance, Order 61

Different Comm. Methods • Unicast – Point-to-Point Communication – Multiple copies are sent. •

Different Comm. Methods • Unicast – Point-to-Point Communication – Multiple copies are sent. • Broadcast – One-to-All Communication – Abuse of Network Bandwidth • Multicast – One-to-multiple Communication 62

Desired Group Communication • Name Abstraction • Efficiency Multicast • Delivery Guarantees Reliability, Ordering

Desired Group Communication • Name Abstraction • Efficiency Multicast • Delivery Guarantees Reliability, Ordering • Dynamic Membership Group membership service 63

Properties of Communication • Ordering – Total ordering, causal ordering • Failure behavior •

Properties of Communication • Ordering – Total ordering, causal ordering • Failure behavior • Reliability – Validity, integrity, agreement 64

Properties of Group • Name of group • Addresses of group members • Dynamic

Properties of Group • Name of group • Addresses of group members • Dynamic group membership • Options: – Peer group or client-server group – Closed or Open Group 65

Peer Group • All the members are equal. • All the members send messages

Peer Group • All the members are equal. • All the members send messages to the group. • All the members receive all the messages. 66

Client-Server Group • Replicated servers. • Clients do not care which server answers. 67

Client-Server Group • Replicated servers. • Clients do not care which server answers. 67

Multicast communication • Use network hardware support for broadcast or multicast when it is

Multicast communication • Use network hardware support for broadcast or multicast when it is available. • Send message over a distribution tree. • Minimize the time and bandwidth utilization 68

Reliability Correct processes: those that never fail. • Integrity A correct process delivers a

Reliability Correct processes: those that never fail. • Integrity A correct process delivers a message at most once. • Validity A message from a correct process will be delivered by the process eventually. • Agreement A message delivered by a correct process will be delivered by all other correct processes in the group. Validity + Agreement = Liveness 69

Ordering Assumptions: a process belongs to at most one group. p 1 • FIFO

Ordering Assumptions: a process belongs to at most one group. p 1 • FIFO p 3 – if mp m’p, all correct processes that deliver m’p will deliver mp (that is from the same sender) before m’p. • Causal – if m m’, all correct processes that deliver m’ will deliver m before m’. • Total – if a correct process delivers m before m’, all other correct processes that deliver m’ will deliver m before m’. p 2 p 1 p 2 p 3 70

Examples • Assumption: – Reliable one-to-one send operation (e. g. TCP) • Basic multicast

Examples • Assumption: – Reliable one-to-one send operation (e. g. TCP) • Basic multicast – Requirement: • All correct processes will eventually deliver the message from a correct sender. – Implementation: • B-multicast( g, m): p g: send( p, m); • On receive( m) at p: B-deliver( m) at p. Properties: integrity, validity. 71

Basic multicast: Agreement? 2 1 crash 3 X Agreement 4 72

Basic multicast: Agreement? 2 1 crash 3 X Agreement 4 72

Examples (cont. ) • Reliable multicast – Requirements: integrity, validity, agreement – Implementation: •

Examples (cont. ) • Reliable multicast – Requirements: integrity, validity, agreement – Implementation: • Received : = {}; • R-multicast( g, m) at process p: B-multicast( g, m); • On B-deliver( m) at process p from process q if( m Received) Received : = Received {m}; if( q p) B-multicast( g, m); R-deliver( m); end if Inefficient: each message is sent O(|g|) times to each process 73

Examples (cont. ) • FIFO-ordered multicast: – Assumption: • a process belongs to at

Examples (cont. ) • FIFO-ordered multicast: – Assumption: • a process belongs to at most one group. – Implementation: • Local variables at p: Sp = 1, Rp[ |g| ]={0}; • FO-multicast( g, m) at p: B-multicast( g, <m, Sp>); Sp++; • On B-deliver( <m, S>) at p from q: if( S = Rp[q] + 1) FO-deliver( m); Rp[q] : = S; else if( S > Rp[q] + 1) place <m, S> in the queue until S = Rp[q] + 1; FO-deliver( m); Rp[q] : = S; end if – Your task: totally ordered multicasts. 74

Group membership service • Four tasks: – Interface for group membership changes – Failure

Group membership service • Four tasks: – Interface for group membership changes – Failure detector – Membership change notification – Group address expansion Multicast Communication Group membership management • Group partition: – Primary-partition – one partition only – Partitionable – many partiations at once 75

Group views • Group views: – Lists of the current ordered group members –

Group views • Group views: – Lists of the current ordered group members – A new one is generated when processes join or leave/fail. • View delivery – When a member is notified of a membership change – Requirements • Order – if p delivers v(g) v’(g), no other process delivers v’(g) v(g). • Integrity – if p delivers v(g), p v(g). • Non-triviality – if q joins a group and becomes indefinitely reachable from p, eventually q is always in the view p delivers. 76

View-synchronous group comm. • Extend the reliable multicast semantics with group views. – Agreement

View-synchronous group comm. • Extend the reliable multicast semantics with group views. – Agreement • Correct processes deliver the same set of messages in any given view – Validity (closed group) • Correct processes always deliver the messages they send. • p v 0(g) does not deliver m in v 0(g) p v 1(g) for processes that deliver m. crash X p q r (p, q, r) (q, r) – Integrity 77

REMOTE INVOCATION 78

REMOTE INVOCATION 78

INTRODUCTION Middleware layers Applications This chapter (and Chapter 6) Remote invocation, indirect communication Underlying

INTRODUCTION Middleware layers Applications This chapter (and Chapter 6) Remote invocation, indirect communication Underlying interprocess communication primitives: Middleware layers Sockets, message passing, multicast support, overlay networks UDP and TCP 79

REMOTE INVOCATION • 3 Methods of programming model: – RPC: Client calls a procedure

REMOTE INVOCATION • 3 Methods of programming model: – RPC: Client calls a procedure implemented and executed on a remote computer – RMI: object invokes methods that resides on a remote computer – Event-based Distributed Programming: notifications of events hare received that is happening on remote computers/processes. 80

REQUEST-REPLY PROTOCOLS • In normal case, request-reply communication is synchronous becoz client process is

REQUEST-REPLY PROTOCOLS • In normal case, request-reply communication is synchronous becoz client process is block until the reply comes from server. • Asynchronous – where clients can afford to retrieve replies • In particular: – Acknowledgements are redundant – Connection b/w two extra pairs of msgs. – Flow control is redundant 81

Request-reply communication Client do. Operation Server Request message (wait) (continuation) Reply message get. Request

Request-reply communication Client do. Operation Server Request message (wait) (continuation) Reply message get. Request select object execute method send. Reply 82

Operations of the request-reply protocol public byte[] do. Operation (Remote. Ref s, int operation.

Operations of the request-reply protocol public byte[] do. Operation (Remote. Ref s, int operation. Id, byte[] arguments) sends a request message to the remote server and returns the reply. The arguments specify the remote server, the operation to be invoked and the arguments of that operation. public byte[] get. Request (); acquires a client request via the server port. public void send. Reply (byte[] reply, Inet. Address client. Host, int client. Port); sends the reply message reply to the client at its Internet address and port. 83

Request-reply message structure message. Type int (0=Request, 1= Reply) request. Id int remote. Reference

Request-reply message structure message. Type int (0=Request, 1= Reply) request. Id int remote. Reference Remote. Ref operation. Id int or Operation arguments array of bytes 84

Message Identifiers • Each and every message can deliver by using unique message identifier

Message Identifiers • Each and every message can deliver by using unique message identifier • It consists of 2 parts: – request. ID- Increasing the sequence number of integers by sending the process – Identifier – for sender process eg: port and inetaddress 85

Failure Model: • 3 primitives – do. Operation – get. Request – send. Reply

Failure Model: • 3 primitives – do. Operation – get. Request – send. Reply are implemented over UDP datagrams. • They also includes: – Omission Failures – Msg are not guaranteed to be delivered in sender order. • In addition, protocol also suffer from the failure of processes. Assume that processes have crash failure. When they halt, they will remain halted-they donot produce. • Timeouts: • Discarding duplicate request messages: When the request msg is retransmitted, the server may receive it more than once • Lost reply messages: 86

Style of Exchange Protocols: • 3 different protocols are used: – request (R) protocol

Style of Exchange Protocols: • 3 different protocols are used: – request (R) protocol – request-reply (RR) protocol – request-reply-acknowledge reply (RRA) protocol 87

RPC exchange protocols Name R RR RRA Messages sent by Client Server Reques t.

RPC exchange protocols Name R RR RRA Messages sent by Client Server Reques t. Reques Reply t Reques t Reply Client Acknowledge reply 88

HTTP request message method GET URL or pathname HTTP version headers message body //www.

HTTP request message method GET URL or pathname HTTP version headers message body //www. dcs. qmw. ac. uk/index. html. HTTP/ 1. 1 HTTP REPLY MESSAGE HTTP version HTTP/1. 1 status code 200 reason OK headers message body resource data 89

PUBLISH-SUBSCRIBE SYSTEMS 90

PUBLISH-SUBSCRIBE SYSTEMS 90

91

91

What is a Publish/Subscribe System? • Distributed event-based systems. • It allows any number

What is a Publish/Subscribe System? • Distributed event-based systems. • It allows any number of publishers to communicate with any number of subscribers via event channel PUBLISHER PUBLISHSUBSCRIBE SYSTEM SUBSCRIBER 92

Key components of Pub/Sub System • Publishers : Publishers generate event data and publishes

Key components of Pub/Sub System • Publishers : Publishers generate event data and publishes them. • Subscribers : Subscribers submit their subscriptions and process the events received • P/S service: It’s the mediator/broker that filters and routes events from publishers to interested subscribers. 93

Programming Model Operation Description Publish(event) Publisher publishes an event Subscribe(filter) Subscriber subscribes to a

Programming Model Operation Description Publish(event) Publisher publishes an event Subscribe(filter) Subscriber subscribes to a set of events through filter Unsubscribe(filter) Unsubscribes from a set of events Notify(event) Deliver events to subscribers Advertise(filter) Publisher declare the nature of events they will produce Unadvertise(filter) Revokes the advertisement 94

Contd… • Orders in pulish-subscribe systems: – Channel based: • publishers -publish events to

Contd… • Orders in pulish-subscribe systems: – Channel based: • publishers -publish events to named channels • Subscribers-subscribes to one of these named channels to receive all events sent to that channel – Type based • Linked with object based approaches where object have some specified type • P/S – types of events and matching in terms of types and subtypes of the filter. 95

Contd… – Topic based(else subject based): • Each notification is expressed in terms of

Contd… – Topic based(else subject based): • Each notification is expressed in terms of number of fields. • It is a well defined path from publisher to all interested subscribers • It is moreover equivalent to channel based. 96

Contd… – Content based: • P/S notifications are not classified according to some predefined

Contd… – Content based: • P/S notifications are not classified according to some predefined external criteria • Msgs from publishers do not contain any address but they are routed through the system based 97

IMPLEMETNATION OF P/S SYSTEM • It can be implemented by centralized server or by

IMPLEMETNATION OF P/S SYSTEM • It can be implemented by centralized server or by a n/w of msg routers • 3 entities: – Publishers – Subscribers – Brokers • 2 ways can be implemented – Centralized Broker model – Peer – to – Model 98

Contd… • Centralized Broker model – Consists of multiple publishers and multiple subscribers and

Contd… • Centralized Broker model – Consists of multiple publishers and multiple subscribers and centralized broker/brokers (an overlay network of brokers interacting with each other). – Subscribers/Publishers will contact 1 broker, and does not need to have knowledge about others. – E. g. CORBA event services, JMS, JEDI etc… 99

Contd… • Peer-to-Peer model – Each node can be publisher, subscriber or broker. –

Contd… • Peer-to-Peer model – Each node can be publisher, subscriber or broker. – Subscribers subscribe to publishers directly and publishers notify subscribers directly. Therefore they must maintain knowledge of each other. – Complex in nature, mechanisms such as DHT and CHORD are employed to locate nodes in the network. – E. g. Java distributed event service 100

ARCHITECTURE 101

ARCHITECTURE 101

Contd… • Matching Layer: The events can match by given subscription and this can

Contd… • Matching Layer: The events can match by given subscription and this can be pushed down towards event routing mechanisms • Flooding: Sending an event notification to all the nodes in n/w and then carry appropriate matching at subscriber. – It can be implemented through broadcast/ multicast 102

Contd… 103

Contd… 103

Contd… • Filtering: upon receive publish(event e) from node x matchlist ← match(e, subscriptions);

Contd… • Filtering: upon receive publish(event e) from node x matchlist ← match(e, subscriptions); //events & list of nodes send notify(e) to matchlist; //match the events and fwd the event to all the nodes fwdlist ← match(e, routing); //match the event from routing table and fwd send publish(e) to fwdlist −x; through path upon receive subscribe(subscription s) from node x if x is client then add s to subscriptions; else add (x, s) to routing send s to neighbors−x; 104

Contd… • Advertisements: the burden can be reduces by propagating the advertisements towards subscriber

Contd… • Advertisements: the burden can be reduces by propagating the advertisements towards subscriber • Rendezvous: upon receive publish(event e) from node x at node i rvlist ← EN(e); if i ∈ rvlist then matchlist ← match(e, subscriptions); send notify(e) to matchlist; else send(e) to rvlist; upon receive subscribe(subscription s) from node x at node i rvlist ← SN(s); if i ∈ rvlist then add s to subscriptions; else send(s) to rvlist; 105

OVERLAY INFRASTRUCUTRES 106

OVERLAY INFRASTRUCUTRES 106

Advantages of Pub/Sub • Highly suited for mobile applications, ubiquitous computing and distributed embedded

Advantages of Pub/Sub • Highly suited for mobile applications, ubiquitous computing and distributed embedded systems • Robust – Failure of publishers or subscribers does not bring down the entire system • Scalability- Suited to build distributed applications consisting a large number of entities • Adaptability- can be varied to suit different environments (mobile, internet game, embedded systems etc…) 107

Disadvantages of Pub/Sub • Reliability – no strong guarantee on broker to deliver content

Disadvantages of Pub/Sub • Reliability – no strong guarantee on broker to deliver content to subscriber. After a publisher publishes the event, it assumes that all corresponding subscribers would receive it. • Potential bottleneck in brokers when subscribers and publishers overload them. (Solve by load balancing techniques) • Security an issue: – Encryption hard to implement when the brokers has to filter out the events according to context. 108

MESSAGE QUEUES 109

MESSAGE QUEUES 109

Definition • It is also referred as Message Oriented Middleware • It provides point

Definition • It is also referred as Message Oriented Middleware • It provides point to point service • Implemented: IBM Web. Sphere MQ, Microsoft MSMQ and oracle streams AQ 110

Programming Model • It is based on FIFO • Message size can be very

Programming Model • It is based on FIFO • Message size can be very large • Operation: – – Select Destination Metadata Body of the message • 3 approaches – Blocking receive – Non blocking receive – Notify 111

Execution Modes • Sender running, Receiver running: Sender Running Receiver Running • Sender running,

Execution Modes • Sender running, Receiver running: Sender Running Receiver Running • Sender running, Receiver passive Sender Running Receiver Passive 112

Contd… • Sender passive, Receiver running Sender Passive Receiver Running Sender passive, Receiver passive

Contd… • Sender passive, Receiver running Sender Passive Receiver Running Sender passive, Receiver passive Sender Passive Receiver Passive 113

Message Brokers • It is store and fwd messaging. It interacts b/w application and

Message Brokers • It is store and fwd messaging. It interacts b/w application and other resources. • Services – Message transformation – Rule processing – Intelligent routing – Message warehousing and flow control – Repository services – Directory services – Management – APIs and adapters 114

Example: IBM’s Websphere MQ • All queues are managed by queue managers, which handle

Example: IBM’s Websphere MQ • All queues are managed by queue managers, which handle outgoing and incoming messages • Queue managers are linked together to form a certain topology and route messages toward their destination queue • Queue managers and applications can run on the same machine or on separate machines – If on separate machines, the application communicates with the queue manager using RPC 115

116

116

CASE STUDY JMS(Java Messaging Service) 117

CASE STUDY JMS(Java Messaging Service) 117

What is JMS? • A specification that describes a common way for Java programs

What is JMS? • A specification that describes a common way for Java programs to create, send, receive and read distributed enterprise messages • loosely coupled communication • Asynchronous messaging • Reliable delivery – A message is guaranteed to be delivered once and only once. • Outside the specification – Security services – Management services 118

Key Roles • JMS Consumer- is a program that receives and consumes messages •

Key Roles • JMS Consumer- is a program that receives and consumes messages • JMS Provider-Multiple system that implements the JMS Specification • Msgs-used to communicate information/w JMs client • Administered Objects – preconfigured JMS objects created by an admin for the use of clients – Connection. Factory, Destination (queue or topic) 119

JMS Administration Administrative Tool Bind(Mappi ng a name to an address) JNDI Namespace Lookup

JMS Administration Administrative Tool Bind(Mappi ng a name to an address) JNDI Namespace Lookup JMS Client Logical Connection JMS Provider 120

JMS Messaging Domains • Point-to-Point (PTP) – built around the concept of message queues

JMS Messaging Domains • Point-to-Point (PTP) – built around the concept of message queues – each message has only one consumer • Publish-Subscribe systems – uses a “topic” to send and receive messages – each message has multiple consumers 121

Point-to-Point Messaging Msg Client 1 sends consumes Client 2 Queue acknowledges 122

Point-to-Point Messaging Msg Client 1 sends consumes Client 2 Queue acknowledges 122

Publish/Subscribe Messaging subscribes Client 2 Msg delivers Client 1 publishes Topic subscribes Client 3

Publish/Subscribe Messaging subscribes Client 2 Msg delivers Client 1 publishes Topic subscribes Client 3 delivers 123

Message Consumptions • Synchronously – A subscriber or a receiver explicitly fetches the message

Message Consumptions • Synchronously – A subscriber or a receiver explicitly fetches the message from the destination by calling the receive method. – The receive method can block until a message arrives or can time out if a message does not arrive within a specified time limit. • Asynchronously – A client can register a message listener with a consumer. – Whenever a message arrives at the destination, the JMS provider delivers the message by calling the listener's on. Message() method. 124

JMS API Programming Model Connection Factory creates Logical connection( Topic Connection/Queue Connection Message Producer

JMS API Programming Model Connection Factory creates Logical connection( Topic Connection/Queue Connection Message Producer creates sends to Destination creates Session creates Msg Header, properties and body Message Consumer receives from Destination 125

JMS Client Example • Setting up a connection and creating a session Initial. Context

JMS Client Example • Setting up a connection and creating a session Initial. Context jndi. Context=new Initial. Context(); //look up for the connection factory Connection. Factory cf=jndi. Context. lookup(connectionfactoryname); //create a connection Connection connection=cf. create. Connection(); //create a session Session session=connection. create. Session (false, Session. AUTO_ACKNOWLEDGE); //create a destination object Destination dest 1=(Queue) jndi. Context. lookup(“/jms/my. Queue”); //for Point. To. Point Destination dest 2=(Topic)jndi. Context. lookup(“/jms/my. Topic”); //for publish-subscribe 126

Producer Sample • Setup connection and create a session • Creating producer Message. Producer

Producer Sample • Setup connection and create a session • Creating producer Message. Producer producer=session. create. Producer(dest 1); • Send a message Message m=session. create. Text. Message(); m. set. Text(“just another message”); producer. send(m); • Closing the connection. close(); 127

JMS Messages • Message Header – used for identifying and routing messages – contains

JMS Messages • Message Header – used for identifying and routing messages – contains vendor-specified values, but could also contain application-specific data – typically name/value pairs • Message Properties (optional) • Message Body(optional) – contains the data – five different message body types in the JMS specification 128

JMS Message Types Message Type Contains Some Methods Text. Message String get. Text, set.

JMS Message Types Message Type Contains Some Methods Text. Message String get. Text, set. Text Map. Message set of name/value pairs set. String, set. Double, set. Lo ng, get. Double, get. String Bytes. Message stream of uninterpreted bytes write. Bytes, read. Bytes Stream. Message stream of primitive values write. String, write. Double, write. Long, read. String Object. Message serialize object set. Object, get. Object 129