Coordination and Agreement Topics Distributed Mutual Exclusion n
Coordination and Agreement
Topics Distributed Mutual Exclusion n Leader Election n
Failure Assumptions n Each pair of processes connected by reliable channels. n Underlying network components may suffer failures, but reliable protocols recover. • Reliable channel eventually delivers message. No bound as in an asynchronous system. • May take a while. • Could have network partition, asynchronous connectivity.
More Failure Assumptions n Processes may only fail by crashing n n No arbitrary (Byzantine) failures Failure Detectors
Figure 11. 1 A network partition
Distributed Mutual Exclusion n Critical Section problem n n Need a solution based only on message passing. Example: Users that update a file NFS is stateless; UNIX provides filelocking service lockd. n More interesting: No server, collection of peer processes. n • Ethernet: who gets to transmit?
Algorithms for Mutual Exclusion n N asynchronous processes that do not share variables. n n Processes do not fail Message delivery is reliable • Every message that is sent is eventually delivered exactly once. n Conditions: n n ME 1: At most one process may execute in CS at any time. ME 2: Requests to enter and exit the CS eventually succeed • No deadlock, starvation. n Might want ME 3: If one request to enter the CS happenedbefore another, then entry to CS is granted in that order.
Criteria for Evaluating Algorithm n n n Bandwidth Consumed (number of messages) sent in each enter and exit operation. Client delay incurred by process at exit and entry. Algorithm’s effect on throughput of system. n Synchronization delay between one process exiting CS and the next entering it.
Simplest: Centralized Server managing a mutual exclusion token for a set of processes
Evaluation of Centralized n n ME 1, ME 2, not ME 3 Messages: n Entering critical section: • Two messages (request/grant) • Delay of round-trip n Exiting: • One release message • No delay. n n Server may become a performance bottleneck. Synchronization delay: round-trip.
A ring of processes transferring a mutual exclusion token
Evaluation of Ring: ME 1, ME 2, not ME 3 n Continuously consumes bandwidth. n Delay: n Entry: 0 to N. n Exit: 1 n n Synch delay: 1 to N
Multicast and Logical Clocks n n Basic Idea: Processes that want entry multicast a request, only enter when all other processes have replied. Conditions under which you reply insure ME 1 through ME 3. Messages are of form <T, Pi>: T is sender’s Lamport timestamp and Pi is the identifier. States: RELEASED/WANTED/HELD
Ricart and Agrawala’s algorithm On initialization state : = RELEASED; To enter the section state : = WANTED; Multicast request to all processes including self; T : = request’s timestamp; Wait until (number of replies received = (N – 1)); state : = HELD; On receipt of a request <Ti, pi> at pj (i ≠ j) if (state = HELD or (state = WANTED and (T, pj) < (Ti, pi))) then queue request from pi without replying; else reply immediately to pi; end if To exit the critical section state : = RELEASED; reply to any queued requests; request processing deferred he
Evaluation: Entry: 2(N-1) messages n Synch delay: only one message transmission time. n n Both previous had roundtrip
Figure 11. 5 Multicast synchronization 41 p 3 Reply 1 34 Reply 41 34 p 2 Reply 34
Maekawa’s Algorithm n In order to enter crit. section, not necessary for all peers to grant access. n n Associate voting set Vi with each process pi n n Only need permission from subset of peers, as long as all subsets overlap. Think of processes “voting for each other” to enter the CS. Processes ensure ME 1 by casting their votes for only one candidate. Want pi an element of Vi Want intersection of Vi and Vj nonempty for all i, j. Each voting set of same size K; each pi in M voting sets. You can get K around sqrt(N) and M-K n Easy to see twice that
Maekawa’s algorithm – Part 1 On initialization state : = RELEASED; voted : = FALSE; For pi to enter the critical section state : = WANTED; Multicast request to all processes in Vi – {pi}; Wait until (number of replies received = (K – 1)); state : = HELD; On receipt of a request from pi at pj (i ≠ j) if (state = HELD or voted = TRUE) then queue request from pi without replying; else send reply to pi; voted : = TRUE; end if Continues on next slide
Maekawa’s algorithm For pi to exit the critical section state : = RELEASED; Multicast release to all processes in Vi – {pi}; On receipt of a release from pi at pj (i ≠ j) if (queue of requests is non-empty) then remove head of queue – from pk, say; send reply to pk; voted : = TRUE; else voted : = FALSE; end if
Discussion of Maekawa’s Algorithm Achieves safety property ME 1. n Deadlock prone n n Can you give an example?
Deadlock Example n 3 processes p 1, p 2, p 3 n n If all 3 concurrently request entry to CS n n n V 1=p 1, p 2 V 2=p 2, p 3 V 3=p 3, p 1 can vote for p 2 can vote for p 3 can vote for p 1 Noone has a quorum Can be made deadlock-free.
Leader Election Choose a unique process to perform a particular role. n Essential that all processes agree on the choice. n
Leader Election n Process calls the election: initiates a run of the algorithm. n Individual algorithm does not call more than one election at a time, but N could call N. • Very important that choice of elected process is unique. n n At any point in time a process is a participant or non-participant. Wolog, require elected process be chosen as the one with the largest identifier.
Leader Election: Requirements n During any particular run of the algorithm: n n n E 1: (safety) A participant process either has not yet defined the leader or has elected P, where P is the noncrashed process at the end of the run with the largest identifier. E 2: (liveness) All processes participate and eventually select a leader or crash. Measure by n n Total number of messages sent Turnaround time: number of serialized message transmission times between initiation and termination of a single run.
Ring-based election algorithm n n n Motivated by token ring Initially everyone is a non-participant. Any process can begin an election. n Marks itself a participant, places identifier in an election message and sends it to clockwise neighbor.
n When a process receives an election message, compares identifier. n n If greater, forwards it If < and receiver is not a participant, substitutes its own identifier in message and forwards it. • Does not forward if its already a participant. n n In any case, if it forwards a message, it marks itself as a participant. If the received identifier is that of the receiver itself, this process’ identifier must be the greatest, and it becomes the coordinator. n n Coordinator marks itself non-participant once more and sends an elected message to its neighbor, announcing its election and enclosing its identity. When receives elected message, marks self as non-participant, sets its elected variable, and forwards message.
Does it work? E 1: yes. For any two processes, the one with the larger identifier will not pass on the other’s identifier. Therefore impossible that both should receive their identifier back. n E 2: Follows from guaranteed traversals of ring. n
Performance n If only a single process starts an election… n Worst case: anti-clockwise neighbor has highest identifier. • • n N-1 messages required to reach this neighbor. Wont announce its election for another N. n N for announcement. 3 N-1. Turnaround time also 3 N-1.
A ring-based election in progress (Figure 11. 7) Note: The election was started by process 17. The highest process identifier encountered so far is 24. Participant processes are shown darkened
Bully Algorithm n Allows processes to crash during an election n Message delivery between processes is reliable. Assumes system is synchronous; uses timeouts to detect a process failure. Also assumes that each process knows which processes have higher identifiers, and that it can communicate with all such processes.
n 3 types of messages: n Election • Announce an election n Answer • Sent in response to an election message n Coordinator • Announce identity of elected process. n A process begins an election when it notices, through timeouts, that coordinator failed.
n n Process that knows it has the highest identifier can elect itself coordinator by sending coordinator message to all lower numbered ones. Process with lower identifier begins an election by sending an election message to those processes that have a higher identifier and awaits an answer message in response. n n If none arrives within time T, process considers itself a coordinator and sends coordinator messages to lowernumbered processes. Otherwise, process waits a while longer for coordinator message to arrive from new coordinator. If none arrives, it starts a new election.
n n n If process receives a coordinator message it notes that as the coordinator. If a process receives an election message it sends back an answer message and begins another election, unless it has begun one already. When a process is started t. o replace a crashed process, it begins an election
Evaluation n Clearly meets liveness condition, by assumption of reliable message delivery. If no process replaced (with same identifier) meets E 1. Impossible for two processes to decide they are coordinators since process with lower identifier will discover that the other exists and defer to it. O(N^2) messages in the worst case.
The Bully Algorithm Figure 11. 8 The election of coordinator p 2 , after the failure of p 4 and then p 3
- Slides: 35