Synchronization Algorithms and Concurrent Programming Gadi Taubenfeld Chapter
Synchronization Algorithms and Concurrent Programming Gadi Taubenfeld Chapter 9 Consensus Version: June 2014 Chapter 9 Synchronization Algorithms and Concurrent Programming Gadi Taubenfeld © 2014 1
Synchronization Algorithms and Concurrent Programming ISBN: 0131972596, 1 st edition A note on the use of these ppt slides: I am making these slides freely available to all (faculty, students, readers). They are in Power. Point (2003) form so you can add, modify, and delete slides and slide content to suit your needs. They obviously represent a lot of work on my part. In return for use, I only ask the following: q That you mention their source, after all, I would like people to use my book! q That you note that they are adapted from (or perhaps identical to) my slides, and note my copyright of this material. Thanks and enjoy! Gadi Taubenfeld All material copyright 2014 Gadi Taubenfeld, All Rights Reserved To get the most updated version of these slides go to: http: //www. faculty. idc. ac. il/gadi/book. htm Chapter 9 Synchronization Algorithms and Concurrent Programming Gadi Taubenfeld © 2014 2
Chapter 9 Consensus 9. 1 The Problem 9. 2 Three Simple Consensus Algorithms 9. 3 Consensus without Memory Initialization 9. 4 Reaching Consensus Using a Shared Queue 9. 5 Impossibility of Consensus with One Faulty Process 9. 6 The Relative Power of Synchronization Primitives 9. 7 The Universality of Consensus Chapter 9 Synchronization Algorithms and Concurrent Programming Gadi Taubenfeld © 2014 3
Section 9. 1 Consensus 1 0 1 p 1 p 2 p 3 p 4 p 5 p 6 1 1 1 0 … pn 1 Each process pi has input value ini q agreement: All non-faulty processes eventually decides on the same value v. q validity: v {in 1, in 2, …, inn }. -- binary consensus ; multi-valued consensus -- we assume crash (fail-stop) failures Chapter 9 Synchronization Algorithms and Concurrent Programming Gadi Taubenfeld © 2014 4
Consensus 1 0 1 p 1 p 2 p 3 p 4 p 5 p 6 1 1 1 0 … pn 1 There is a trivial solution when there are no faults, but what happens when there are faults? Chapter 9 Synchronization Algorithms and Concurrent Programming Gadi Taubenfeld © 2014 5
Impossibility of Consensus with One Faulty Process Theorem: There is no (asynchronous) algorithm that solves the consensus problem using atomic read/write registers in the presence of a single faulty process. We give a detailed proof of this important result in Chapter 9. 5 of the book. Same result holds also for message passing systems What can be done? q Strong primitives. q Timing assumptions. q Randomization. Chapter 9 Synchronization Algorithms and Concurrent Programming Gadi Taubenfeld © 2014 6
Three Simple Consensus Algorithms Section 9. 2 Chapter 9 Synchronization Algorithms and Concurrent Programming Gadi Taubenfeld © 2014 7
RMW: n processes one 3 -valued register (trivial) 3 -valued RMW register z The Algorithm q The first process to access z sets z to its input. q All the processes adopt this value as the decision value. The algorithm is wait-free (i. e. , can tolerate any number of failures) Chapter 9 Synchronization Algorithms and Concurrent Programming Gadi Taubenfeld © 2014 8
RMW: Two processes three bits (trivial) atomic r/w bit r 1 r 2 RMW bit z 0 The Algorithm 1. Each process pi uses one bit ri to announce its input. 2. Then, each process tries to set z from 0 to 1. 3. The decision value is the input of the process that succeeds in step 2. The algorithm is wait-free Chapter 9 Synchronization Algorithms and Concurrent Programming Gadi Taubenfeld © 2014 9
RMW: Two processes two bits (not trivial) RMW bit x RMW bit 0 The algorithm is wait-free Chapter 9 y 0 Synchronization Algorithms and Concurrent Programming Gadi Taubenfeld © 2014 10
RMW: Two processes two bits (not trivial) RMW bit x RMW bit 0 y 0 Algorithm for process pi with input ini 1. If x =1 then pi decides 1 and halts, otherwise pi sets x to ini and continues. 2. If y =0 then pi sets y to 1, decides ini and halts, otherwise pi continues. If x =0 then pi decides 0 and halts, otherwise pi decides 1 ini. The algorithm is wait-free 3. Chapter 9 Synchronization Algorithms and Concurrent Programming Gadi Taubenfeld © 2014 11
RMW: Two processes two bits (not trivial) RMW bit x RMW bit 0 y 0 NO Is the following modified version of the algorithm correct? Algorithm for process pi with input ini 1. If x =1 then pi decides 1 and halts, otherwise pi sets x to ini if ini =1 than pi decides 1 and halts else pi continues. 2. If y =0 then pi sets y to 1, decides ini and halts, otherwise pi continues. 3. If x =0 then pi decides 0 and halts, otherwise pi decides 1 -ini. Chapter 9 Synchronization Algorithms and Concurrent Programming Gadi Taubenfeld © 2014 12
RMW: Three processes two bits RMW bit x RMW bit 0 y 0 IMPOSSIBLE! There is NO wait-free algorithm!!! Chapter 9 Synchronization Algorithms and Concurrent Programming Gadi Taubenfeld © 2014 13
Three processes many RMW bits and atomic register RMW bits atomic read/write registers 0 0 0 IMPOSSIBLE! There is NO wait-free algorithm!!! Theorem: There is NO consensus algorithm for n processes that can tolerate 2 faults, for any n. Chapter 9 Synchronization Algorithms and Concurrent Programming Gadi Taubenfeld © 2014 14
Two processes two atomic read/write registers atomic register x atomic register 0 y 0 IMPOSSIBLE! There is NO wait-free algorithm!!! Chapter 9 Synchronization Algorithms and Concurrent Programming Gadi Taubenfeld © 2014 15
Two processes many atomic register atomic read/write registers 0 0 0 IMPOSSIBLE! There is NO wait-free algorithm!!! Theorem: There is NO consensus algorithm for n processes that can tolerate one fault, for any n. We will prove this very interesting theorem later. Chapter 9 Synchronization Algorithms and Concurrent Programming Gadi Taubenfeld © 2014 16
RMW: many processes, four bits, at most one fault RMW bits Chapter 9 0 0 x y decision finish Synchronization Algorithms and Concurrent Programming Gadi Taubenfeld © 2014 17
RMW: n processes, four bits, at most one fault RMW bits 0 0 x y decision finish The Algorithm 1. Process 1 and process 2 execute the algorithm for two processes using x and y. 2. Then, they write the decision value into the decision bit and set finish to 1. 3. Each other process busy waits until finish=1, and then decides on the value of the decision bit. Chapter 9 Synchronization Algorithms and Concurrent Programming Gadi Taubenfeld © 2014 18
RMW: n processes, four bits, at most one fault RMW bits 0 0 x y decision finish Is it important that all the four bits are initially 0 ? No, the initial value of the decision bit is immaterial. Chapter 9 Synchronization Algorithms and Concurrent Programming Gadi Taubenfeld © 2014 19
Important Conclusion RMW bits read/write registers Theorem: There is a consensus algorithm for n processes that can tolerate one fault, for any n. Theorem: There is NO consensus algorithm for n processes that can tolerate one fault, for any n. It is not possible to implement a single RMW bit for two processes or more, from atomic read/write registers, that can tolerate one fault. Chapter 9 Synchronization Algorithms and Concurrent Programming Gadi Taubenfeld © 2014 20
Question Crash-safe semaphores Why is it not possible to implement a semaphore S for three processes or more, from RMW bits (or test-andset bits) and atomic read/write registers (no kernel calls), which supports the usual up(S) and down(S) operations, and in addition, it is possible to infer which process holds the semaphore S ? (Semaphores are defined in Section 4. 6, page 176. ) Chapter 9 Synchronization Algorithms and Concurrent Programming Gadi Taubenfeld © 2014 21
Consensus without Memory Initialization Section 9. 3 Chapter 9 Synchronization Algorithms and Concurrent Programming Gadi Taubenfeld © 2014
Consensus without Memory Initialization RMW register (not bit) ? Q: How big? A: 3 n values. Solve the consensus problem in the following model: q There are n processes that communicate via a single “large enough” RMW register. q The initial value of the register is not a priori known. q As always, processes are asynchronous. q Processes may crash. How would we do it in a With a known initial value, fault-free it can be donemodel? with a 3 valued register. Chapter 9 We assume that at most n/2 1 processes may crash. Impossible if n/2 processes may crash! Synchronization Algorithms and Concurrent Programming Gadi Taubenfeld © 2014 23
Consensus without Memory Initialization A single RMW register with 3 n values bit ? b n Chapter 9 1. 5 n -1 ? c 0 0 circle b=1 n/2 B = {0, 1} C = {0, 1. 5 n -2 n n/2 Synchronization Algorithms and Concurrent Programming Gadi Taubenfeld © 2014 24
Consensus without Memory Initialization Example: n = 10 n 10 Chapter 9 0 4 b c n = 10 B = {0, 1} C = {0, …, 13} 0 0 circle b=1 n n/2 5 n/2 10 Synchronization Algorithms and Concurrent Programming Gadi Taubenfeld © 2014 5 25
Consensus without Memory Initialization n 0 0 circle b=1 n n/2 10 5 10 n =10 n/2 5 0 4 5 11 b c The Algorithm: q Each process reads the value of the shared register, stores it, and increments c by one (modulo |C|). q A process becomes the master if it learns (by inspecting c) that more than half of the processes have waked up and the value of b has not changed. Chapter 9 Synchronization Algorithms and Concurrent Programming Gadi Taubenfeld © 2014 0 4 26
Consensus without Memory Initialization 10 Example: v=0 0 0 n =10 0 -interval Example: 0 v = 11 1 1 -interval 5 10 1 -interval 5 b c The master: q The master decides on its own input value, say v ; q if v = 0 then c 0 else c n/2 ; q if b = 0 then b 1 else b 0 ; Chapter 9 Synchronization Algorithms and Concurrent Programming Gadi Taubenfeld © 2014 one atomic action 27
Consensus without Memory Initialization 10 0 0 0 -interval 1 -interval 5 10 1 -interval n =10 5 1 5 b c The master: q continues forever setting c to the beginning of the interval corresponding to the decision. (i. e. , if v = 0 then c 0 else c n/2 ) Chapter 9 Synchronization Algorithms and Concurrent Programming Gadi Taubenfeld © 2014 28
Consensus without Memory Initialization 10 0 0 0 -interval 1 -interval 5 10 1 -interval n =10 5 1 5 b c Every other process: q q Chapter 9 find outs that a decision has been made: Ø b changed value, or Ø c has been incremented by more than n (modulo |C|) ; makes a decision according to the interval in which c lies ; Synchronization Algorithms and Concurrent Programming Gadi Taubenfeld © 2014 29
Consensus without Memory Initialization 10 0 0 0 -interval 1 -interval 5 10 1 -interval n =10 5 1 5 b c Every process: q Chapter 9 after making its decision, continues forever setting c to the beginning of the interval corresponding to its decision. Synchronization Algorithms and Concurrent Programming Gadi Taubenfeld © 2014 30
Consensus without Memory Initialization Comments: Chapter 9 q There is a fault-free solution that uses only 5 values. (Based on the See-Saw Barrier, page 215. ) q The known lower bound (when there are faults) is Ω(n 0. 63) values. Synchronization Algorithms and Concurrent Programming Gadi Taubenfeld © 2014 31
Reaching Consensus using a Shared Queue Section 9. 4 Chapter 9 Synchronization Algorithms and Concurrent Programming Gadi Taubenfeld © 2014
Reaching Consensus using a Shared Queue Requirements: q Solution must be wait-free. q Can use many queues & atomic registers Observations: q With a peek operation – trivial for many processes. q Three processes – Impossible! (without peek) q Two processes with initialized queue – trivial (without peek) q Two processes with empty queue – not trivial (without peek) Chapter 9 Synchronization Algorithms and Concurrent Programming Gadi Taubenfeld © 2014 33
Reaching Consensus using a Shared Queue Program for process pi with input ini if ini = 0 then enqueue(Q, 0) if R=0 then decide(0) else if dequeue(Q) = empty then decide(0) else decide (1) fi fi else R: =1 if dequeue(Q) = empty then decide(1) else decide (0) fi fi Chapter 9 Queue, initially empty Q Atomic bit R Synchronization Algorithms and Concurrent Programming Gadi Taubenfeld © 2014 0 34
Impossibility of Consensus with One Faulty Process Section 9. 5 Chapter 9 Synchronization Algorithms and Concurrent Programming Gadi Taubenfeld © 2014
Section 9. 5 Impossibility of Consensus with One Faulty Process Theorem: There is no (asynchronous) algorithm that solves the consensus problem using atomic read/write registers in the presence of a single faulty process. Same result holds also for message passing systems. Why? Chapter 9 Synchronization Algorithms and Concurrent Programming Gadi Taubenfeld © 2014 36
Section 9. 5 Impossibility of Consensus with One Faulty Process Notions: q run, empty run, state event Chapter 9 Synchronization Algorithms and Concurrent Programming Gadi Taubenfeld © 2014 37
bivalent & univalent bivalent step by p time bivalent 0 step by q 1 bivalent 0 -valent 0 0 0 1 1 -valent 1 1 final decision values Chapter 9 Synchronization Algorithms and Concurrent Programming Gadi Taubenfeld © 2014 38
Lemma 1 (simple) Any consensus algorithm that can tolerate one crash failure has a bivalent empty run (i. e. , bivalent initial state). bivalent 0 Chapter 9 1 Synchronization Algorithms and Concurrent Programming Gadi Taubenfeld © 2014 39
Lemma 1 Proof Any consensus algorithm that can tolerate one crash failure has a bivalent empty run. Chapter 9 0000 0001 0 0 0011 0 1 0111 1 1 Synchronization Algorithms and Concurrent Programming Gadi Taubenfeld © 2014 40
Proof of theorem Assume to the contrary that CONS is a consensus algorithm that can tolerate one crash failure. . . step by p 1 bivalent other processes may take steps, but p 1 must take at least one step by p 2 Since CONS is correct, this run can not be extended forever. step by pn step by p 1 x step by p 2 step by pn Chapter 9 for some process p, for every extension y of x the run yp is univalent. Synchronization Algorithms and Concurrent Programming Gadi Taubenfeld © 2014 41
Proof of theorem for every extension y of x the run yp is univalent single step by p x no steps by p the shortest extension of x which is Chapter 9 Synchronization Algorithms and Concurrent Programming Gadi Taubenfeld © 2014 42
Proof of theorem single step by p x no steps by p single step by p Chapter 9 option 1: single step by p Synchronization Algorithms and Concurrent Programming Gadi Taubenfeld © 2014 43
Proof of theorem single step by p x no steps by p option 2: not a step by p single step by p Chapter 9 Synchronization Algorithms and Concurrent Programming Gadi Taubenfeld © 2014 44
Proof of theorem So, in both option 1 and option 2 we get the following picture Four cases p q read write read single step by p read write x no steps by p p write q p Chapter 9 Synchronization Algorithms and Concurrent Programming Gadi Taubenfeld © 2014 45
Proof of theorem Four cases p q read write p q p Chapter 9 Synchronization Algorithms and Concurrent Programming Gadi Taubenfeld © 2014 46
Proof of theorem Four cases p p q read write q does not take steps decide 0 A contradiction Chapter 9 Synchronization Algorithms and Concurrent Programming Gadi Taubenfeld © 2014 47
Proof of theorem Four cases p p q q p q read write p does not take steps decide 0 A contradiction Chapter 9 Synchronization Algorithms and Concurrent Programming Gadi Taubenfeld © 2014 48
Proof of theorem Four cases p p read q q write read write p All processes take steps q write • different registers • same register decide 0 A contradiction Chapter 9 Synchronization Algorithms and Concurrent Programming Gadi Taubenfeld © 2014 49
Proof of theorem Four cases p p q read q write read p read write q does not take steps • different registers • same register decide 0 A contradiction Chapter 9 Synchronization Algorithms and Concurrent Programming Gadi Taubenfeld © 2014 50
Proof of theorem QED Chapter 9 Synchronization Algorithms and Concurrent Programming Gadi Taubenfeld © 2014 51
The Relative Power of Synchronization Primitives Section 9. 6 Chapter 9 Synchronization Algorithms and Concurrent Programming Gadi Taubenfeld © 2014
The Relative Power of Synchronization Primitives Definition: The consensus number of an object of type o, denoted CN(o), is the largest n for which it is possible to solve consensus for n processes using any number of objects of type o and any number of atomic registers. If no largest n exists, the consensus number of o is infinite. Examples: • CN(atomic-register) = 1 • CN(read-modify-write bit) = 2 • CN(queue) = 2 • CN(3 -valued read-modify-write register) = • CN(compare-and-swap) = Chapter 9 Synchronization Algorithms and Concurrent Programming Gadi Taubenfeld © 2014 53
Section 9. 6 The Relative Power of Synchronization Primitives Consensus Number object 1 2 atomic-register, atomic-snapshot, safe m, regularm test-and-set, fetch-and-increment, fetch-and-add, swap, queue, stack, read-modify-write bit, fetch-and-incrementm, fetch-and-addm ( m) swapm 2 m-2 m-register assignment, atomic-register m (m>1) compare-and-swap, LL/SC, sticky-bit, queue 2, 3 -valued read-modify-write, augmented-queue Om is an object in which a process is allowed to atomically access m objects of type O Chapter 9 Synchronization Algorithms and Concurrent Programming Gadi Taubenfeld © 2014 54
Section 9. 6 The Relative Power of Synchronization Primitives Consensus Number object 1 2 atomic-register, atomic-snapshot, safe m, regularm test-and-set, fetch-and-increment, fetch-and-add, swap, queue, stack, read-modify-write bit, fetch-and-incrementm, fetch-and-addm ( m) swapm 2 m-2 m-register assignment, atomic-register m (m>1) compare-and-swap, LL/SC, sticky-bit, queue 2, 3 -valued read-modify-write, augmented-queue Theorem: Let o 1 and o 2 be two objects such that CN(o 1) < CN(o 2). Then, in a system with CN(o 2) processes, q There is no wait-free implementation of an object of type o 2 from objects of type o 1 and atomic registers; q Chapter 9 There is a wait-free implementation of an object of type o 1 from objects of type o 2 and atomic registers. Synchronization Algorithms and Concurrent Programming Gadi Taubenfeld © 2014 55
Section 9. 6 The Relative Power of Synchronization Primitives Consensus Number object 1 2 atomic-register, atomic-snapshot, safe m, regularm test-and-set, fetch-and-increment, fetch-and-add, swap, queue, stack, read-modify-write bit, fetch-and-incrementm, fetch-and-addm ( m) swapm 2 m-2 m-register assignment, atomic-register m (m>1) compare-and-swap, LL/SC, sticky-bit, queue 2, 3 -valued read-modify-write, augmented-queue Theorem: Let o 1 and o 2 be two objects with consensus number n. Then, using any number of additional atomic registers, q The objects o 1 and o 2 can wait-free implement each other when the number of processes is less than or equal to n; q Chapter 9 The objects o 1 and o 2 do not necessarily wait-free implement each other when the number of processes is more than n. Synchronization Algorithms and Concurrent Programming Gadi Taubenfeld © 2014 56
Open Question Consensus Number 1 2 object atomic-register test-and-set, fetch-and-increment, fetch-and-add, swap, queue, stack, read-modify-write bit ( m) 2 m-2 Chapter 9 Synchronization Algorithms and Concurrent Programming Gadi Taubenfeld © 2014 57
The Universality of Consensus Section 9. 7 Chapter 9 Synchronization Algorithms and Concurrent Programming Gadi Taubenfeld © 2014
Section 9. 7 Universality Definition: An object o is universal for n processes if Ø any object that has sequential specification has Ø wait-free Ø linearizable implementation Ø using atomic registers and objects of type o Ø in a system with n processes. Chapter 9 Synchronization Algorithms and Concurrent Programming Gadi Taubenfeld © 2014 59
Section 9. 7 The Universality of Consensus Definition: An object o is universal for n processes if Ø any object that has sequential specification has Ø wait-free Ø linearizable implementation Ø using atomic registers and objects of type o Ø in a system with n processes. Theorem: A consensus object for n processes is universal in a system with n processes, for any positive n. Chapter 9 Synchronization Algorithms and Concurrent Programming Gadi Taubenfeld © 2014 60
Section 9. 6 Consensus Numbers & Universality Consensus Number object 1 2 atomic-register, atomic-snapshot, consensus(1) test-and-set, fetch-and-increment, fetch-and-add, swap, queue, stack, consensus(2) fetch-and-incrementm, fetch-and-addm ( m) swapm consensus( m) 2 m-2 m-register assignment, consensus(2 m-2) compare-and-swap, LL/SC, sticky-bit, queue 2, 3 -valued read-modify-write, consensus( ) Theorem: A consensus object for n processes is universal in a system with n processes, for any positive n. Chapter 9 Synchronization Algorithms and Concurrent Programming Gadi Taubenfeld © 2014 61
Section 9. 7 The Universality of Consensus: Proof A detailed proof appears in Section 9. 7 Chapter 9 Synchronization Algorithms and Concurrent Programming Gadi Taubenfeld © 2014 62
- Slides: 62