Synchronization Algorithms and Concurrent Programming Gadi Taubenfeld Chapter
Synchronization Algorithms and Concurrent Programming Gadi Taubenfeld Chapter 2 Mutual Exclusion using atomic registers: Basic Topics Version: June 2014 Chapter 2 Synchronization Algorithms and Concurrent Programming Gadi Taubenfeld © 2014 1
Synchronization Algorithms and Concurrent Programming ISBN: 0131972596, 1 st edition A note on the use of these power-point slides: I am making these slides freely available to all (faculty, students, readers). They are in Power. Point form so you can add, modify, and delete slides and slide content to suit your needs. They obviously represent a lot of work on my part. In return for use, I only ask the following: q That you mention their source, after all, I would like people to use my book! q That you note that they are adapted from (or perhaps identical to) my slides, and note my copyright of this material. Thanks and enjoy! Gadi Taubenfeld All material copyright 2014 Gadi Taubenfeld, All Rights Reserved To get the most updated version of these slides go to: http: //www. faculty. idc. ac. il/gadi/book. htm Chapter 2 Synchronization Algorithms and Concurrent Programming Gadi Taubenfeld © 2014 2
Chapter 2 Mutual Exclusion using atomic registers: Basic Topics 2. 1 Algorithms for Two Processes 2. 2 Tournament Algorithms 2. 3 A Fast Mutual Exclusion Algorithms 2. 4 Starvation-free Algorithms 2. 5 Tight Space bounds 2. 6 Automatic Discovery of algorithms Chapter 2 Synchronization Algorithms and Concurrent Programming Gadi Taubenfeld © 2014 3
The Mutual Exclusion Problem Basic definitions Chapter 1 Synchronization Algorithms and Concurrent Programming Gadi Taubenfeld © 2014 4
The mutual exclusion problem remainder code entry code critical section exit code The problem is to design the entry and exit code in a way that guarantees that the mutual exclusion and deadlock-freedom properties are satisfied. Chapter 2 Synchronization Algorithms and Concurrent Programming Gadi Taubenfeld © 2014 5
The mutual exclusion problem q Mutual Exclusion: No two processes are in their critical sections at the same time. q Deadlock-freedom: If a process is trying to enter its critical section, then some process, not necessarily the same one, eventually enters its critical section. q Starvation-freedom: If a process is trying to enter its critical section, then this process must eventually enter its critical section. Chapter 2 Synchronization Algorithms and Concurrent Programming Gadi Taubenfeld © 2014 remainder entry code critical section exit code 6
Question: true or false ? Algorithm A Algorithm B Algorithm C remainder entry code of A remainder entry code of B entry code critical section exit code of B exit code Chapter 2 Synchronization Algorithms and Concurrent Programming Gadi Taubenfeld © 2014 exit code of A 7
Question: true or false ? Algorithm C q A and B are deadlock-free C is deadlock-free. q A and B are starvation-free C is starvation-free. q A or B satisfies mutual exclusion C satisfies mutual exclusion. remainder entry code of A entry code of B q A is deadlock-free and B is starvation critical section q A is starvation-free and B is deadlock exit code of B -free C is starvation-free. Chapter 2 Synchronization Algorithms and Concurrent Programming Gadi Taubenfeld © 2014 exit code of A 8
Atomic read/write registers Proposed solution 1 entry code exit code Thread 0 while (true} { remainder code while (turn = 1) {skip} critical section turn = 1 } Thread 1 while (true} { remainder code while (turn = 0) {skip} critical section turn = 0 } turn 0/1 ümutual exclusion C deadlock-freedom Does it work? Chapter 2 Synchronization Algorithms and Concurrent Programming Gadi Taubenfeld © 2014 9
Convention Thread 0 while (true} { remainder code entry code critical section exit code } Chapter 2 Thread 1 while (true} { remainder code } Synchronization Algorithms and Concurrent Programming Gadi Taubenfeld © 2014 10
Convention Thread 0 Chapter 2 Thread 1 entry code critical section exit code Synchronization Algorithms and Concurrent Programming Gadi Taubenfeld © 2014 11
Proposed solution 2 Thread 0 Thread 1 flag[0] = true flag[1] = true while (flag[1]) {skip} while (flag[0]) {skip} critical section flag[0] = false flag[1] = false flag 0 false 1 false ümutual exclusion C deadlock-freedom Does it work? Chapter 2 Synchronization Algorithms and Concurrent Programming Gadi Taubenfeld © 2014 12
Proposed solution 3 Thread 0 Thread 1 while (flag[1]) {skip} while (flag[0]) {skip} flag[0] = true critical section flag[0] = false flag[1] = true critical section flag[1] = false flag 0 false 1 false C mutual exclusion üDeadlock-freedom Does it work? Chapter 2 Synchronization Algorithms and Concurrent Programming Gadi Taubenfeld © 2014 13
Algorithms for Two Processes Section 2. 1 Chapter 2 Synchronization Algorithms and Concurrent Programming Gadi Taubenfeld © 2014 14
Section 2. 1. 1 Peterson’s algorithm Thread 0 flag[0] = true turn = 1 while (flag[1] and turn = 1) {skip} critical section flag[0] = false Thread 1 flag[1] = true turn = 0 while (flag[0] and turn = 0) {skip} critical section flag[1] = false flag 0 false 1 false turn 0/1 Chapter 2 Synchronization Algorithms and Concurrent Programming Gadi Taubenfeld © 2014 15
A variant of Peterson’s algorithm Is it correct ? Thread 0 turn = 1 flag[0] = true while (flag[1] and turn = 1) {skip} critical section flag[0] = false Thread 1 turn = 0 flag[1] = true while (flag[0] and turn = 0) {skip} critical section flag[1] = false flag true 0 false 1 false true turn 0/1 0 1 Chapter 2 Synchronization Algorithms and Concurrent Programming Gadi Taubenfeld © 2014 16
Schematic for Peterson’s algorithm Indicate contending b[i] : = true Barrier turn : = i Contention? b[i] = true ? no / maybe yes First to cross the barrier? turn = j ? no critical section yes exit code b[i] = false ? Chapter 2 Synchronization Algorithms and Concurrent Programming Gadi Taubenfeld © 2014 17
Properties of Peterson’s Solution q Satisfies mutual exclusion and starvation-freedom q Memory accesses are assumed to be atomic q Solution for two processes only Chapter 2 Synchronization Algorithms and Concurrent Programming Gadi Taubenfeld © 2014 18
Section 2. 1. 2 Question Using only single-writer registers Replace the multi-writer register turn with two single-writer registers. What is new algorithm? Hint: use Solution #4 for the too-much-milk problem. Answer (Kessels’ Alg. ) turn = 0 turn[0] = turn[1] turn = 1 turn[0] turn[1] Chapter 2 Synchronization Algorithms and Concurrent Programming Gadi Taubenfeld © 2014 19
Algorithms for Many Processes Section 2. 2 How can we use two-process algorithm to construct an algorithm for many processes? Chapter 2 Synchronization Algorithms and Concurrent Programming Gadi Taubenfeld © 2014 20
Section 2. 2 Tournament Algorithms Assume that the base two. Yes process algorithm satisfies starvation-freedom. Does the tournament algorithm also satisfies starvation-freedom ? 1 Chapter 2 2 3 4 5 6 7 8 Synchronization Algorithms and Concurrent Programming Gadi Taubenfeld © 2014 21
Models, Properties & Complexity Basic definitions and notations Chapter 2 Synchronization Algorithms and Concurrent Programming Gadi Taubenfeld © 2014 22
Shared Memory Models P . . . P P . . . C M P C . . . M P M M Simple Shared Memory Chapter 2 P Coherent Caching Distributed Shared Memory Synchronization Algorithms and Concurrent Programming Gadi Taubenfeld © 2014 23
Properties & complexity q Time complexity Fast m Adaptive Fairness m FIFO, . . . Fault-tolerance Local spinning Space complexity Communication primitives m q q q Chapter 2 Synchronization Algorithms and Concurrent Programming Gadi Taubenfeld © 2014 24
Complexity Measures q Counting steps contention-free time complexity m process step complexity q Counting time units m system response time m process response time q Counting communications m distributed shared memory m coherent caching q Space complexity m Chapter 2 Synchronization Algorithms and Concurrent Programming Gadi Taubenfeld © 2014 25
Contention-free time complexity What is the contention-free time complexity? Thread 0 flag[0] = true turn = 1 while (flag[1] and turn = 1) {skip} critical section flag[0] = false Thread 1 flag[1] = true turn = 0 while (flag[0] and turn = 0) {skip} critical section flag[1] = false 4 (3 in the entry + 1 in the exit) Chapter 2 Synchronization Algorithms and Concurrent Programming Gadi Taubenfeld © 2014 26
Process time complexity q Informal: The longest black interval, P 1 P 2 counting steps. q Formal: The max # of (shared memory) steps a winning process need to take in its entry/exit sections since the last time some process released its critical section. time q Process step complexity: # of steps, since it has started… Given an algorithm divide it steps … • A – entry/exit steps; • B – critical section steps; • C – other steps (remainder). Chapter 2 A single Synchronization Algorithms and Concurrent Programming Gadi Taubenfeld © 2014 run 27
Process time complexity What is the process time/step complexity? Thread 0 flag[0] = true turn = 1 while (flag[1] and turn = 1) {skip} critical section flag[0] = false Thread 1 flag[1] = true turn = 0 while (flag[0] and turn = 0) {skip} critical section flag[1] = false Theorem: There is no two-process mutual exclusion with an upper bound on the process time/step complexity. [1992] ∞ Proof: see Section 3. 2. 5. Chapter 2 Synchronization Algorithms and Concurrent Programming Gadi Taubenfeld © 2014 28
System response time P 1 P 2 one time unit time A single run Chapter 2 Synchronization Algorithms and Concurrent Programming Gadi Taubenfeld © 2014 29
System response time q Informal: The longest black interval, P 1 P 2 counting time units. q Formal: The longest time interval where some process is in its entry code while no process is in its CS, assuming an upper bound of one time unit for step time, and no lower bound. time Given an algorithm divide it steps … • A – entry/exit steps; • B – critical section steps; • C – other steps (remainder). Chapter 2 A single Synchronization Algorithms and Concurrent Programming Gadi Taubenfeld © 2014 run 30
How many time units P 1 P 2 1 2 3 time 4 5 6 7 A single run Chapter 2 Synchronization Algorithms and Concurrent Programming Gadi Taubenfeld © 2014 31
How many time units P 1 P 2 1 2 3 4 5 6 time 7 8 9 10 A single run Chapter 2 Synchronization Algorithms and Concurrent Programming Gadi Taubenfeld © 2014 32
What is the system response time? Show a run of Peterson’s algorithm where the system response time is 5 time units. Thread 0 flag[0] = true turn = 1 while (flag[1] and turn = 1) {skip} critical section flag[0] = false 1 Chapter 2 2 3 4 Thread 1 flag[1] = true turn = 0 while (flag[0] and turn = 0) {skip} critical section flag[1] = false P 1 P 2 1 2 3 4 5 5 Synchronization Algorithms and Concurrent Programming Gadi Taubenfeld © 2014 33
A Fast Mutual exclusion Algorithm Section 2. 3 Chapter 2 Synchronization Algorithms and Concurrent Programming Gadi Taubenfeld © 2014 34
A Fast Mutual exclusion Algorithm q Mutual exclusion and deadlock-freedom q Starvation of individual processes is possible q fast access in the absence of contention, only 7 accesses to shared memory are needed q With contention m Even if only 2 processes contend, the winner may need to check all the 0(n) shared registers m System response time is of order n time units q n+2 shared registers are used m Chapter 2 Synchronization Algorithms and Concurrent Programming Gadi Taubenfeld © 2014 35
Splitter 1 q At most n-1 can move right 2 3 4 5 n q At most n-1 can move down q At most 1 can win q In solo run 1 win 1 n-1 right n-1 down x=i if y 0 then go right fi y=1 if x i then go down fi win Chapter 2 x y 0 Synchronization Algorithms and Concurrent Programming Gadi Taubenfeld © 2014 36
Notation await (condition) == while (not condition) do skip od Example: await (x=5) == wait until x=5 Chapter 2 Synchronization Algorithms and Concurrent Programming Gadi Taubenfeld © 2014 37
Fast Mutual exclusion Algorithm code of process i , i {1 , . . . , n} 1 x y 0 b false 2 false n false start: b[i] = true x=i right if y 0 then b[i] = false await y = 0 goto start fi y=i if x i then b[i] = false for j = 1 to n do await b[j] = false od win if y i then await y = 0 goto start fi fi critical section down y=0 b[i] = false Chapter 2 Synchronization Algorithms and Concurrent Programming Gadi Taubenfeld © 2014 38
Can we switch the order of the last two statements? What is the maximum number of processes that can be in their CS at the same time? start: b[i] = true x=i if y 0 then b[i] = false await y = 0 goto start fi y=i if x i then b[i] = false for j = 1 to n do await b[j] = false od if y i then await y = 0 goto start fi fi critical section b[i] = false y=0 x Chapter 2 y 0 Synchronization Algorithms and Concurrent Programming Gadi Taubenfeld © 2014 39
Fast Mutual exclusion Algorithm code of process i , i {1 , . . . , n} 1 x y 0 b false 2 false n false start: b[i] = true x=i Can we replace the if y 0 then b[i] = false order of these two await y = 0 statements ? goto start fi y=i if x i then b[i] = false for j = 1 to n do await b[j] = false od if y i then await y = 0 goto start fi fi critical section y=0 No. Mutual exclusion is not satisfied. b[i] = false Chapter 2 Synchronization Algorithms and Concurrent Programming Gadi Taubenfeld © 2014 40
Can we replace y=0 in line 12 with. . . ? start: b[i] = true x=i if y 0 then b[i] = false await y = 0 goto start fi y=i if x i then b[i] = false for j = 1 to n do await b[j] = false od if y i then await y = 0 goto start fi fi critical section y = 0 replace with: if y = i then y = 0 fi b[i] = false Yes Chapter 2 Synchronization Algorithms and Concurrent Programming Gadi Taubenfeld © 2014 41
Can we remove await y=0 in line 4 ? start: b[i] = true x=i if y 0 then b[i] = false remove ? await y = 0 goto start fi y=i if x i then b[i] = false for j = 1 to n do await b[j] = false od if y i then await y = 0 goto start fi fi critical section y=0 b[i] = false No. Deadlock is possible Chapter 2 Synchronization Algorithms and Concurrent Programming Gadi Taubenfeld © 2014 42
Can we remove await y=0 in line 9 ? start: b[i] = true x=i if y 0 then b[i] = false await y = 0 goto start fi y=i if x i then b[i] = false for j = 1 to n do await b[j] = false od if y i then await y = 0 remove ? goto start fi fi critical section y=0 b[i] = false Yes Chapter 2 Synchronization Algorithms and Concurrent Programming Gadi Taubenfeld © 2014 43
Would this work ? add start: b[i] = true and x=i if y 0 and y =i then b[i] = false await y = 0 remove goto start fi y=i if x i then b[i] = false for j = 1 to n do await b[j] = false od if y i then await y = 0 goto start fi fi critical section y=0 No. Mutual exclusion b[i] = false is not satisfied. Chapter 2 Synchronization Algorithms and Concurrent Programming Gadi Taubenfeld © 2014 44
Schematic for the fast algorithm Indicate contending b[i] : = true Contention? y=0? yes Wait until CS is released no The last to cross the barrier! Barrier turn : = i Contention? x i? yes Continue only after it is guaranteed that no one can cross the barrier yes Last to cross the barrier? y=i? no critical section no exit code Chapter 2 Wait until CS is released Synchronization Algorithms and Concurrent Programming Gadi Taubenfeld © 2014 45
Starvation-free Algorithms Section 2. 4. 1 Basic Definitions (i. e. , FIFO) 2. 4. 2 The Bakery Algorithm 2. 4. 3 The Black-White Bakery Algorithm Chapter 2 Synchronization Algorithms and Concurrent Programming Gadi Taubenfeld © 2014 46
The mutual exclusion problem q Mutual Exclusion q Deadlock-freedom q Starvation-freedom remainder doorway waiting entry code critical section q FIFO exit code Chapter 2 Synchronization Algorithms and Concurrent Programming Gadi Taubenfeld © 2014 47
Section 2. 4. 2 The Bakery Algorithm entry remainder 1 2 3 4 5 n 0 0 0 doorway 1 2 3 2 4 waiting 1 2 3 2 4 CS 1 2 2 exit 1 2 2 Chapter 2 Synchronization Algorithms and Concurrent Programming Gadi Taubenfeld © 2014 time 48
Implementation 1 code of process i , i {1 , . . . , n} number[i] = 1 + max {number[j] | (1 j n)} for j = 1 to n { await (number[j] = 0) (number[j] > number[i]) } critical section number[i] = 0 number 1 2 3 4 0 0 n 0 0 integer Answer: No! can deadlock Chapter 2 Synchronization Algorithms and Concurrent Programming Gadi Taubenfeld © 2014 49
Implementation 1: deadlock entry remainder 1 2 3 4 5 n 0 0 0 doorway 1 2 2 waiting 1 2 2 CS 1 deadlock exit 1 Chapter 2 Synchronization Algorithms and Concurrent Programming Gadi Taubenfeld © 2014 time 50
Implementation 1 code of process i , i {1 , . . . , n} number[i] = 1 + max {number[j] | (1 j n)} for j = 1 to n { await (number[j] = 0) (number[j] > number[i]) } critical section What if we replace > with ? number[i] = 0 number 1 2 3 4 0 0 n 0 0 integer Answer: does not satisfy mutual exclusion Chapter 2 Synchronization Algorithms and Concurrent Programming Gadi Taubenfeld © 2014 51
Implementation 2 code of process i , i {1 , . . . , n} number[i] = 1 + max {number[j] | (1 j n)} for j = 1 to n { await (number[j] = 0) (number[j], j) number[i], i) // lexicographical order } critical section number[i] = 0 number 1 2 3 4 0 0 n 0 0 integer Answer: does not satisfy mutual exclusion Chapter 2 Synchronization Algorithms and Concurrent Programming Gadi Taubenfeld © 2014 52
Implementation 2: no mutual exclusion remainder 1 2 3 4 5 n 0 0 0 entry 2 doorway 0 1 0 2 waiting 1 2 2 CS 1 2 2 exit 1 Chapter 2 Synchronization Algorithms and Concurrent Programming Gadi Taubenfeld © 2014 time 53
The Bakery Algorithm code of process i , i {1 , . . . , n} choosing[i] = true number[i] = 1 + max {number[j] | (1 j n)} choosing[i] = false for j = 1 to n { await choosing[j] = false await (number[j] = 0) (number[j], j) (number[i], i) } critical section number[i] = 0 choosing number Chapter 2 1 2 3 4 n false false 0 0 0 Synchronization Algorithms and Concurrent Programming Gadi Taubenfeld © 2014 bits integer 54
Properties of the Bakery Algorithm q Satisfies mutex & FIFO. q The size of number[i] is unbounded. q Safe registers: reads which are concurrent with writes may return arbitrary value. Chapter 2 Synchronization Algorithms and Concurrent Programming Gadi Taubenfeld © 2014 55
Is the following version correct? code of process i , i {1 , . . . , n} choosing[i] = true number[i] = 1 + max {number[j] | (1 j n)} choosing[i] = false for j = 1 to n { if i > j { await choosing[j] = false } await (number[j] = 0) (number[j], j) (number[i], i) } critical section number[i] = 0 choosing number Chapter 2 1 2 3 4 n false false 0 0 0 Synchronization Algorithms and Concurrent Programming Gadi Taubenfeld © 2014 bits integer 57
No entry remainder 1 2 3 4 5 n 0 0 0 doorway 0 1 0 2 0 1 waiting 1 2 1 CS 1 2 1 exit 1 Chapter 2 1 Synchronization Algorithms and Concurrent Programming Gadi Taubenfeld © 2014 time 58
Is the following version correct? code of process i , i {1 , . . . , n} Yes number[i] = -1 number[i] = 1 + max {number[j] | (1 j n) , 0} for j = 1 to n { await number[j] -1 await (number[j] 0) (number[j], j) (number[i], i) } Can we replace with = ? critical section number[i] = 0 choosing number Chapter 2 1 2 3 4 n false false 0 0 0 Synchronization Algorithms and Concurrent Programming Gadi Taubenfeld © 2014 bits integer 59
Computing the Maximum #1 code of process i , i {1 , . . . , n} Correct implementation number[i] = 1 + max {number[j] | (1 j n)} local 1 = 0 for local 2 = 1 to n do local 3 = number[local 2] if local 1 < local 3 then local 1 = local 3 fi od number[i] = 1 + local 1 number Chapter 2 1 0 2 0 3 0 4 0 0 Synchronization Algorithms and Concurrent Programming Gadi Taubenfeld © 2014 n 0 integer 60
Computing the Maximum #2 code of process i , i {1 , . . . , n} Is the following implementation correct ? number[i] = 1 + max {number[j] | (1 j n)} local 1 = i for local 2 = 1 to n do if number[local 1] < number[local 2] then local 1 = local 2 fi od number[i] = 1 + number[local 1] number Chapter 2 1 0 2 0 3 0 4 0 0 Synchronization Algorithms and Concurrent Programming Gadi Taubenfeld © 2014 n 0 integer 61
local 1 No 2 ? 2 3 4 5 n remainder 0 0 0 doorway ? 1 1 1 waiting 1 1 1 CS 1 1 1 entry 1 exit Chapter 2 Passed process 1 Waiting for process 2 time 1 Synchronization Algorithms and Concurrent Programming Gadi Taubenfeld © 2014 62
Computing the Maximum #3 code of process i , i {1 , . . . , n} Is the following implementation correct ? A difficult question !!! local 1 = i for local 2 = 1 to n do if number[local 1] number[local 2] then local 1 = local 2 fi od number[i] = 1 + number[local 1] This is Problem 2. 39 Chapter 2 Synchronization Algorithms and Concurrent Programming Gadi Taubenfeld © 2014 63
Section 2. 4. 3 The Black-White Bakery Algorithm Bounding the space of the Bakery Algorithm Bakery (FIFO, unbounded) The Black-White Bakery Algorithm FIFO Bounded space + one bit Chapter 2 Synchronization Algorithms and Concurrent Programming Gadi Taubenfeld © 2014 64
The Black-White Bakery Algorithm color bit entry remainder 1 2 3 4 5 n 0 0 0 doorway 0 1 0 2 waiting 1 2 CS 1 2 exit 1 2 Chapter 2 Synchronization Algorithms and Concurrent Programming Gadi Taubenfeld © 2014 time 65
The Black-White Bakery Algorithm Data Structures 1 2 3 4 n choosing bits mycolor number bits {0, 1, . . . , n} color bit Chapter 2 {black, white} Synchronization Algorithms and Concurrent Programming Gadi Taubenfeld © 2014 66
The Black-White Bakery Algorithm code of process i , i {1 , . . . , n} choosing[i] = true mycolor[i] = color number[i] = 1 + max{number[j] | (1 j n) (mycolor[j] = mycolor[i])} choosing[i] = false for j = 0 to n do await choosing[j] = false if mycolor[j] = mycolor[i] then await (number[j] = 0) (number[j], j) (number[i], i) (mycolor[j] mycolor[i]) else await (number[j] = 0) (mycolor[i] color) (mycolor[j] = mycolor[i]) fi od critical section if mycolor[i] = black then color = white else color = black fi number[i] = 0 Chapter 2 Synchronization Algorithms and Concurrent Programming Gadi Taubenfeld © 2014 67
Question What happens if in the algorithm each process chooses its identifier instead of choosing 1 + max ? Would the algorithm be correct? What properties would it satisfy? Answer: Incorrect. Run process i first until it enters its CS; Now run a process with a smaller id until it also enters its CS. Chapter 2 Synchronization Algorithms and Concurrent Programming Gadi Taubenfeld © 2014 68
Question Does it matter if we change the order of the last two statements (in the exit code) of the Black-White Bakery Algorithm? Answer: Yes, it matters. Chapter 2 Synchronization Algorithms and Concurrent Programming Gadi Taubenfeld © 2014 69
Question Replace the multi-writer bit color with n single-writer bits. Bakery Black-White Bakery Use single-writer bits only! Answer: See page 57. Chapter 2 Synchronization Algorithms and Concurrent Programming Gadi Taubenfeld © 2014 70
Tight Space Bounds Section 2. 5 Chapter 2 Synchronization Algorithms and Concurrent Programming Gadi Taubenfeld © 2014 71
A Simple Observation A very simple Lower Bound Theorem: Any deadlock-free mutual exclusion algorithm for n processes using only SWMR registers must use at least n such registers. Proof: Before entering its critical section a process must write at least once … Question: Can we do better using MWMR registers ? Answer: No. (SWMR == Single Writer Multi Reader) Chapter 2 Synchronization Algorithms and Concurrent Programming Gadi Taubenfeld © 2014 72
Tight Space Bounds Section 2. 5. 1 A Lower Bound (very difficult to prove!) Theorem: Any deadlock-free mutual exclusion algorithm for n processes must use at least n shared registers. Proof: (see next few slides) Chapter 2 Synchronization Algorithms and Concurrent Programming Gadi Taubenfeld © 2014 73
Definitions q run – a sequence of events q run x looks like run y to process p q process p is hidden in run x q process p covers register r in run x Chapter 2 Synchronization Algorithms and Concurrent Programming Gadi Taubenfeld © 2014 74
Example run x looks like run y to process p • • • run x p reads 5 from r q writes 6 to r p writes 7 to r q writes 8 to r p reads 8 from r q write 6 to r 1 r 68 • • • run y p reads 5 from r p writes 7 to r q writes 6 to r q reads 6 from r q writes 8 to r p reads 8 from r r 8 The values of the shared registers must also be the same Chapter 2 Synchronization Algorithms and Concurrent Programming Gadi Taubenfeld © 2014 75
Example process p is hidden in run x • • • p reads 5 from r q reads 5 from r p writes 7 to r q writes 8 to r p reads 8 from r q writes 6 from r r writes must be overwritten before any other process has read the value written Chapter 2 Synchronization Algorithms and Concurrent Programming Gadi Taubenfeld © 2014 76
Example process p covers register r in run x • • p reads 5 from r q reads 5 from r p writes 7 to r q writes 8 to r p reads 8 from r p writes 2 to r p covers r at these three points r writes must be overwritten before any other process has read the value written Chapter 2 Synchronization Algorithms and Concurrent Programming Gadi Taubenfeld © 2014 77
Lemma 2. 17 (very simple) run x looks like run y to processes in P P events only x y then, this is also a run z Proof: By induction on the number of events in (z-x) … Chapter 2 Synchronization Algorithms and Concurrent Programming Gadi Taubenfeld © 2014 78
Lemma 2. 18 (very simple) If p is in its critical section in run z then p is not hidden in z. p is in its critical section then, p is not hidden z Proof: Assume to the contrary … by Lemma 2. 17 … Chapter 2 Synchronization Algorithms and Concurrent Programming Gadi Taubenfeld © 2014 79
Lemma 2. 19 Then, for any p there exists y such that x looks like y to p all the processes are hidden y all processes, except maybe p, are in their remainders x Proof: By induction on the number of steps of processes other than p in x. . . Chapter 2 Synchronization Algorithms and Concurrent Programming Gadi Taubenfeld © 2014 80
Lemma 2. 19: Example 5 r • • • Chapter 2 w is in its remainder w reads 5 q is in its p reads 5 remainder q reads 5 p writes 7 q writes 8 q reads 8 w writes 3 w reads 3 p write 9 p in its critical section p is in its p exits ü w is hidden remove w ü important: q is still hidden !!! ü q is hidden remove q Formal proof: By induction on the number of steps of processes other than p in x. . . remainder Synchronization Algorithms and Concurrent Programming Gadi Taubenfeld © 2014 81
Lemma 2. 20 (simple) all the processes are hidden x Then, for every p there exists z only p takes steps z Chapter 2 p covers unique register Synchronization Algorithms and Concurrent Programming Gadi Taubenfeld © 2014 82
Lemma 2. 20: Proof By Lemma 2. 19: there exists y … By the deadlock-freedom assumption … By Lemma 2. 17 … … p is hidden … imp. by Lemma 2. 18 x looks like y to p all processes, except maybe p, are in their remainders all the processes are hidden x p covers unique register z p is in its critical section Chapter 2 Synchronization Algorithms and Concurrent Programming Gadi Taubenfeld © 2014 83
Lemma 2. 21 (main lemma) all the processes are in their remainder x Then, for every set of processes P there exists z only P take steps z P are hidden and cover |P| distinct registers Proof: By induction on k the size of P. Chapter 2 Synchronization Algorithms and Concurrent Programming Gadi Taubenfeld © 2014 84
Lemma 2. 21 (main lemma) Proof: By induction on k the size of P. q Base: k = 0 is trivial. q Induction hypothesis: Assume that it holds for k 0. q Step: We prove that it holds for k+1. m m m let P be a set of processes where |P|= k; let p be a process not in P. We show that: only P+p take steps z Chapter 2 P+p are hidden and cover |P+p| distinct registers Synchronization Algorithms and Concurrent Programming Gadi Taubenfeld © 2014 85
Lemma 2. 21 each process in PBy takes the induction hypothesis: one step first; all in their remainders ; . . . only P take steps x W 1 W 2 Wi y 1 y 2 . . . yi only p takes steps yj . . . P are hidden and P are hidden? Yes cover |P| distinct P cover W 1 ? Yes zregisters z 2 p covers w 1? z. Yes 1 i p is hidden? Maybe p covers a unique register w 1 W 1 (Lemma 2. 20) This Chapter 2 Wj w 1 w 2 wi completes the lower bound . . . equals P are hidden? Yes P cover Wj ? Yes proof p covers wj? Yes p is hidden? Yes! Synchronization Algorithms and Concurrent Programming Gadi Taubenfeld © 2014 . . . zj wj 86
Question What breaks in the lower bound proof if we change the definition of “process p is hidden in run x” as follows: • • • p reads 5 from r q reads 5 to r p writes 7 to r q writes 8 to r p reads 8 from r p writes 8 to r r Chapter 2 this write does not change the value of r! 8 Synchronization Algorithms and Concurrent Programming Gadi Taubenfeld © 2014 87
Tight Space Bounds Section 2. 5. 2 An Upper Bound Theorem: There is a deadlock-free mutual exclusion algorithm for n processes that uses n shared bits. Chapter 2 Synchronization Algorithms and Concurrent Programming Gadi Taubenfeld © 2014 88
The One-Bit Algorithm code of process i , i {1 , . . . , n} p 3 writes false true 1 b false 2 false 3 false true pn writes false true 4 false true n false true bits critical section Chapter 2 Synchronization Algorithms and Concurrent Programming Gadi Taubenfeld © 2014 89
The One-Bit Algorithm code of process i , i {1 , . . . , n} repeat b[i] = true; j = 1; while (b[i] = true) and (j < i) do if b[j] = true then b[i] = false; await b[j] =false fi j = j+1 od until b[i] = true for j = i+1 to n do await b[j] = false od critical section b[i] = false b Chapter 2 1 2 3 4 false n false Synchronization Algorithms and Concurrent Programming Gadi Taubenfeld © 2014 bits 90
Question Does the correctness of the One-bit algorithm depend on the order in which the bits are read within the loops ? Chapter 2 Synchronization Algorithms and Concurrent Programming Gadi Taubenfeld © 2014 91
Properties of the One-Bit Algorithm • • • Satisfies mutual exclusion and deadlock-freedom Starvation is possible It is not fast It is not symmetric It uses only n shared bits and hence it is space optimal • Would it work with safe bits? Chapter 2 Synchronization Algorithms and Concurrent Programming Gadi Taubenfeld © 2014 92
Section 2. 5. 3 Question Computing with Infinitely Many Process Does it follow from the lower bound that there is no solution, using atomic registers, when the number of processes is not known in advance or when the number is infinite ? No. It follows that in such a case infinitely many registers are needed. A simple algorithm for infinitely many processes is presented in Section 2. 5. 3. Chapter 2 Synchronization Algorithms and Concurrent Programming Gadi Taubenfeld © 2014 93
Section 2. 5. 3 Computing with Infinitely Many Process: The algorithm 1 2 3 4 5 6 7 8 9 10 owner 0 0 0 1 0 0 … other 0 1 10 10 0 1 0 0 0 … loser 0 0 0 1 0 0 0 0 … winner 0 0 1 0 0 0 0 … process 4 runs alone … process 4 is the winner process 3 runs. . . process 3 lost process 6 runs. . . process 3 lost This is just one game, we need infinitely many … Chapter 2 Synchronization Algorithms and Concurrent Programming Gadi Taubenfeld © 2014 94
Automatic Discovery of Algorithms Section 2. 6 Chapter 2 Synchronization Algorithms and Concurrent Programming Gadi Taubenfeld © 2014 95
Section 2. 6 Automatic Discovery of Algorithms How do we invent algorithms? eureka This is one way See next slide for another way. . . (maybe they are the same. . . ) Chapter 2 Synchronization Algorithms and Concurrent Programming Gadi Taubenfeld © 2014 96
Automatic Discovery Algorithm Generator Chapter 2 algorithms Algorithm Verifier correct algorithms Synchronization Algorithms and Concurrent Programming Gadi Taubenfeld © 2014 97
System Overview Algorithm Generator algorithms verification results Algorithm Verifier correct algorithms User Interface parameters (# of: processes, bits, lines of code) Chapter 2 Synchronization Algorithms and Concurrent Programming Gadi Taubenfeld © 2014 98
Results User-defined parameters Results Shared Entry Exit Complex Starvation Tested bits conditions freedomalgorithms 2 2 3 3 4 4 4 5 6 6 7 4 6 4 5 6 5 5 1 1 1 1 1 Yes Yes Correct appx. alg. running hours 7, 196, 536, 269 0 216 846, 712, 059 66 39 25, 221, 389 105 0. 4 1, 838, 128, 995 10 47 129, 542, 873 480 1 129, 190, 403 56 1 *900, 000 80 12 *22, 000 106 0. 4 *70, 000 96 1 * This run was stopped after few solutions were found. Not all algorithms were teste Chapter 2 Synchronization Algorithms and Concurrent Programming Gadi Taubenfeld © 2014 99
Next Chapter In Chapter 3 we will consider more advanced solutions to the mutual exclusion problem using atomic registers. -- Gadi Chapter 2 Synchronization Algorithms and Concurrent Programming Gadi Taubenfeld © 2014 100
- Slides: 99