Synchronization Algorithms and Concurrent Programming Gadi Taubenfeld Chapter
Synchronization Algorithms and Concurrent Programming Gadi Taubenfeld Chapter 4 Blocking and Non-blocking Synchronization Version: June 2014 Chapter 4 Synchronization Algorithms and Concurrent Programming Gadi Taubenfeld © 2014 1
Synchronization Algorithms and Concurrent Programming ISBN: 0131972596, 1 st edition A note on the use of these power-point slides: I am making these slides freely available to all (faculty, students, readers). They are in Power. Point form so you can add, modify, and delete slides and slide content to suit your needs. They obviously represent a lot of work on my part. In return for use, I only ask the following: q That you mention their source, after all, I would like people to use my book! q That you note that they are adapted from (or perhaps identical to) my slides, and note my copyright of this material. Thanks and enjoy! Gadi Taubenfeld All material copyright 2014 Gadi Taubenfeld, All Rights Reserved To get the most updated version of these slides go to: http: //www. faculty. idc. ac. il/gadi/book. htm Chapter 4 Synchronization Algorithms and Concurrent Programming Gadi Taubenfeld © 2014 2
Chapter 4 Blocking and Non-blocking Synchronization 4. 1 Synchronization Primitives 4. 2 Collision Avoidance using Test-and-set Bits 4. 3 The Ticket Algorithm using RMW Objects 4. 4 Local Spinning using Strong Primitives 4. 5 Concurrent Data Structures -- Progress Conditions -- Consistency Conditions -- Non-blocking Queue -- Memory Barriers These topics are covered in this presentation 4. 6 Semaphores 4. 7 Monitors 4. 8 Fairness of Shared Objects Chapter 4 Synchronization Algorithms and Concurrent Programming Gadi Taubenfeld © 2014 3
Concurrent Data Structures Using locks P 1 P 2 Without locks P 3 data structure Chapter 4 P 1 P 2 P 3 data structure data Synchronization Algorithms and Concurrent Programming Gadi Taubenfeld © 2014 4
Concurrent Data Structures Using locks P 1 P 2 Without locks P 3 data structure P 1 P 2 P 3 data structure data • simple programming model • resilient to failures, etc. • false conflicts • often complex • fault-free solutions only • memory consuming • sequential bottleneck • sometime -- weak progress cond. Chapter 4 Synchronization Algorithms and Concurrent Programming Gadi Taubenfeld © 2014 5
Progress Conditions Section 4. 5. 1 Chapter 4 Synchronization Algorithms and Concurrent Programming Gadi Taubenfeld © 2014 6
Progress Conditions Using locks P 1 P 2 lock-free P 1 P 3 data structure P 2 P 3 data structure data es r lu ai f o n deadlock (livelock) freedom Chapter 4 starvation freedom FIFO (+ df ) obstruction freedom non-blocking Synchronization Algorithms and Concurrent Programming Gadi Taubenfeld © 2014 wait freedom 7
Obstruction-freedom P 1 P 2 P 3 P 4 Done Chapter 4 Synchronization Algorithms and Concurrent Programming Gadi Taubenfeld © 2014 8
Non-blocking P 1 P 2 P 3 P 4 Done Chapter 4 Synchronization Algorithms and Concurrent Programming Gadi Taubenfeld © 2014 9
Wait-freedom P 1 Chapter 4 P 2 P 3 Synchronization Algorithms and Concurrent Programming Gadi Taubenfeld © 2014 P 4 10
Wait-freedom Chapter 4 P 1 P 2 P 3 P 4 Done Synchronization Algorithms and Concurrent Programming Gadi Taubenfeld © 2014 11
Lock-free Data Structures Obstruction-freedom • too weak progress condition • not complex Chapter 4 Non-blocking Wait-freedom • strong enough • strong/desirable • not so complex • complex/less efficient Synchronization Algorithms and Concurrent Programming Gadi Taubenfeld © 2014 12
Consistency Conditions How do we define the correctness of a concurrent object ? Ø Linearizability Ø Sequential Consistency Ø … Chapter 4 Synchronization Algorithms and Concurrent Programming Gadi Taubenfeld © 2014 13
Why are consistency conditions important Ø Formal specification of concurrent objects Ø Help to understand possible behaviors of an implementation of a concurrent object and to reason about correctness Ø Clarifies what optimizations are acceptable Chapter 4 Synchronization Algorithms and Concurrent Programming Gadi Taubenfeld © 2014 14
Example: Concurrent Queue Chapter 4 q. enq( ) q. deq( ) Synchronization Algorithms and Concurrent Programming Gadi Taubenfeld © 2014 15
Sequential Execution Is this execution reasonable? q. enq P 1 P 2 Yes q. deq q. enq q. deq time Chapter 4 Synchronization Algorithms and Concurrent Programming Gadi Taubenfeld © 2014 16
Concurrent Execution Is this execution reasonable? q. enq P 1 P 2 ? ? ? q. deq q. enq q. deq time Chapter 4 Synchronization Algorithms and Concurrent Programming Gadi Taubenfeld © 2014 17
Concurrent Execution Is this execution reasonable? q. enq P 1 P 2 Yes q. deq q. enq q. deq time Chapter 4 Synchronization Algorithms and Concurrent Programming Gadi Taubenfeld © 2014 18
Concurrent Execution Is this execution reasonable? q. enq P 1 P 2 Yes q. deq q. enq q. deq time Chapter 4 Synchronization Algorithms and Concurrent Programming Gadi Taubenfeld © 2014 19
Sequential Specification of an object Queue: In sequential executions, the enqueue operation inserts a value to the queue and the dequeue operation returns and deletes the oldest value in the queue. Sequential specification Chapter 4 Synchronization Algorithms and Concurrent Programming Gadi Taubenfeld © 2014 21
Linearizability for an object Each concurrent execution is “equivalent’’ to an execution in the sequential specification concurrent executions Chapter 4 Each sequential execution is included in the sequential specification Sequential executions All executions Synchronization Algorithms and Concurrent Programming Gadi Taubenfeld © 2014 22
Linearizability for an object Ø Need to preserve real time order Ø Each operation should appear to “take effect” instantaneously at some moment between its invocation and response Ø Object is correct if this “sequential” behavior is correct q. enq P 1 P 2 q. deq q. enq Chapter 4 Synchronization Algorithms and Concurrent Programming Gadi Taubenfeld © 2014 q. deq 23
Linearizability Ø Permits programmers to specify and reason about concurrent objects using known techniques from the sequential domain Ø provides the illusion that each operation applied by concurrent processes takes effect instantaneously between its invocation and its response Chapter 4 Synchronization Algorithms and Concurrent Programming Gadi Taubenfeld © 2014 24
Terminology Invocation Linearization Point Response q. enq Chapter 4 Synchronization Algorithms and Concurrent Programming Gadi Taubenfeld © 2014 25
Linearizability Is this execution linearizable? q. enq P 1 P 2 Yes q. deq q. enq q. deq time Chapter 4 Synchronization Algorithms and Concurrent Programming Gadi Taubenfeld © 2014 26
Linearizability Is this execution linearizable? q. enq P 1 P 2 Yes X 2 q. deq q. enq q. deq time Chapter 4 Synchronization Algorithms and Concurrent Programming Gadi Taubenfeld © 2014 27
Linearizability Is this execution linearizable? P 1 q. enq P 2 No, BUT … q. deq q. enq q. deq time Chapter 4 Synchronization Algorithms and Concurrent Programming Gadi Taubenfeld © 2014 28
Linearizability Is this execution linearizable? P 1 q. enq P 2 No, BUT … q. deq q. enq q. deq time Chapter 4 Synchronization Algorithms and Concurrent Programming Gadi Taubenfeld © 2014 29
Linearizability Sequential Consistency Is this execution linearizable? P 1 q. enq P 2 No, BUT … q. deq q. enq q. deq time Chapter 4 Synchronization Algorithms and Concurrent Programming Gadi Taubenfeld © 2014 30
Sequential Consistency Ø No need to preserve real time order – Cannot re-order operations done by the same thread – Can re-order non-overlapping operations done by different threads Ø Each operation should “take effect” instantaneously between invocation and response events Ø Object is correct if this “sequential” behavior is correct Chapter 4 Synchronization Algorithms and Concurrent Programming Gadi Taubenfeld © 2014 31
Sequential Consistency Is this execution sequential consistent? Yes P 1 q. enq P 2 q. deq q. enq time Chapter 4 Synchronization Algorithms and Concurrent Programming Gadi Taubenfeld © 2014 32
Sequential Consistency Is this execution sequential consistent? Yes Its also linearizable! p. enq P 1 P 2 p. enq p. deq time Chapter 4 Synchronization Algorithms and Concurrent Programming Gadi Taubenfeld © 2014 33
Sequential Consistency Is this execution sequential consistent? p. enq P 1 P 2 p. enq p. deq time Chapter 4 Synchronization Algorithms and Concurrent Programming Gadi Taubenfeld © 2014 34
Sequential Consistency Is this execution sequential consistent? No !!! P 1 q. enq P 2 p. enq q. deq q. enq p. deq time Chapter 4 Synchronization Algorithms and Concurrent Programming Gadi Taubenfeld © 2014 35
Sequential Consistency Is this execution sequential consistent? No !!! P 1 q. enq P 2 p. enq q. deq q. enq p. deq time Chapter 4 Synchronization Algorithms and Concurrent Programming Gadi Taubenfeld © 2014 36
Compositionality Theorem: Sequential consistency is not compositional. Proof: The example from the previous slide. Theorem: Linearizability is compositional. Proof: . . . Chapter 4 Synchronization Algorithms and Concurrent Programming Gadi Taubenfeld © 2014 37
A Non-blocking Concurrent Queue Algorithm Section 4. 5. 2 Chapter 4 Synchronization Algorithms and Concurrent Programming Gadi Taubenfeld © 2014
Implementing a Non-blocking Queue enqueue Chapter 4 dequeue Synchronization Algorithms and Concurrent Programming Gadi Taubenfeld © 2014 39
Implementing a Non-blocking Queue Empty queue tail head Dummy Node Chapter 4 Synchronization Algorithms and Concurrent Programming Gadi Taubenfeld © 2014 40
Enqueue lnode Chapter 4 tail head Synchronization Algorithms and Concurrent Programming Gadi Taubenfeld © 2014 41
2 concurrent enqueue operations What can go wrong when using registers? tail Chapter 4 head Synchronization Algorithms and Concurrent Programming Gadi Taubenfeld © 2014 42
Dequeue tail Chapter 4 head Synchronization Algorithms and Concurrent Programming Gadi Taubenfeld © 2014 43
2 concurrent dequeue operations What can go wrong when using registers? tail Chapter 4 head Synchronization Algorithms and Concurrent Programming Gadi Taubenfeld © 2014 44
Enqueue + Dequeue (Non-empty queue) tail Chapter 4 head Synchronization Algorithms and Concurrent Programming Gadi Taubenfeld © 2014 45
Enqueue + Dequeue (Empty queue) tail Chapter 4 head Synchronization Algorithms and Concurrent Programming Gadi Taubenfeld © 2014 46
Enqueue + Dequeue (Empty queue) tail head Lost tail !!! The dequeuer must advance tail before redirecting head Chapter 4 Synchronization Algorithms and Concurrent Programming Gadi Taubenfeld © 2014 47
Impossibility Result Theorem: There is a wait-free consensus algorithm for 2 processes, using queues and atomic registers. Theorem: There is NO wait-free (or non-blocking) consensus algorithm for 2 processes, using only atomic registers. There is NO wait-free or non-blocking implementation of a queue for two processes or more, from atomic registers. Chapter 4 Synchronization Algorithms and Concurrent Programming Gadi Taubenfeld © 2014 48
Compare & swap (CAS) shared register old new CAS ( A, B, C ) if A=B then A: =C; return(true) else return(false) Supported by Sun, Intel, AMD, … Chapter 4 Synchronization Algorithms and Concurrent Programming Gadi Taubenfeld © 2014 49
CAS: The ABA problem shared register old new local: = A CAS ( A, B, C ) if A=B then A: =C CAS (A, local, 100) local: value tag A: value tag CAS (A, local, <100, local. tag+1> ) In the following, we will ignore the ABA problem and assume that it Synchronization Algorithms and Concurrent Programming Chapter 4 50 is resolved using additional tag fields. Gadi Taubenfeld © 2014
Section 4. 5 A Non-blocking Queue Ø The algorithm is due to M. M. Michael and M. L. Scott (1996). Ø The algorithm is included in the Standard Java Concurrency Package. Ø Every process must be prepared to encounter a half-finished enqueue operation, and help to finish it. Ø Invariant: The tail refers to either the last node or to the node just before the last node. Chapter 4 Synchronization Algorithms and Concurrent Programming Gadi Taubenfeld © 2014 51
lnode Enqueue tail head ltail true false done repeat ltail lnext ltail. next if ltail = tail then if lnext = NULL then if CAS (ltail. next, lnode) then done true fi else CAS (tail, lnext) fi fi until done = true CAS (tail, lnode) Chapter 4 try to link lnode to the end queue enqueue failed: tryof tothe swing tail to a node that inserted enqueue done: trywas to swing tailby to some other process the inserted node. Synchronization Algorithms and Concurrent Programming Gadi Taubenfeld © 2014 52
lnode Enqueue tail head ltail true false done repeat ltail lnext ltail. next if ltail = tail then if lnext = NULL then if CAS (ltail. next, lnext, node) then done true fi else CAS (tail, lnext) fi fi until done = true CAS (tail, lnode) Chapter 4 Question: Would the enqueue operation be correct if the last line is omitted? Answer: Yes, but… Synchronization Algorithms and Concurrent Programming Gadi Taubenfeld © 2014 53
lnode Enqueue tail head ltail true false done repeat ltail lnext ltail. next if ltail = tail then if lnext = NULL then Question: Would the enqueue operation be correct if this line is omitted? if CAS (ltail. next, lnext, node) then done true fi else CAS (tail, lnext) fi fi until done = true CAS (tail, lnode) Chapter 4 No, when it is assumed that cells can be released and then reallocated. Otherwise, yes. Synchronization Algorithms and Concurrent Programming Gadi Taubenfeld © 2014 54
tail Dequeue repeat head lvalue lhead ltail lnext lhead. next lnext true false done ltail lhead if lhead = head then if lhead = ltail then /* empty or tail behind? if lnext = NULL then return(NULL) fi ; /* empty CAS (tail, lnext) /* try to advance tail else lvalue lnext. value /* no need to deal with tail if CAS (head, lnext) then done true fi fi fi until done = true free(lhead) ; return(lvalue) Synchronization Algorithms and Concurrent Programming Chapter 4 Gadi Taubenfeld © 2014 55
tail Dequeue repeat head lvalue lhead ltail lnext lhead. next lnext true false done ltail lhead Question: Would the dequeue operation be correct if this line is omitted? if lhead = head then if lhead = ltail then /* empty or tail behind? if lnext = NULL then return(NULL) fi ; /* empty CAS (tail, lnext) /* try to advance tail else lvalue lnext. value /* no need to deal with tail if CAS (head, lnext) then done true fi fi fi until done = true free(lhead) ; return(lvalue) Synchronization Algorithms and Concurrent Programming Chapter 4 Gadi Taubenfeld © 2014 56
A note about Memory Barriers Chapter 4 Synchronization Algorithms and Concurrent Programming Gadi Taubenfeld © 2014
Example: Using Flags x and y : atomic bits, initially 0 x 0 y 0 Process A Process B write. x(1) write. y(1) read. y read. x Q: Is it possible that both processes read the value 0 ? Chapter 4 Synchronization Algorithms and Concurrent Programming Gadi Taubenfeld © 2014 58
Example: Using Flags x and y : atomic bits, initially 0 x 0 y 0 Process A Process B write. x(1) write. y(1) read. y read. x Fact: Many hardware architectures do not support sequential consistency because they think it is too strong Chapter 4 Synchronization Algorithms and Concurrent Programming Gadi Taubenfeld © 2014 59
Example: Using Flags x and y : atomic bits, initially 0 x 0 y 0 Process A Process B write. x(1) write. y(1) read. y read. x Solution: Memory barriers Chapter 4 Synchronization Algorithms and Concurrent Programming Gadi Taubenfeld © 2014 60
Binary Semaphores Section 4. 6 Chapter 4 Synchronization Algorithms and Concurrent Programming Gadi Taubenfeld © 2014
Section 4. 6 Binary Semaphores Operations down(S) up(S) 1 Types unfair semaphore weak semaphore strong semaphore Chapter 4 down(S) m if S > 0, then S = 0 otherwise, the process is blocked until the value becomes greater than 0. m Testing and decrementing the semaphore are executed atomically without interruption. up(S) m S=1 Synchronization Algorithms and Concurrent Programming Gadi Taubenfeld © 2014 62
Semaphores Deadlock-free Mutual exclusion Algorithm down(S) critical section up(S) Chapter 4 1 Synchronization Algorithms and Concurrent Programming Gadi Taubenfeld © 2014 63
Semaphores Constant Space Starvation-free Algorithm Weak semaphores queue S. 0 1 10 S. 1 1 Shared bits queue 0 S. 0 10 1 1 S. 1 0 empty false local bits doorkeeper critical section Chapter 4 empty myqueue true false otherqueue Synchronization Algorithms and Concurrent Programming Gadi Taubenfeld © 2014 64
Semaphores Constant Space Starvation-free Algorithm Weak semaphores queue S. 0 1 1 Shared bits queue 0 S. 0 0 1 S. 1 0 empty false local bits critical section Chapter 4 empty myqueue true false otherqueue Synchronization Algorithms and Concurrent Programming Gadi Taubenfeld © 2014 65
Semaphores Constant Space Starvation-free Algorithm Weak semaphores queue S. 0 1 1 Shared bits queue 0 S. 0 1 S. 1 0 empty false local bits critical section Chapter 4 empty myqueue true false otherqueue Synchronization Algorithms and Concurrent Programming Gadi Taubenfeld © 2014 66
Semaphores Constant Space Starvation-free Algorithm Weak semaphores queue S. 0 1 1 Shared bits queue 0 S. 0 10 1 S. 1 0 empty false local bits critical section Chapter 4 empty myqueue true false otherqueue Synchronization Algorithms and Concurrent Programming Gadi Taubenfeld © 2014 67
Semaphores Constant Space Starvation-free Algorithm Weak semaphores queue S. 0 1 1 Shared bits queue 0 S. 0 1 S. 1 0 empty false local bits critical section Chapter 4 empty myqueue true false otherqueue Synchronization Algorithms and Concurrent Programming Gadi Taubenfeld © 2014 68
Semaphores Constant Space Starvation-free Algorithm Weak semaphores queue S. 0 1 1 Shared bits queue 0 S. 0 10 1 S. 1 0 empty false local bits critical section Chapter 4 empty myqueue true false otherqueue Synchronization Algorithms and Concurrent Programming Gadi Taubenfeld © 2014 69
Semaphores Constant Space Starvation-free Algorithm Weak semaphores queue S. 0 1 1 Shared bits queue 0 S. 0 0 1 S. 1 10 empty false local bits critical section Chapter 4 empty myqueue true otherqueue Synchronization Algorithms and Concurrent Programming Gadi Taubenfeld © 2014 70
Semaphores Constant Space Starvation-free Algorithm myeuque = queue down (S. myqueue) if queue = myqueue then otherqueue = 1 – myqueue down(S. otherqueue) queue = otherqueue repeat empty = true up(S. myqueue) down(S. myqueue) until empty critical section up(S. otherqueue) else empty = false critical section fi up(S. myqueue) Chapter 4 Weak semaphores S. 0 1 S. 1 1 Shared bits queue 0 empty false local bits myqueue otherqueue Synchronization Algorithms and Concurrent Programming Gadi Taubenfeld © 2014 71
Semaphores Constant Space Starvation-free Algorithm Question: Is it possible Yes that some process will be in its CS, while no process is a doorkeeper? queue 10 S. 0 10 1 S. 1 1 empty critical section Chapter 4 true false Synchronization Algorithms and Concurrent Programming Gadi Taubenfeld © 2014 72
Properties of the Algorithm q Satisfies mutex & deadlock-freedom. q Satisfies 2 -bounded-waiting: A process can enter its CS ahead of anoter only twice. q There is no need to know the # of processes q System response time is a constant q Two weak semaphores and two atomic bits are used. Chapter 4 Synchronization Algorithms and Concurrent Programming Gadi Taubenfeld © 2014 73
The End Chapter 4 Synchronization Algorithms and Concurrent Programming Gadi Taubenfeld © 2014 74
- Slides: 73