Lecture 6 2 Concurrent Queues and Stacks Companion

Lecture 6 -2 : Concurrent Queues and Stacks Companion slides for The Art of Multiprocessor Programming by Maurice Herlihy & Nir Shavit

pool • Data Structure similar to Set – Does not necessarily provide contains() method – Allows the same item to appear more than once – get() and set() public interface Pool<T> { void put(T item); T get(); } Art of Multiprocessor Programming© Herlihy-Shavit 2007 2

Queues & Stacks • Both: pool of items • Queue – enq() & deq() – First-in-first-out (FIFO) order • Stack – push() & pop() – Last-in-first-out (LIFO) order Art of Multiprocessor Programming© Herlihy-Shavit 2007 3

Bounded vs Unbounded • Bounded – Fixed capacity – Good when resources an issue • Unbounded – Holds any number of objects Art of Multiprocessor Programming© Herlihy-Shavit 2007 4

Blocking vs Non-Blocking • Problem cases: – Removing from empty pool – Adding to full (bounded) pool • Blocking – Caller waits until state changes • Non-Blocking – Method throws exception Art of Multiprocessor Programming© Herlihy-Shavit 2007 5

Queue: Concurrency enq(x) tail head y=deq() enq() and deq() work at different ends of the object Art of Multiprocessor Programming© Herlihy-Shavit 2007 6

Concurrency he ad l ai enq(x) y=deq() t Challenge: what if the queue is empty or full? Art of Multiprocessor Programming© Herlihy-Shavit 2007 7

A Bounded Lock-Based Queue public class Bounded. Queue<T> { Reentrant. Lock enq. Lock, deq. Lock; Condition not. Empty. Condition, not. Full. Condition; Atomic. Integer size; Node head; Node tail; int capacity; public Bounded. Queue(int capacity) { this. capacity = capacity; this. head = new Node(null); this. tail = head; this. size = new Atomic. Integer(0); this. enq. Lock = new Reentrant. Lock(); this. not. Full. Condition = enq. Lock. new. Condition(); this. deq. Lock = new Reentrant. Lock(); this. not. Empty. Condition = deq. Lock. new. Condition(); } protected class Node { public T value; public Node next; public Node(T x) { value = x; next = null; } } Art of Multiprocessor Programming© Herlihy-Shavit 2007 8

public void enq(T x) { boolean must. Wake. Dequeuers = false; enq. Lock. lock(); try { while (size. get() == capacity) not. Full. Condition. await(); Node e = new Node(x); tail. next = e; tail = e; if (size. get. And. Increment() == 0) { must. Wake. Dequeuers = true; } } finally { enq. Lock. unlock(); } if (must. Wake. Dequeuers) { deq. Lock. lock(); try { not. Empty. Condition. signal. All(); } finally { deq. Lock. unlock(); } } } public T deq() { T result; boolean must. Wake. Enqueuers = false; deq. Lock. lock(); try { while (size. get() == 0) { not. Empty. Condition. await(); result = head. next. value; head = head. next; if (size. get. And. Decrement() == capacity) { must. Wake. Enqueuers = true; } } finally { deq. Lock. unlock(); } if (must. Wake. Enqueuers) { enq. Lock. lock(); try { not. Full. Condition. signal. All(); } finally { enq. Lock. unlock(); } } return result; } 9

lock • enq. Lock/deq. Lock – At most one enqueuer/dequeuer at a time can manipulate the queue’s fields • Two locks – Enqueuer does not lock out dequeuer – vice versa • Association – enq. Lock associated with not. Full. Condition – deq. Lock associated with not. Empty. Condition Art of Multiprocessor Programming© Herlihy-Shavit 2007 10

enqueue 1. 2. 3. 4. 5. 6. 7. 8. Acquires enq. Lock Reads the size field If full, enqueuer must wait until dequeuer makes room enqueuer waits on not. Full. Condition field, releasing enq. Lock temporarily, and blocking until that condition is signaled. Each time thread awakens, it checks whethere is a room, and if not, goes back to sleep Insert new item into tail Release enq. Lock If queue was empty, notify/signal waiting dequeuers Art of Multiprocessor Programming© Herlihy-Shavit 2007 11

dequeue 1. 2. 3. 4. 5. 6. 7. 8. 9. Acquires deq. Lock Reads the size field If empty, dequeuer must wait until item is enqueued dequeuer waits on not. Empty. Condition field, releasing deq. Lock temporarily, and blocking until that condition is signaled. Each time thread awakens, it checks whether item was enqueued, and if not, goes back to sleep Assigne the value of head’s next node to “result” and reset head to head’s next node Release deq. Lock If queue was full, notify/signal waiting enqueuers Return “result” Art of Multiprocessor Programming© Herlihy-Shavit 2007 12

Bounded Queue head tail Sentinel Art of Multiprocessor Programming 13

Bounded Queue head tail First actual item Art of Multiprocessor Programming 14

Bounded Queue head tail deq. Lock out other deq() calls Art of Multiprocessor Programming 15

Bounded Queue head tail deq. Lock enq. Lock out other enq() calls Art of Multiprocessor Programming 16

Not Done Yet head tail deq. Lock enq. Lock Need to tell whether queue is full or empty Art of Multiprocessor Programming 17

Not Done Yet head tail deq. Lock enq. Lock size 1 Max size is 8 items Art of Multiprocessor Programming 18

Not Done Yet head tail deq. Lock enq. Lock size 1 Incremented by enq() Decremented by deq() Art of Multiprocessor Programming 19

Enqueuer head tail deq. Lock enq. Lock size 1 Art of Multiprocessor Programming 20

Enqueuer head tail deq. Lock enq. Lock Read size 1 OK Art of Multiprocessor Programming 21

Enqueuer head tail deq. Lock No need to lock tail enq. Lock size 1 Art of Multiprocessor Programming 22

Enqueuer head tail deq. Lock enq. Lock Enqueue Node size 1 Art of Multiprocessor Programming 23

Enqueuer head tail deq. Lock enq. Lock size 2 1 get. Andincrement() Art of Multiprocessor Programming 24

Enqueuer head tail deq. Lock enq. Lock size Release lock 8 2 Art of Multiprocessor Programming 25

Enqueuer head tail deq. Lock enq. Lock size 2 If queue was empty, notify waiting dequeuers Art of Multiprocessor Programming 26

Unsuccesful Enqueuer … head tail deq. Lock enq. Lock Read size 8 Uhoh Art of Multiprocessor Programming 27

Dequeuer head tail deq. Lock enq. Lock size Lock deq. Lock 2 Art of Multiprocessor Programming 28

Dequeuer head tail deq. Lock enq. Lock size Read sentinel’s next field 2 OK Art of Multiprocessor Programming 29

Dequeuer head tail deq. Lock enq. Lock Read value size 2 Art of Multiprocessor Programming 30

Make first Node new sentinel Dequeuer head tail deq. Lock enq. Lock size 2 Art of Multiprocessor Programming 31

Dequeuer head tail deq. Lock enq. Lock size Decrement size 1 Art of Multiprocessor Programming 32

Dequeuer head tail deq. Lock enq. Lock size 1 Release deq. Lock Art of Multiprocessor Programming 33

Monitor Locks • Java Reentrant. Locks are monitors • Allow blocking on a condition rather than spinning • Threads: – acquire and release lock – wait on a condition Art of Multiprocessor Programming© Herlihy-Shavit 2007 34

The Java Lock Interface public interface Lock { void lock(); void lock. Interruptibly() throws Interrupted. Exception; boolean try. Lock(); boolean try. Lock(long time, Time. Unit unit); Condition new. Condition(); void unlock; } Create condition to wait on Art of Multiprocessor Programming© Herlihy-Shavit 2007 35

Lock Conditions public interface Condition { void await(); boolean await(long time, Time. Unit unit); … void signal(); void signal. All(); } Art of Multiprocessor Programming© Herlihy-Shavit 2007 36

Lock Conditions public interface Condition { void await(); boolean await(long time, Time. Unit unit); … void signal(); void signal. All(); Release lock and } wait on condition Art of Multiprocessor Programming© Herlihy-Shavit 2007 37

Lock Conditions public interface Condition { void await(); boolean await(long time, Time. Unit unit); … void signal(); void signal. All(); } Wake up one waiting thread Art of Multiprocessor Programming© Herlihy-Shavit 2007 38

Lock Conditions public interface Condition { void await(); boolean await(long time, Time. Unit unit); … void signal(); void signal. All(); } Wake up all waiting threads Art of Multiprocessor Programming© Herlihy-Shavit 2007 39

Await q. await() • • Releases lock associated with q Sleeps (gives up processor) Awakens (resumes running) Reacquires lock & returns Art of Multiprocessor Programming© Herlihy-Shavit 2007 40

Signal q. signal(); • Awakens one waiting thread – Which will reacquire lock Art of Multiprocessor Programming© Herlihy-Shavit 2007 41

Signal All q. signal. All(); • Awakens all waiting threads – Which will each reacquire lock Art of Multiprocessor Programming© Herlihy-Shavit 2007 42

Unbounded Lock-Free Queue (Nonblocking) • Unbounded – No need to count the number of items • Lock-free – Use Atomic. Reference<V> • An object reference that may be updated atomically. – boolean compare. And. Set(V expect, V update) • Atomically sets the value to the given updated value if the current value == the expected value. • Nonblocking – No need to provide conditions on which to wait 43

A Lock-Free Queue head tail Sentinel Art of Multiprocessor Programming 44

Enqueue head tail Enq( Art of Multiprocessor Programming 45 )

Enqueue head tail Art of Multiprocessor Programming 46

Logical Enqueue CAS head tail Art of Multiprocessor Programming 47

Physical Enqueue head tail CAS Art of Multiprocessor Programming 48

Enqueue • These two steps are not atomic • The tail field refers to either – Actual last Node (good) – Penultimate Node (not so good) • Be prepared! Art of Multiprocessor Programming 49

Enqueue • What do you do if you find – A trailing tail? • Stop and help fix it – If tail node has non-null next field – CAS the queue’s tail field to tail. next • As in the universal construction Art of Multiprocessor Programming 50

When CASs Fail • During logical enqueue – Abandon hope, restart – Still lock-free (why? ) • During physical enqueue – Ignore it (why? ) Art of Multiprocessor Programming 51

Dequeuer head tail Read value Art of Multiprocessor Programming 52

Make first Node new sentinel Dequeuer CAS head tail Art of Multiprocessor Programming 53

Unbounded Lock-Free Queue (Nonblocking) public class Lock. Free. Queue<T> { private Atomic. Reference<Node> head; private Atomic. Reference<Node> tail; public Lock. Free. Queue() { this. head = new Atomic. Reference<Node>(null); this. tail = new Atomic. Reference<Node>(null); } public class Node { public T value; public Atomic. Reference<Node> next; public Node(T value) { this. value = value; this. next = new Atomic. Reference<Node>(null); } } Art of Multiprocessor Programming© Herlihy-Shavit 2007 54

Unbounded Lock-Free Queue (Nonblocking) public void enq(T item) { Node node = new Node(item); while (true) { Node last = tail. get(); Node next = last. next. get(); if (last == tail. get()) { if (next == null) { if (last. next. compare. And. Set(next, node)) { tail. compare. And. Set(last, node); return; } } else { tail. compare. And. Set(last, next); } } public T deq() throws Empty. Exception { while (true) { Node first = head. get(); Node last = tail. get(); Node next = first. next. get(); if (first == head. get()) { if (first == last) { if (next = null) { throw new Empty. Exception(); } tail. compare. And. Set(last, next); } else { T value = next. value; if (head. compare. And. Set(first, next)) return value; } } 55

Concurrent Stack • Methods – push(x) – pop() • Last-in, First-out (LIFO) order • Lock-Free! Art of Multiprocessor Programming 56

Empty Stack Top Art of Multiprocessor Programming 57

Push Top Art of Multiprocessor Programming 58

Push Top CAS Art of Multiprocessor Programming 59

Push Top Art of Multiprocessor Programming 60

Push Top Art of Multiprocessor Programming 61

Push Top Art of Multiprocessor Programming 62

Push Top CAS Art of Multiprocessor Programming 63

Push Top Art of Multiprocessor Programming 64

Pop Top Art of Multiprocessor Programming 65

Pop Top CAS Art of Multiprocessor Programming 66

Pop Top CAS mine ! Art of Multiprocessor Programming 67

Pop Top CAS Art of Multiprocessor Programming 68

Pop Top Art of Multiprocessor Programming 69

Lock-free Stack public class Lock. Free. Stack { private Atomic. Reference top = new Atomic. Reference(null); public boolean try. Push(Node node){ Node old. Top = top. get(); node. next = old. Top; return(top. compare. And. Set(old. Top, node)) } public void push(T value) { Node node = new Node(value); while (true) { if (try. Push(node)) { return; } else backoff(); }} Art of Multiprocessor Programming 70

Lock-free Stack public class Lock. Free. Stack { private Atomic. Reference top = new Atomic. Reference(null); public Boolean try. Push(Node node){ Node old. Top = top. get(); node. next = old. Top; return(top. compare. And. Set(old. Top, node)) } public void push(T value) { Node node = new Node(value); while (true) { if (try. Push(node)) { return; } else attempts backoff() try. Push to push a node }} Art of Multiprocessor Programming 71

Lock-free Stack public class Lock. Free. Stack { private Atomic. Reference top = new Atomic. Reference(null); public boolean try. Push(Node node){ Node old. Top = top. get(); node. next = old. Top; return(top. compare. And. Set(old. Top, node)) } public void push(T value) { Node node = new Node(value); while (true) { if (try. Push(node)) { return; Read top value } else backoff() }} Art of Multiprocessor Programming 72

Lock-free Stack public class Lock. Free. Stack { private Atomic. Reference top = new Atomic. Reference(null); public boolean try. Push(Node node){ Node old. Top = top. get(); node. next = old. Top; return(top. compare. And. Set(old. Top, node)) } public void push(T value) { Node node = new Node(value); while (true) { if (try. Push(node)) { return; } else current topbackoff() will be new node’s successor }} Art of Multiprocessor Programming 73

ry Lock-free Stack public class Lock. Free. Stack { private Atomic. Reference top = new Atomic. Reference(null); public boolean try. Push(Node node){ Node old. Top = top. get(); node. next = old. Top; return(top. compare. And. Set(old. Top, node)) } public void push(T value) { Node node = new Node(value); while (true) { if (try. Push(node)) { return; } elsetop, backoff() to swing return success or failure }} Art of Multiprocessor Programming 74

Lock-free Stack public class Lock. Free. Stack { private Atomic. Reference top = new Atomic. Reference(null); public boolean try. Push(Node node){ Node old. Top = top. get(); node. next = old. Top; return(top. compare. And. Set(old. Top, node)) } public void push(T value) { Node node = new Node(value); while (true) { if (try. Push(node)) { return; } else backoff() Push calls try. Push }} Art of Multiprocessor Programming 75

Lock-free Stack public class Lock. Free. Stack { private Atomic. Reference top = new Atomic. Reference(null); public boolean try. Push(Node node){ Node old. Top = top. get(); node. next = old. Top; return(top. compare. And. Set(old. Top, node)) } public void push(T value) { Node node = new Node(value); while (true) { if (try. Push(node)) { return; } else backoff() Create new node }} Art of Multiprocessor Programming 76

Lock-free Stack public class Lock. Free. Stack { private Atomic. Reference top = new Atomic. Reference(null); public boolean try. Push(Node node){ Node old. Top top. get(); If= try. Push() fails, node. next = old. Top; back off before node)) return(top. compare. And. Set(old. Top, } retrying public void push(T value) { Node node = new Node(value); while (true) { if (try. Push(node)) { return; } else backoff() }} Art of Multiprocessor Programming 77

Unbounded Lock-Free Stack protected boolean try. Push(Node node) { Node old. Top = top. get(); node. next = old. Top; return (top. compare. And. Set(old. Top , node)); } public void push( T value ) { Node node = new Node( value ); while (true) { if (try. Push(node)) { return; } else { backoff( ); } } } protected Node try. Pop( ) throws Empty. Exception { Node old. Top = top. get(); if ( old. Top == null ) { throw new Empty. Exception( ); } Node new. Top = old. Top. next; if ( top. compare. And. Set( old. Top, new. Top ) ) { return old. Top; } else { return null; } } public T pop() throws Empty. Exception { while (true) { Node return. Node = try. Pop( ); if ( return. Node != null ) { return. Node. value; } else { backoff( ); } } } 78

Lock-free Stack • Good – No locking • Bad – Without GC, fear ABA – Without backoff, huge contention at top – In any case, no parallelism Art of Multiprocessor Programming 79

Question • Are stacks inherently sequential? • Reasons why – Every pop() call fights for top item • Reasons why not – Think about it! Art of Multiprocessor Programming 80