Lecture 9 Synchronization 1 Java Synchronization concurrent objects

Lecture 9 Synchronization 1

Java Synchronization - concurrent objects safety • The problem: • Threads may access the same object concurrently. • @invariant/correctness is not guaranteed. • Solutions (achieve safety) • Design concurrent execution using copies of (immutable) objects. • Synchronize access to internal state of the object. 2

class Even 3

Visibility and Reordering – unsafe constructs in concurrent execution • Visibility - when threads see values written to memory by other threads. • Reordering - compiler / RTE change code to optimize it. 4

Visability example 5

• thread t may not "see" the changes to the state of d performed in main thread • main calls d. set(10): 10 is written to i_ member of Visibility. Demo object • t reads from i_: (d. get() ) • At the same time d. set(20) is performed (race condition). • Java does not guarantee t sees any recent value written to this memory location. 6

Re-ordering • compiler must be able to re-order simple instructions to optimize our code • compiler guarantees safe reordering in non-concurrent execution environments. 7

“must-happen-before” • compiler would not reorder if "must-happenbefore" constraint 8

Monitors: synchronized construct in Java • Any object has a monitor, used as a lock. • Allows only one thread at a time to enter the object. • synchronized keyword on a non-null object: 9

The house analogy • Object = house. • Object’s Monitor = key. • People = threads. • Only one thread can have the key and enter the house. 10

synchronized construct in Java • Similarly to the previous case, we may write: • This is just “syntactic sugar”. 11

synchronized • Java ensures that only one thread access this Object. • public synchronized int add() - only one thread is allowed to enter all the Object’s methods. • solves visibility and reordering problems; • "write" to memory during synchronized code section is guaranteed to be returned to all "read" operations following it. 12

How "synchronized" is implemented? • Each object inside JVM (at runtime) has a lock ("monitor") • JVM maintain/initialize locks. • We cannot access locks explicitly • We can access implicitly using synchronized 13

Monitors (locks) • locks are in one of two states; 1. in possession of thread 2. available • thread T tries to acquire the lock of any object. • if available – lock transferred to T’s possession • otherwise, T sleeps (in a queue) until lock available • T wakes up and tries to acquire lock 14

synchronized • each thread, calling the method, must first acquire lock. • when thread exits – thread releases lock 15

The cost of locks – time, concurrency • Memory sync. : after thread exits, it synchronizes with the main memory • thread may cache copies of memory in its “own” memory (e. g. , CPU registers / CPU cache). • Blocking: threads must wait for each other 16

When to Use Synchronization • When we want atomic (all-at-once) actions. • When we want to solve the re-ordering and visibility problems. • monitor - internal state of object is encapsulated. • objects are responsible to synchronize access to state: synchronized get()/set(). • Try to avoid and reduce Synchronization!!! • Our threads are waiting for synchronization. 17

Synchronized Properties • synchronized keyword is not part of the method signature. • synchronized keyword cannot be specified in an interface. • constructors cannot be synchronized (syntax error) • no point - as long as constructor is not complete, object instance is not accessible to other threads, unless… • static method as synchronized - lock associated with the class to be used • (each class in Java is also an object on its own) • Java locks are Reentrant: same thread holding the lock can get it again and again. 18

Partially synchronized Objects (example) 19

Design patterns 20

Is the do. Something function "thread safe"? 21

Is the do. Something function "thread safe"? • No. After calling the l. contains other thread may add 42 to our list, and then do. Something will add another one. . 22

• Is this solution correct? 23

• No. The lock protecting do. Something() is different from the lock protecting the add(). 24

Option: add a add. If. Absent()method. • Extending Linked. List class and add the function we want? • do not know how the class Linked. List protects itself from concurrent access. 25

Composition / Containment • wrapper around Linked. List class: • Enforces right synchronization policy • Delegates all calls (after synchronization is verified) to underlying implementation class. 26

Composition / Containment • We know exactly which lock is used to synchronize access to our list. • best applied in situations where: • We are not sure/ do not know the internal implementation of an object. • Implementation of object may change without our knowledge. 27

Client Side Locking • If we know the identity of the object which is used for synchronization • If Linked. List class uses its own monitor (lock) to achieve synchronization, then: 28

Client side locking – not so good approach • only when we know internal implementation • not when implementation may change • not Object Oriented: • responsibility of synchronization to clients instead of object • cannot ensure consistency among ALL clients : • If client does not synchronize correctly, then ALL the code will behave incorrectly. • better to enforce safety on provider side 29

Version Iterators 30

what will happen if while executing print. List() another thread will change l? • Optional solution: create a temporary copy of the Linked. List, and then iterate over this copy. • It may be too expensive. • We still need to iterate the Linked. List in order to copy it. • synchronize the loop on Linked. List? • body of loop may be arbitrarily long - locking the entire list for the entire loop. 32

Solution - fail-fast iterators • A better solution is a fail-fast iterators - which are iterators that: • Detect that a change to the data structure happened while they are in use. • Throw an exception in order to indicate that they are inappropriate for usage any more. • useful for many readers few writers model. • Version iterators – implementation of fail fast iterators. 33

34

Balking • Popular in serial codes. • Methods “fail” if precondition does not hold. 35

Guarded suspension: Wait! • Don’t throw exception. • Precondition may hold in the future - other threads may change it. • wait until precondition holds. • Another thread will make required state changes… 36

Guarded Suspension (wait) constructs: wait() and notify(), notify. All() 37

wait() • Threads wait() until some other thread wakes them up • each object has a wait set - similar to locks • maintained internally by the JVM • set holds threads blocked by wait() on object • until notifications are invoked or waits released 38

notify() • threads enter wait queue of an object o by invoking the wait() of o. • thread wakeups threads from queue of o by: notify()/notify. All() 39

Attic: for wait()ing threads 40

wait/notify cycle • wait() - wait invocation • current thread calling wait() is blocked. • JVM places thread in wait set(queue) of o. • notify() - notify invocation • arbitrary thread T is removed from o’s wait set • T is resumed from the point of wait(). • notify. All() • like notify except for all threads in o’s wait set 41

Summary: Policies for failed @pre/@inv • Balking. throw exception if precondition fails. • Guarded suspension. suspend method invocation (and thread) until precondition becomes true. • Time-outs. Something in between balking and suspension. bounded wait for precondition to become true. Implemented using Object’s wait(long timeout) 42

Common pitfalls - guard atomically • Can we put “if” instead of “while”? 43

Common pitfalls - guard atomically 44

Common pitfalls - guard atomically • Can we put “if” instead of “while”? No! • If two threads are waiting, and only one add() is invoked: • Two threads are notified. • Only one of the notified threads can remove item. • The other one needs to keep waiting. 45

Common pitfalls - wait atomically 46

Common pitfalls - wait atomically • Scenario: • exactly one producer and one consumer. • Consumer calls remove. First() but stopped after the condition by the scheduler. • Producer calls add() successfully. • Producer: Notify. All() but no one in queue! • Consumer: checks the condition and wait() forever. • Sometimes threads are woken up without notify(). 47

Rules of Thumb for Wait and Notify • Guarded wait - Blocking a thread to wait() for a condition. • Guard atomicity - Condition checks: always in while loops. • Multiple guard atomicity If there are several conditions which need to be waited for, place them all in the same loop. • Don't forget to wake() up - To ensure liveness, classes must wake up waiting threads. • notify() Vs. notify. All(): • When multiple threads wait for multiple conditions on the same object, we must use notify. All() to ensure that no thread misses an event. 48

Disadvantages of synchronized • Only one thread can enter the synchronized code. • Maybe we can allow several. • We cannot release the lock in the middle of the function. • No fairness: threads may wait a long time when others get more chances to run. 49

Semaphores • Object controls bounded number of permits (permit = train ticket): • Threads ask semaphore for permit (a “ticket”). • If semaphore has permits available, one permit is assigned to requesting thread. • If no permit available, requesting thread is blocked until permit is available (when a thread returns back a permit). 50

Usage: 51

Example implementation 52

Java Semaphore properties • not re-entrant – thread calls acquire() twice, must release() twice! • Also, a thread who acquired, but semaphore ran out, will still block if acquire() again. • semaphore can be released by a thread other than owner (unlike lock) – no ownership. • Right managing of locks is on the user. 53

Java Semaphore properties • Can include services as try. Acquire() and managing permits: 54

Readers/Writers Example • We wish to allow readers and writers to access a shared resource under different policies. • Common policy: • several readers can access resource together. • only one writer can access resource at given time. • Example: • Readers wait if writer is pending. • One writer at a time can access the shared resource, if no readers are reading from it. 55

java. util. concurrent. locks. Read. Write. Lock interface Reentrant. Read. Write. Lock implementation. 56

57

58

RW implementation: properties • Several readers can access the resource together. • Only one writer can access it at any given time. • Only if no readers are • Readers will wait if any writer is pending. Policy is “pro-writers”. Prevents “starvation” of writers. • But, Reads in principle can be “starved”. 59

Atomic Instructions • Atomic: happens “all-at-once”. • Most CPU operations (like add, mov etc. ) are atomic. • CPUs today offer a set of atomic instructions for multithreading. • The “Compare. And. Set” (cas) example: 60

Atomic Instructions • Atomic: the scheduler cannot stop a thread in the middle of this operation - only before or after it. • The compare. And. Set instruction is extremely useful and powerful. • Java has several classes (all beginning with Atomic* that allow us to use compare. And. Set). 61

Even using synchronized 62

Even counter class using Atomic. Integer • Atomic. Integer: holds int so that we can call compare. And. Set(expect, value) on. • There is no synchronized anywhere in class. 63

Even counter class using Atomic. Integer • If there are n threads t_1, . . . , t_n that attempt to invoke add() one time all at once then, without the loss of generality: • t_1 will enter the while loop once, • t_2 at most twice, • … • and t_n at most n times. 64

Guarded Suspension (wait) constructs: wait() and notify(), notify. All() 65

Linked. List using cas: 66

Atomic Instructions: Advantages • Threads are given a choice • When their requested action cannot be performed - they are given a choice • on what to do, code wise • This is instead of just being blocked in blocking algorithms • No thread suspension • Code runs significantly faster than the synchronized counterpart • Threads are never blocked – thus no system calls used! Called “lock-free”. • Reduced Thread Latency

Limitations: • Sometimes we want to block a thread, but cannot. • Lock-Free data structures are much harder to write. • Some complex data structures may need to copy large amounts of data for lock-free implementation. 68

Generic lock-free implementation? public class My. Class { private Atomic. Boolean lock = new Atomic. Boolean(false); public void foo(. . . ) { while !lock. compare. And. Set(false, true); . . lock. set(false); } } Disadvantage: • Not really “lock-free”. • A thread acquiring the lock may sleep during the execution, effectively blocking other threads on “busy-wait”. Advantage: • Completely generic. Suitable ONLY when the code is very short but cannot be checked in a single compare. And. Set(). 69

Generic lock-free implementation? 70

Generic lock-free implementation? • Using synchronized, threads go to a blocked state, which may be wasteful if the code is short and simple. • We now distiguish between two aspects of lock-free sync: • In lock-free implementation, we aim to prevent frequent changes in the state, and keep the Threads running. • In the original lock-free examples, the Threads are not blocked and run the code in parallel, except for the final assignment. This improves the performance, since the final assignment is not limited to a specific thread - only to the first one to complete the code. • The last "generic" example fulfills the first aspect, but not the second one. • It is suitable only when the "right" lock-free synchronization is impossible (i. e. the state change depends on more than one assignment), and the code is short. • If the code is long, it is better to use the synchronized mechanism instead of the generic lock-free implementation. 71

Volatile variables 72

Output: • With the volatile keyword the output is : Incrementing MY_INT to 1 Got Change for MY_INT : 1 Incrementing MY_INT to 2 Got Change for MY_INT : 2 Incrementing MY_INT to 3 Got Change for MY_INT : 3 Incrementing MY_INT to 4 Got Change for MY_INT : 4 Incrementing MY_INT to 5 Got Change for MY_INT : 5 73

Output: • Without the volatile keyword the output is (can be different): Incrementing MY_INT to 1 Incrementing MY_INT to 2 Incrementing MY_INT to 3 Incrementing MY_INT to 4 Incrementing MY_INT to 5 74