Carnegie Mellon Synchronization Carnegie Mellon Single and Multithreaded

Carnegie Mellon Synchronization

Carnegie Mellon Single and Multithreaded Processes

Carnegie Mellon Single and Multithreaded Processes ● Single- to multi-threaded conversion ● In a simple world ● Identify functions as parallel activities. ● Run them as separate threads. ● In real world ● Single-threaded programs use global variables, library functions (malloc). ● Be careful with them. ● Global variables are good for easy-communication but need special care.

Carnegie Mellon Single and Multithreaded Processes ● Single-threaded process example:

Carnegie Mellon Single and Multithreaded Processes ● Multi-threaded process example:

Carnegie Mellon Synchronization ● ● Multi-threaded processes require synchronization. Synchronize threads (coordinate their activities) so that when you access shared data (e. g. , global variables) you are not having a trouble.

Carnegie Mellon Synchronization ● ● The part of the process that is accessing and changing shared data is called its critical section. Synchronization is all about critical section handling.

Carnegie Mellon Synchronization ● The part of the process that is accessing and changing shared data is called its critical section. Thread 1 Code Thread 2 Code Thread 3 Code Change X Change Y Change X Assuming X and Y are shared data.

Carnegie Mellon Synchronization ● ● How do we achieve synchronization? Answer: Ensure that no two threads are in their critical section at the same time, aka Mutual Exclusion (mutex).

Carnegie Mellon Synchronization ● ● ● How do we achieve synchronization? Answer: Ensure that no two threads are in their critical section at the same time, aka Mutual Exclusion (mutex). Must assume threads interleave executions arbitrarily (preemptive scheduling) and at different rates. ● Scheduling is not under application’s control.

Carnegie Mellon Synchronization ● ● How do we achieve synchronization? Answer: Ensure that no two threads are in their critical section at the same time, aka Mutual Exclusion (mutex). Must assume threads interleave executions arbitrarily (preemptive scheduling) and at different rates. ● Scheduling is not under application’s control. We control coordination using data synchronization. ● We restrict interleaving of executions to ensure consistency. ● Low-level mechanism to do this: locks, ● High-level mechanisms: mutexes, semaphores, monitors, condition variables.

Carnegie Mellon Synchronization ● General way to achieve synchronization:

Carnegie Mellon Race Condition ● An example: race condition. : critical section Critical section respected : ) Critical section not respected : (

Carnegie Mellon Race Condition ● ● ● Another example: Assume we had 5 items in the buffer. Then ● Assume producer just produced a new item, put it into buffer, and about to do count++ ● Assume consumer just retrieved an item from the buffer, and about to do count-- Producer Consumer or Producer Consumer

Carnegie Mellon Race Condition ● Another example: ● Critical region is where we manipulate count. ● count++ could be implemented as (similarly, count--) ● register 1 = count; //read value register 1 += 1; //increase value count = register 1; //write back

Carnegie Mellon Race Condition ● Another example: register 1 6 5 register 2 4 5 count 5 4 6 PRODUCER (count++) register 1 = count register 1 = register 1 + 1 count = register 1 CONSUMER (count--) CPU : critical section register 2 = count register 2 = register 2 – 1 count = register 2 Main Memory

Carnegie Mellon Race Condition ● ● Yet another example: Two threads executing their critical section code. Balance = 1000 TL balance = get_balance(account); Execution sequence as seen by CPU balance -= amount; Local = 900 TL balance = get_balance(account); balance -= amount; Local = 900 TL ● put_balance(account, balance); Balance = 900 TL! Although two customers withdrew 100 TL each, final balance is 900 TL, not 800 TL : (

Carnegie Mellon Mutual Exclusion ● ● ● To achieve synchronization, ensure only one thread at a time can execute code in their critical section. All other threads are forced to wait on entry. When one thread leaves the critical section, another can enter. Critical Section Thread 1 (modify account balance)

Carnegie Mellon Mutual Exclusion ● ● ● To achieve synchronization, ensure only one thread at a time can execute code in their critical section. All other threads are forced to wait on entry. When one thread leaves the critical section, another can enter. Critical Section Thread 2 2 nd thread must wait for critical section to clear Thread 1 (modify account balance)

Carnegie Mellon Mutual Exclusion ● ● ● To achieve synchronization, ensure only one thread at a time can execute code in their critical section. All other threads are forced to wait on entry. When one thread leaves the critical section, another can enter. Critical Section Thread 2 (modify account balance) 2 nd thread free to enter 1 st thread leaves critical section

Carnegie Mellon Mutual Exclusion ● pthread library provides us mutex variables to control the critical section access. ● pthread_mutex_lock(&my. Mutex) ●. . //critical section stuff ● pthread_mutex_unlock(&my. Mutex) ● See this in action here: http: //user. ceng. metu. edu. tr/~ys/ceng 334 os/threadster 1. c

Carnegie Mellon Critical Section ● Critical section requirements. ● Mutual exclusion: at most 1 thread is currently executing in the critical section. ● Progress: if thread T 1 is outside the critical section, then T 1 cannot prevent T 2 from entering the critical section. ● No starvation: if T 1 is waiting for the critical section, it’ll eventually enter. ● Assuming threads eventually leave critical sections. ● Performance: the overhead of entering/exiting critical section is small w. r. t. the work being done within it.

Carnegie Mellon Mutual Exclusion ● ● ● A toy solution to mutual exclusion. Programming at the application (sw solution; no hw or kernel support). Is this solution OK? ● ● Set global variable lock = 1. A thread that wants to enter critical section checks lock == 1. ● If true, enter. Do lock--. ● If false, another thread decremented it so not enter.

Carnegie Mellon Mutual Exclusion ● ● ● A toy solution to mutual exclusion. Programming at the application (sw solution; no hw or kernel support). Is this solution OK? ● ● Set global variable lock = 1. A thread that wants to enter critical section checks lock == 1. ● If true, enter. Do lock--. ● If false, another thread decremented it so not enter. No because lock itself is a shared global variable. Just using a single variable without any other protection is not enough.

Carnegie Mellon Mutual Exclusion ● ● Peterson’s solution to mutual exclusion. Programming at the application (sw solution; no hw or kernel support). ● ● Peterson. enter //similar to pthread_mutex_lock(&my. Mutex). . //critical section stuff Peterson. exit//similar to pthread_mutex_unlock(&my. Mutex) Works for 2 threads/processes (not more).

Carnegie Mellon Mutual Exclusion ● ● ● Peterson’s solution to mutual exclusion. Assume that the LOAD and STORE machine instructions are atomic; that is, cannot be interrupted. The two threads share two variables: ● int turn; ● boolean flag[2]; The variable turn indicates whose turn it is to enter the critical section. ● turn = i means thread Ti can execute (i=0, 1). The flag array is used to indicate if a thread is ready to enter the critical section. flag[i] = true implies that thread Ti is ready (wants to enter).

Carnegie Mellon Mutual Exclusion ● ● ● The variable turn indicates whose turn it is to enter the critical section. ● turn = i means thread Ti can execute (i=0, 1). The flag array is used to indicate if a thread is ready to enter the critical section. flag[i] = true implies that thread Ti is ready (wants to enter). Algorithm for Ti; the other thread is. Tj. I want to enter but, be nice to other thread busy-wait (spinlock)

Carnegie Mellon Mutual Exclusion ● Peterson’s algorithm from both threads’ perspectives: THREAD i (0) THREAD j (1) do { flag[i] = TRUE; turn = j; while (flag[j] && turn == j); critical section. . flag[i] = FALSE; remainder section. . } while (1) flag[j] = TRUE; turn = i; while (flag[i] && turn == i); critical section. . flag[j] = FALSE; remainder section. . } while (1) Shared Variables: flag[] turn i=0, j=1 are local.

Carnegie Mellon Mutual Exclusion ● ● ● Peterson’s algorithm is a software solution to mutex. Let’s get an hardware support for mutual exclusion. Strategy 1: kernel code disables clock interrupts (hence, no context switches) Disable interrupts (no switch) Enable interrupts (schedulable)

Carnegie Mellon Mutual Exclusion ● ● ● Strategy 1 works for single-CPU systems. Multi-CPU fails because you are disabling the interrupt only for your processor. That does not mean other processors do not get interrupts. ● Each processor has its own interrupt mechanism. Hence another process/thread running in another processor can touch the shared data. Too inefficient to disable interrupts on all the available processors.

Carnegie Mellon Mutual Exclusion ● ● Strategy 2: Complex machine instructions from hardware that are atomic (not interruptible). Locks (not just simple integers). do { acquire lock critical section release lock remainder section } while (TRUE); ● How to implement acquire/release lock? ● Use special machine instructions: Test. And. Set, Swap.

Carnegie Mellon Mutual Exclusion ● ● ● Strategy 2: Complex machine instructions from hardware that are atomic (not interruptible). Test. And. Set is a machine/assembly instruction. You must write the acquire-lock portion (entry section code) of your code in assembly. But here is a C code for easy understanding: --Definition of Test. And. Set Instruction-- boolean Test. And. Set (boolean *target) { boolean rv = *target; *target = TRUE; return rv: } //atomic (not interruptible)

Carnegie Mellon Mutual Exclusion ● ● ● Strategy 2: Complex machine instructions from hardware that are atomic (not interruptible). Test. And. Set in action: We use a shared boolean variable lock, inited to false. do { while ( Test. And. Set (&lock ) ) ; //do nothing; busy wait // critical section lock = FALSE; //release lock // } while (TRUE); remainder section

Carnegie Mellon Mutual Exclusion ● ● ● Strategy 2: Complex machine instructions from hardware that are atomic (not interruptible). Test. And. Set in assembly: Can be suspended/interrupted between Test. And. Set & CMP, but not during Test. And. Set.

Carnegie Mellon Mutual Exclusion ● ● Strategy 2: Complex machine instructions from hardware that are atomic (not interruptible). Writing assembly in C is not a big deal:

Carnegie Mellon Mutual Exclusion ● ● ● Strategy 2: Complex machine instructions from hardware that are atomic (not interruptible). Swap is a machine/assembly instruction. You must write the acquire-lock portion (entry section code) of your code in assembly. But here is a C code for easy understanding: --Definition of Swap Instruction-- boolean Swap (boolean* a, boolean* b) { boolean temp = *a; *a = *b; *b = temp; } //atomic (not interruptible)

Carnegie Mellon Mutual Exclusion ● ● Strategy 2: Complex machine instructions from hardware that are atomic (not interruptible). Swap in action: We use a shared boolean variable lock, inited to false. Each thread also has a local boolean variable key. do { key = TRUE; while (key == TRUE) Swap (&lock, &key ); // critical section lock = FALSE; // } while (TRUE); remainder section

Carnegie Mellon Mutual Exclusion ● ● Strategy 2: Complex machine instructions from hardware that are atomic (not interruptible). A comment on Test. And. Set & Swap. ● Although they both guarantee mutual exclusion, they make one thread (X) wait a lot: ● A thread X may be waiting, but we can have the other process Y going into the critical region repeatedly. ● One toy/bad solution: keep the remainder section code so long that scheduler kicks Y out of the CPU before it reaches back to the entry section.

Carnegie Mellon Semaphores ● ● A better solution for mutual exclusion is using semaphores. Idea: avoid busy waiting: waste of CPU cycles by waiting in a loop until the lock is available, aka spinlock. ● ● Example 1: while (flag[i] && turn == i); //from Peterson’s algo. Example 2: while (Test. And. Set (&lock )); //from Test. And. Set algo.

Carnegie Mellon Semaphores ● ● ● A better solution for mutual exclusion is using semaphores. Idea: avoid busy waiting: waste of CPU cycles by waiting in a loop until the lock is available, aka spinlock. How to avoid? ● If a process P calls wait() on a semaphore with a value of zero, P is added to the semaphore’s queue and then blocked. ● The state of P is switched to the waiting state, and control is transferred to the CPU scheduler, which selects another process to execute (instead of busy waiting on P). ● When another process increments the semaphore by calling signal() and there are tasks on the queue, one is taken off of it and resumed. ● wait() = P() = down() //modify semaphore s via these functs. ● signal() = V() = up() //modify semaphore s via these functs.

Carnegie Mellon Semaphores ● wait() and signal() can be implemented in kernel as system calls. Kernel makes sure that wait(s) & signal(s) are atomic. ● Less complicated entry & exit sections. ●

Carnegie Mellon Semaphores ● Operations (kernel codes). Busy-waiting ● Actually, s. value--; s. list. add(this); but keeping the code simple above. ● vs. Efficient

Carnegie Mellon Semaphores ● ● Operations (kernel codes). wait(s): if s positive s-- and return else s-- and block/wait (‘till somebody wakes you up; then return)

Carnegie Mellon Semaphores ● ● Operations (kernel codes). signal(s): s++ if 1 or more processes are waiting (s<=0) wake one of them up return

Carnegie Mellon Semaphores ● ● ● Types. Binary semaphore s. ● Integer value of s can range only between 0 and 1; can be simpler to implement; aka mutex locks. ● Provides mutual exclusion; can be used for the critical section problem. Counting semaphore s. ● Integer value of s can range over an unrestricted domain. ● Can be used for other synchronization problems; for example for resource allocation. ● Example: you have 10 instances of a resource. Init s to 10 in this case.

Carnegie Mellon Semaphores ● ● Implementation. Semaphore s can be shared by multiple threads. s can be modified only by atomic system calls: wait() & signal(). s has a queue of waiting processes/threads that might be sleeping on it. typedef struct { int value; struct process *list; } semaphore; ● ● Atomic: when thread X is executing wait(), Y can execute wait() if X finished executing wait() or X is blocked in wait(). When X is executing signal(), Y can execute signal() if X finished.

Carnegie Mellon Semaphores ● ● ● Binary semaphores can be used to solve critical section problems. A semaphore variable (lets say mutex) can be shared by N processes, and initialized to 1. Each process is structured as follows: do { wait (mutex); // Critical Section signal (mutex); // remainder section } while (TRUE);

Carnegie Mellon Semaphores ● Binary semaphores can be used to solve critical section problems. Thread T 0 do { wait (mutex); // Critical Section signal (mutex); // remainder section } while (TRUE); Thread T 1 do { wait (mutex); // Critical Section signal (mutex); // remainder section } while (TRUE); wait() {…} signal() {…} Semaphore mutex; //initialized to 1 Kernel

Carnegie Mellon Semaphores ● ● Assume T 0 in critical region, i. e. mutex=0. Kernel has a list of processes waiting on mutex; so when T 1 tries to enter, kernel adds T 1 to this list by calling block() (in wait()) on it. When a wakeup() (in signal()) is called, kernel picks a process from that list and resumes it (waiting ready state). In busy-wait solution, we were just looping around, but here we just put T 1 to sleep mode & do something else. Thread T 0 do { wait (mutex); // Critical Section signal (mutex); // remainder section } while (TRUE); Thread T 1 do { wait (mutex); // Critical Section signal (mutex); // remainder section } while (TRUE); wait() {…} signal() {…} Semaphore mutex; //initialized to 1 Kernel

Carnegie Mellon Semaphores ● Kernel puts processes/threads waiting on s in a FIFO queue. Why FIFO?

Carnegie Mellon Semaphores ● ● Kernel puts processes/threads waiting on s in a FIFO queue. Why FIFO? To prevent starvation. If you remove the last thread added (stack) we can have starvation and never serve/wakeup the first process. Priority queues can be used though, to prioritize some threads, e. g. , admin threads.

Carnegie Mellon Semaphores ● ● ● We addressed synchronization for critical section access, e. g. , via mutexes. Now let’s solve another synchronization problem using a binary semaphore. Ensure statement S 1 definitely executes before S 2. T 0 … S 1; …. T 1 … S 2; …. Solution? Hint: use Semaphore x = 0; //inited to 0

Carnegie Mellon Semaphores ● ● ● We addressed synchronization for critical section access, e. g. , via mutexes. Now let’s solve another synchronization problem using a binary semaphore. Ensure statement S 1 definitely executes before S 2. T 0 … S 1; …. T 1 … S 2; …. Solution: Semaphore x = 0; //inited to 0 T 1 … … S 1; wait(x); signal(x); S 2; …. ….

Carnegie Mellon Semaphores ● ● ● Solve yet another synchronization problem using a counting semaphore. Resource allocation: We have N threads that want a resource that has 5 instances. Solution? Hint: use Semaphore rs = 5;

Carnegie Mellon Semaphores ● ● ● Solve yet another synchronization problem using a counting semaphore. Resource allocation: We have N threads that want a resource that has 5 instances. Solution: Semaphore rs = 5; Every process that wants to use R will do wait(rs); ● If some instance is available, that means rs will be nonnegative no blocking. ● If all 5 instances are used, that means rs will be negative block until rs is nonnegative. Every process that finishes with R will do signal(rs); ● A blocked processes will change state from waiting to ready.

Carnegie Mellon Semaphores ● ● Solve yet another synchronization problem using a counting semaphore. Producer-Consumer problem discussed before (Slide 14). Enforce consumer to sleep while there’s no item in the buffer. Solution? Hint: use Semaphore Full. Cells = 0; //initialized to 0.

Carnegie Mellon Semaphores ● ● Solve yet another synchronization problem using a counting semaphore. Producer-Consumer problem discussed before (Slide 14). Enforce consumer to sleep while there’s no item in the buffer. Solution: do { //PRODUCER // produce item. . put item into buffer. . signal (Full_Cells); } while (TRUE); do { //CONSUMER wait (Full_Cells); //instead of busy-waiting, go to sleep mode and give CPU back to producer for faster production (efficiency!!). . . remove item from buffer. . } while (TRUE); Semaphore Full_Cells = 0; //initialized to 0 Kernel

Carnegie Mellon Semaphores ● ● In the previous Producer-Consumer problem, there is no limit on the buffer size, which is unrealistic. Let’s handle this by blocking Producer when the buffer is full, aka Bounded-Buffer Problem. Here is the problem restated with these constraints: ● Producer should not produce any item if the buffer is full: Semaphore full = 0; //inited ● Consumer should not consume any item if the buffer is empty: Semaphore empty = N; ● Producer and consumer should access the buffer in a mutually exclusive manner: Semaphore mutex = 1; buffer prod cons full = 4 empty = 6

Carnegie Mellon Semaphores ● ● In the previous Producer-Consumer problem, there is no limit on the buffer size, which is unrealistic. Let’s handle this by blocking Producer when the buffer is full, aka Bounded-Buffer Problem. Here is the solution with 2 counting and 1 binary semaphores:

Carnegie Mellon Semaphores ● ● ● Another synch problem: Readers-Writers problem. A data* set is shared among a number of concurrent processes. Readers: only read the data set; they do not perform any updates. Writers: can both read and write. Problem: allow multiple readers to read at the same time. Only one single writer can access the shared data at the same time (no reader/writer when writer is active). * a file, a shared global var, a shared stack, a var in shared memo, grades in oibs. metu, etc.

Carnegie Mellon Semaphores ● ● ● ● Another synch problem: Readers-Writers problem. Readers: only read the data set; they do not perform any updates. Writers: can both read and write. Solution? Hint: use Integer readcount initialized to 0. ● Number of readers reading the data at the moment. Semaphore mutex initialized to 1. ● Protects the readcount variable (multiple readers may try to modify it). Semaphore wrt initialized to 1. ● Protects the shared data (either writer or reader(s) should access data at a time). Hint # 2: First and last reader should do something special.

Carnegie Mellon Semaphores ● ● Another synch problem: Readers-Writers problem. Readers: only read the data set; they do not perform any updates. Writers: can both read and write. Solution: Recall wait/signal:

Carnegie Mellon Semaphores ● ● ● Another synch problem: Readers-Writers problem. Readers: only read the data set; they do not perform any updates. Writers: can both read and write. Solution: Recall wait/signal: wait(wrt): ensure that I’m the only writer (no reader or other writer) updating shared data. wait(mutex): ensure that I’m the only reader updating readcount.