Chapter 4 Threads A fundamental unit of CPU

Chapter 4: Threads A fundamental unit of CPU utilization Thanks to the author of the textbook [SGG] for providing the base slides. I made several changes/additions. These slides may incorporate materials kindly provided by Prof. Dakai Zhu. So I would like to thank him, too. Turgay Korkmaz Operating System Concepts 4. 1 SGG

Chapter 4: Threads n Overview * n Multithreading Models ***** n Thread Libraries **** n Pthreads and Java thread n Threading Issues *** n Operating System Examples * n Windows XP Threads * n Linux Threads *** Operating System Concepts 4. 2 SGG

Objectives n To introduce the notion of a thread n a fundamental unit of CPU utilization that forms the basis of multithreaded computer systems n To examine issues related to multithreaded programming n To discuss the APIs for the Pthreads and Java thread libraries (optional Win 32) Operating System Concepts 4. 3 SGG

. Example: A Multi-Activity Text Editor n Process approach on data l P 1: read from keyboard l P 2: format document l P 3: write to disk When in the Course of human events, it becomes necessary for one people to dissolve the political bands which have connected them with another, and to assume among the powers of the earth, the separate and equal station to which the Laws of Nature and of Nature's God entitle them, a decent respect to the opinions of mankind requires that they should declare the causes which impel them to the separation. We hold these truths to be self-evident, that all men are created equal, that they are endowed by their Creator with certain unalienable Rights, that among these are Life, Liberty and the pursuit of Happiness. --That to secure these rights, Governments are instituted among Men, deriving their just powers from the consent of the governed, --That whenever any Form of Government becomes destructive of these ends, it is the Right of the People to alter or to abolish it, and to institute new Government, laying its foundation on such principles and organizing its powers in such form, as to them shall seem most likely to effect their Safety and Happiness. Prudence, indeed, will dictate that Governments long established should not be changed for light and transient causes; and accordingly all The processes will actively access the same set of data. How do the processes exchange data? Kernel Context Switch for Processes- costly Operating System Concepts 4. 4 SGG

. Ideal Solution for the Text Editor Threads n Three activities within one process l Single address space l Same execution environment l Data shared easily When in the Course of human events, it becomes necessary for one people to dissolve the political bands which have connected them with another, and to assume among the powers of the earth, the separate and equal station to which the Laws of Nature and of Nature's God entitle them, a decent respect to the opinions of mankind requires that they should declare the causes which impel them to the separation. We hold these truths to be self-evident, that all men are created equal, that they are endowed by their Creator with certain unalienable Rights, that among these are Life, Liberty and the pursuit of Happiness. --That to secure these rights, Governments are instituted among Men, deriving their just powers from the consent of the governed, --That whenever any Form of Government becomes destructive of these ends, it is the Right of the People to alter or to abolish it, and to institute new Government, laying its foundation on such principles and organizing its powers in such form, as to them shall seem most likely to effect their Safety and Happiness. Prudence, indeed, will dictate that Governments long established should not be changed for light and transient causes; and accordingly all n Switch between activities l Only running context l No change in address space Operating System Concepts 4. 5 Kernel SGG

Another Example: Web servers Operating System Concepts 4. 6 SGG

Thread vs. Process Both are methods for Concurrency and Parallelism n Processes l Independent execution units use their own states, address spaces, and interact with each other via IPC l Traditional Processes have single flow of control (thread) n Thread l Flow of control (activity) within a process l A single process on a modern OS may have multiple threads l All threads share the code, data, and files while having some separated resources l Threads directly interact with each other using shared resources Operating System Concepts 4. 7 Concurrency vs. Parallelism? SGG

Benefits n Responsiveness l Concurrent Execution on a Single-core System Interactive application n Resource sharing l Address space and other resources n Economy: less overhead l Solaris: process creation 30 X overhead than thread; l Context switching threads within a process 5 X faster Parallel Execution on a Multicore System n Scalability l Better utilization of multiprocessor/multicore systems Operating System Concepts 4. 8 SGG

Multicore Programming n Multithreaded programming provides a mechanism for efficient use of multicore systems n Programmers face new challenges l Dividing activities l Balance l Data splitting l Data dependency l Testing and debugging n Multicore programming will require entirely new approach to design SW Operating System Concepts 4. 9 SGG

MULTI-THREADING MODELS Operating System Concepts 4. 10 SGG

. Thread Implementations: Issues n Process usually starts with a single thread and creates others n Thread management operations (similar to process management) l Creation: procedure/method for the new thread to run l Scheduling: runtime properties/attributes l Destruction: release resources l Thread Synchronization 4 join, wait, etc. n Who manages threads and where l User space: managed by applications using some libraries l Kernel space: managed by OS 4 all modern OSes have kernel level support (more efficient) Operating System Concepts 4. 11 SGG

. User-Level Threads n User threads: thread library at the user level n Run-time system provides support for thread creation, scheduling and management Process Run-time system User Kernel has NO knowledge of threads! Space Thread table Kernel Thread Kernel Space Process Table Operating System Concepts 4. 12 SGG

. User-Level Threads (cont. ) n Each process needs its own private thread table to keep track of the threads in that process. n The thread-table keeps track of the per-thread items (program counter, stack pointer, register, state. . ) n When a thread does something n Advantages l Fast thread switching: no kernel activity involved l Customized/flexible thread scheduling algorithm l Application portability: different machines with library n Problems/disadvantages: that may cause it to become blocked locally (e. g. wait for another thread), l Kernel only knows process it calls a run-time system l Blocking system calls: procedure. kernel blocks process, so one thread blocks all n If the thread must be put into activities (many-to-one mapping) blocked state, the procedure l All threads share one CPU, performs thread switching so cannot use multiproc. /core Operating System Concepts 4. 13 SGG

. Kernel-Level Threads Supported directly by OS Kernel performs thread creation, scheduling & management in kernel space Process Thread User Space Kernel Space Thread table Operating System Concepts 4. 14 Process Table SGG

. Kernel-Level Threads (cont. ) n Advantages l User activity/thread with blocking I/O does NOT block other activities/threads from the same user l When a thread blocks, the kernel may choose another thread from the same or different process l Multi-activities in applications can use multi-proc/cores n Problems/disadvantages l Thread management could be relatively costly: 4 all methods that might block a thread are implemented as system calls l Non-flexible scheduling policy l Non-portability: application can only run on same type of machine What is the relationship between user level and kernel level threads? How to map user level threads to kernel level threads? Operating System Concepts 4. 15 SGG

Mapping: User Kernel Threads n Many-to-one l Many user threads one kernel thread (-/+ are same as in user-level threads) Examples: Solaris Green Threads, GNU Portable Threads n One-to-One l One user thread one kernel thread; l + more concurrency l - limited number of kernel threads Examples: Windows NT/XP/2000, Linux, Solaris 9 and later n Many-to-Many l Many user threads many kernel threads l + no limit on the number of user threads l - not true concurrency because kernel has limited number of threads Examples: Solaris prior to version 9, Windows NT/2000 with the Thread. Fiber package Operating System Concepts 4. 16 SGG

Two-level Model n Similar to M: M, except that it allows a user thread to be bound to kernel thread n Examples l IRIX l HP-UX l Tru 64 UNIX l Solaris 8 and earlier Operating System Concepts 4. 17 SGG

* Hybrid Implementation combine the best of both approaches n Use kernel-level threads, and then multiplex user-level threads onto some or all of the kernel threads. Operating System Concepts 4. 18 SGG

. Light-Weight Process (LWP) n Lightweight process (LWP): intermediate data structure l For user-level threads, LWP is a Virtual processor l Each LWP attaches to a kernel thread n Multiple user-level threads a single LWP l Normally from the same process n A process may be assigned multiple LWPs l Typically, an LWP for each blocking system call n OS schedules kernel threads (hence, LWPs) on the CPU Operating System Concepts 4. 19 SGG

LWP: Advantages and Disadvantages n + User level threads are n Occasionally, we still easy to create/destroy/sync need to create/destroy LWP (as expensive as kernel threads) n + A blocking call will not suspend the process if we have enough LWP n Makes up calls (scheduler activation) n + Application does not l need to know about LWP + simplifies LWP management l - Violates the layered structure n +LWP can be executed on different CPUs, hiding multiprocessing Operating System Concepts 4. 20 SGG

Provide programmers with API for creating and managing threads THREAD LIBRARIES Operating System Concepts 4. 21 SGG

Thread Libraries n Two primary ways of implementing l User-level library 4 Entirely in user space 4 Everything l is done using function calls (no system calls) Kernel-level library supported by the OS 4 Code and data structures for kernels are in kernel space 4 Function calls result in system calls to kernel n Three primary thread libraries: l POSIX Threads Pthread l Win 32 threads (kernel-level) l Java threads (either a user-level or kernel-level) (JVM manages threads by using host system threads) 4 Threads are fundamental model of prog exec, 4 Java provides rich set of features for thread creation and mng. Operating System Concepts 4. 22 SGG

POSIX Threads: Pthread n POSIX l Portable Operating System Interface [for Unix] l Standardized programming interface n Pthread l A POSIX standard (IEEE 1003. 1 c) API for thread creation and synchronization l API specifies behavior of the thread library, implementation is up to development of the library (Defined as a set of C types and procedure calls) l Common in UNIX operating systems (Solaris, Linux, Mac OS X) n Implementations l User-level vs. kernel-level https: //computing. llnl. gov/tutorials/pthreads/ Operating System Concepts 4. 23 SGG

. Pthread APIs: Four Groups Thread Call n Thread management Routines for creating, detaching, joining, etc. l Routines for setting/querying thread attributes l n Mutexes: abbreviation for "mutual exclusion" Routines for creating, destroying, locking/unlocking l Functions to set or modify attributes with mutexes. l C h a pn t e r 6 Conditional variables Communications for threads that share a mutex l Functions to create, destroy, wait and signal based on specified variable values l Functions to set/query condition variable attributes l pthread_create Create a new thread in the caller’s address space pthread_exit Terminate the calling thread pthread_join Wait for a thread to terminate pthread_mutex_init Create a new mutex pthread_mutex_destroy Destroy a mutex pthread_mutex_lock Lock a mutex pthread_mutex_unlock Unlock a mutex pthread_cond_init Create a condition variable pthread_cond_destroy Destroy a condition variable pthread_cond_wait Wait on a condition variable pthread_cond_signal Release one thread waiting on a condition variable n Synchronization l Routines that manage read/write locks and barriers Operating System Concepts 4. 24 Description SGG

. Thread Creation pthread_t thread. ID; pthread_create (&thread. ID, *attr, method. Name, *para); n 1 st argument is the ID of the new thread n 2 nd argument is a pointer to pthread_attr_t n 3 rd argument is thread (function/method) name n 4 th argument is a pointer to the arguments for the thread’s method/function Operating System Concepts 4. 25 SGG

. Thread: Join and Exit n Join with a non-detached thread by using pthread_join ( pthread_t thread, void **status) (All threads are created non-detached by default, so they are “joinable” by default) n Exit from threads: l If threads use exit( ), process terminates. l A thread (main, or another thread ) can exit by calling pthread_exit( ), this does not terminate the process. n More information about Pthread programming https: //computing. llnl. gov/tutorials/pthreads/ Operating System Concepts 4. 26 SGG

. An Example: testthread. c #include <pthread. h> #include <stdio. h> #include <stdlib. h> #define NUM_THREADS To compile, link with the pthread library. 5 > gcc testthread. c -o test –lpthread void *Print. Hello(void *threadid){ > test long tid; In main: creating thread 0 tid = (long)threadid; In main: creating thread 1 printf("Hello World! It's me, thread #%ld!n", tid); Hello World! It's me, thread pthread_exit(NULL); In main: creating thread 2 } Hello World! It's me, thread In main: creating thread 3 int main(int argc, char *argv[]){ Hello World! It's me, thread pthread_t threads[NUM_THREADS]; In main: creating thread 4 int rc; Hello World! It's me, thread long t; Hello World! It's me, thread for(t=0; t<NUM_THREADS; t++){ printf("In main: creating thread %ldn", t); rc = pthread_create(&threads[t], NULL, Print. Hello, (void *)t); if (rc){ printf("ERROR; return code from pthread_create() is %dn", rc); exit(-1); } } // for(t=0; t<NUM_THREADS; t++) // pthread_join(threads[t], NULL); // wait for other threads pthread_exit(NULL); //to return value; } Operating System Concepts 4. 27 #0! #1! #2! #3! #4! SGG

Threads are fundamental model of program execution in Java. So, Java provides a rich set of features for thread creation and management JAVA THREADS http: //docs. oracle. com/javase/7/docs/api/java/lang/Thread. html Operating System Concepts 4. 28 SGG

How to Create Threads in Java (1) There are two ways in Java: 1. Create a class My. Th that directly extends Thread class The code in My. Th. run() will be the new thread l Then in a driver program l 4 My. Th th = new My. Th( … ); 4 th. start(); l public class Thread { … Not recommended (why? ) 4 4 4 public class My. Th extends Thread { public My. Th(…){ … } public void run() { //overwrite this … } } public String get. Name(); A bad habit for industrial strength development public void interrupt(); The methods of the worker class and the Thread class get all tangled up public void join(); Makes it hard to migrate to Thread Pools and other more efficient approaches public void set. Name(String name); public boolean is. Alive(); public void set. Daemon(boolean on); public void set. Priority(int level); public static Thread current. Thread(); public static void sleep(long ms); public static void yield(); Operating System Concepts 4. 29 } SGG

How to Create Threads in Java (2) 2. Define a class My. Th that implements Runnable interface l The code in My. Th. run() will be the new thread l Then in a driver program 4 Thread th = new Tread( new My. Th(…) ); 4 th. start(); public class My. Th implements Runnable { public My. Th(…){ … } public void run() { //overwrite this … } } Operating System Concepts 4. 30 SGG

. Example 1: Extend Thread Class public class Simple. Thread extends Thread { String msg; int repetition; public Simple. Tread(String msg, int r){ this. msg = msg; this. repetition = r; } public void run() { //overwrite run method for (int i = 0; i < repetition; i++) System. out. println("[" + i + "]" + msg); } public class Simple. Thread. Main { } public static void main(String[] args) { Simple. Thread t 1 = new Simple. Thread("T 1", 100); t 1. start(); Simple. Thread t 2 = t 2. start(); new Simple. Thread("T 2", 100); } } Operating System Concepts 4. 31 SGG

. Example 2: Implement Runnable interface public class Simple. Runnable implements Runnable { String msg; int repetition; public Simple. Runnable(String msg, int r) { this. msg = msg; this. repetition = r; } public void run( ) { //overwrite run method for (int i = 0; i < repetition; i++) System. out. println("[" + i + "]" + msg); } public class Simple. Runnable. Main { } public static void main(String[] args) { Simple. Runnable r 1 = new Simple. Runnable("T 1", 100); Thread t 1 = new Thread(r 1); t 1. start( ); [0]T 1 [0]T 2 [1]T 1 [1]T 2 [2]T 1 [2]T 2 [3]T 1 [3]T 2 [4]T 1 … Simple. Runnable r 2 = new Simple. Runnable("T 2", 100); Thread t 2 = new Thread(r 2); t 2. start( ); } } Operating System Concepts 4. 32 SGG

. Use join() to wait for a thread to finish http: //docs. oracle. com/javase/7/docs/api/java/lang/Thread. html The join method of Thread throws Interrupted. Exception and must be placed in a try-catch. … try { t 1. join( ); } catch (Interrupted. Exception e) {} System. out. println("Both are done "); main [0]T 1 [0]T 2 [1]T 1 main [1]T 2 [2]T 1 [2]T 2 [3]T 1 main [3]T 2 [4]T 1 …… Both are done Operating System Concepts for (int i = 0; i < 100; i++) System. out. println(“main”); 4. 33 [0]T 1 [0]T 2 [1]T 1 [1]T 2 [2]T 1 [2]T 2 [3]T 1 [3]T 2 [4]T 1 …… main …… Both are done SGG

Java Thread Example - Output public class Thread. Example implements Runnable { public void run() { for (int i = 0; i < 3; i++) System. out. println(i); } public static void main(String[] args) { new Thread( new Thread. Example()). start(); System. out. println("Done"); } What are the possible outputs? Why doesn’t the 0, 1, 2, Done // thread 1, thread 2, main() } program quit as 0, 1, 2, Done, 0, 1, 2, 0, 1, 2 0, 0, 1, 1, 2, Done, 2 // thread 1, main(), thread 2 // main(), thread 1, thread 2 // main() & threads interleaved soon as “Done” is printed? JVM shuts down when all non-daemon threads terminate! Operating System Concepts 4. 34 SGG

Life-Time of Java Threads n When are thread objects created? n When do they run? n When does the program finish? n A thread object exists when it is constructed, but it doesn't start running until the start method is called. n A thread completes (or dies) when its run method finishes or when it throws an exception. The object representing this thread can still be accessed. Operating System Concepts 4. 35 SGG

Transitions between Threads n Transitions between states caused by l Invoking methods in class Thread 4 start(), yield(), sleep(), wait(), join() 4 The join, wait, and sleep methods of Thread throw Interrupted. Exception and must be placed in a try-catch. n Other (external) events l Scheduler, I/O, returning from run()… n Scheduler (ch 5) Determines which runnable threads to run l Part of OS or Java Virtual Machine (JVM) l Many computers can run multiple threads simultaneously (or nearly so) l Operating System Concepts 4. 36 SGG

Another Example: Java Threads Define a class that implements Runnable interface Operating System Concepts 4. 37 SGG

Another Example: Producer-Consumer Define a class that implements Runnable interface Operating System Concepts 4. 38 SGG

Semantics of fork() and exec() system calls Thread cancellation of target thread Signal handling Thread pools Thread-specific data Scheduler activations THREADING ISSUES Operating System Concepts 4. 39 SGG

Semantics of fork() and exec() n What will happen if one thread in a process call fork( ) to create a new process? l Does fork() duplicate only the calling thread or all threads? l How many threads in the new process? n Duplicate only the invoking thread l exec(): will load another program l Everything will be replaced anyway n Duplicate all threads l If exec() is not the next step after forking l What about threads performed blocking system call? ! Operating System Concepts 4. 40 SGG

Thread Cancellation n Terminating a thread before it has finished l Examples 4 Threads search in parallel of database: one finds others stop 4 Stop fetching web contents (images) n Two general approaches: l Asynchronous cancellation terminates the target thread immediately 4 - l Thread resources and data consistency Deferred cancellation allows the target thread to periodically check if it should be cancelled 4+ Wait for self cleanup cancellation safety points Operating System Concepts 4. 41 SGG

Signal Handling 1. 2. 3. Signal is generated by particular event Signal is delivered to a process Signal is handled n Signals are used in UNIX systems to notify a process that a particular event has occurred. l Depending on the source, we can classify them as 4 Synchronously [Running prog generates it] (e. g. , div by 0, memory access) 4 Asynchronously [External src generates it] (e. g. , ready of I/O or Ctrl+C) n Which threads to notify? l All threads (Ctrl-C) l Single thread to which the signal applies (illegal memory, div by 0) l Subset of threads: thread set what it wants (mask) l Thread handler: kernel default or user-defined n Unix allows threads to specify which one to block or accept n Windows has no support for signals but it can be emulated Operating System Concepts 4. 42 SGG

Thread Pool n Recall web server example, l We created a thread for every request l This is better than creating a process, but still time consuming l No limit is put on the number of threads n Pool of threads l Create some number of treads at the startup l These threads will wait to work and put back into pool n Advantages: l Usually slightly faster to service a request with an existing thread than create a new thread l Allows the number of threads in the application(s) to be bound to the size of the pool n Adjust thread number in pool l According to usage pattern and system load Operating System Concepts 4. 43 SGG

. Thread Pool Example: Web server Dispatcher thread while(TRUE) { get. Next. Request(&buf); handoff. Work(&buf); } Worker thread Kernel Network connection Operating System Concepts Web page cache while(TRUE) { wait. For. Work(&buf); look. For. Page. In. Cache(&buf, &page); if(page. Not. In. Cache(&page)) { read. Page. From. Disk(&buf, &page); } return. Page(&page); } 4. 44 SGG

. Java thread pool example class Task implements Runnable { public void run() { System. out. println("I am working on a task. "); } } import java. util. concurrent. *; n Work queue public class TPExample { l Fixed number public static void main(String[] args) { int num. Tasks = Integer. parse. Int(args[0]. trim()); // create thread pool Executor. Service pool = Executors. new. Cached. Thread. Pool(); // run each task using a thread in the pool for (int i = 0; i < 5; i++) pool. execute(new Task()); // sleep for 5 seconds try { Thread. sleep(5000); } catch (Interrupted. Exception ie) { } pool. shutdown(); of threads Possible problems l Deadlock l Resource thrashing l Thread leakage l Overload } } Operating System Concepts 4. 45 SGG

Thread Specific Data n Allows each thread to have its own copy of data n We may not want to share all data n Thread libraries have support for this n Useful when you do not have control over the thread creation process (i. e. , when using a thread pool) Operating System Concepts 4. 46 SGG

Scheduler Activations n Both M: M and Two-level models require communication to maintain the appropriate number of kernel threads allocated to the application n Scheduler activations provide upcalls - a communication mechanism from the kernel to the thread library l Events to invoke upcall 4 A thread make a blocking system calls 4 A blocking system call complete returns 4 To ask user-level thread scheduler (runtime systems) to select the next runnable thread n This communication allows an application to maintain the correct number of kernel threads Operating System Concepts 4. 47 SGG

SKIP Windows XP Threads Linux Thread OPERATING SYSTEM EXAMPLES Operating System Concepts 4. 48 SGG

Windows XP Threads n Implements the one-to-one mapping, kernel-level n Each thread contains l A thread id l Register set l Separate user and kernel stacks l Private data storage area n The register set, stacks, and private storage area are known as the context of the threads n The primary data structures of a thread include: l ETHREAD (executive thread block) l KTHREAD (kernel thread block) l TEB (thread environment block) Operating System Concepts 4. 49 SGG

Windows XP Threads Operating System Concepts 4. 50 SGG

* Linux Threads n Linux uses the term task (rather than process or thread) when referring to a flow of control n Linux provides clone() system call to create threads l A set of flags, passed as arguments to the clone() system call determine how much sharing is involved (e. g. open files, memory space, etc. ) n Linux: 1 -to-1 thread mapping l NPTL (Native POSIX Thread Library) 51 Operating System Concepts 4. 51 SGG

Linux Threads Operating System Concepts 4. 52 SGG

End of Chapter 4 Operating System Concepts 4. 53 SGG