Processes and Threads Case studies Windows and Linux

Processes and Threads Case studies: Windows and Linux Lecture 6 ~ Fall, 2007 ~

Contents • Windows 2000 process management • Linux process management Fall, 2007 TUCN. Operating Systems. Lecture 6 No. 2

Windows processes Fundamental concepts • Jobs • Each job has one or more processes • Processes • Each process has one or more threads • Threads • Kernel threads - each thread has one or more fibers • Fiber • User space threads Fall, 2007 TUCN. Operating Systems. Lecture 6 No. 3

Windows processes Jobs • A collection of one or more processes managed as a single unit • Each job object has quotas and resource limits associated with it • • maximum number of processes CPU time available per process and per job maximum memory usage per process and per job security restrictions imposed on processes • Win 32 API • • Fall, 2007 Create. Job. Object Assign. Process. To. Job. Object Set. Information. Job. Object Query. Information. Job. Object TUCN. Operating Systems. Lecture 6 No. 4

Windows processes Processes • Containers for resources • A 4 GB address space • the bottom 2 GB or 3 GB = user space • the rest = OS space • Information associated with a process • a unique process ID • a list of handles • an access token holding security information • Each process has at least one thread • the first thread is created at process creation • Win 32 API • Create. Process, Create. Process. As. User, Create. Process. With. Logon. W • Exit. Process, Terminate. Process, Get. Exit. Code. Process • Get. Current. Process. Id, Get. Environment. Strings Fall, 2007 TUCN. Operating Systems. Lecture 6 No. 5

Windows processes Threads • Describes an independent execution within a process • Threads form the basis of CPU scheduling • Information associated to a thread • • • a state (ready, running, blocked etc. ) two stacks for user and kernel execution mode a unique thread ID an access token a context used to save its state a private area for its own local variables • There are some kernel threads • perform administrative tasks • Win 32 API • Create. Thread, Create. Remote. Thread • Exit. Thread, Get. Exit. Code. Thread, Terminate. Thread • Set. Thread. Priority, Get. Current. Thread. Id Fall, 2007 TUCN. Operating Systems. Lecture 6 No. 6

Windows processes Fibers • Similar with threads, but scheduled entirely in user space • The context switch is not so expensive as with threads – does not need trap to kernel • Called lightweight threads • Each thread can have multiple fibers • The OS is not aware of thread’s fibers • Win 32 API • Convert. Thread. To. Fiber, Convert. Fiber. To. Thread • Create. Fiber, Delete. Fiber Fall, 2007 TUCN. Operating Systems. Lecture 6 No. 7

Windows processes Relationship – processes, threads and fibers Fall, 2007 TUCN. Operating Systems. Lecture 6 No. 8

Windows processes Synchronization mechanisms • Semaphores – are kernel objects • down () = Wait. For. Single. Object() • up () = Release. Semaphore() • Mutexes (locks) – are kernel objects • lock () = Wait. For. Single. Object() • unlock () = Release. Mutex() • Critical sections – similar to mutexes, but local to a process • Enter. Critical. Section() • Exit. Critical. Section() • Events – are kernel objects • manual-reset and auto-reset events • Wait. For. Single. Object() • Set. Event(), Reset. Events(), Pulse. Event() • All of them work on threads, not processes Fall, 2007 TUCN. Operating Systems. Lecture 6 No. 9

Windows processes A list of API Calls Fall, 2007 TUCN. Operating Systems. Lecture 6 No. 10

Windows processes API Calls Examples (1) Fall, 2007 TUCN. Operating Systems. Lecture 6 No. 11

Windows processes API Calls Examples (2) Fall, 2007 TUCN. Operating Systems. Lecture 6 No. 12

Windows processes Scheduling • There is no central scheduling thread • Windows 2000 is fully preemptive • thread switches can occur at any time • A thread runs (in kernel mode) the scheduler code when: • it blocks on a semaphore, mutex, I/O event etc. • it signals an object • its quantum expires (usually 20 msec) • The scheduler is also called when • an I/O operation completes • a timed wait expires Fall, 2007 TUCN. Operating Systems. Lecture 6 No. 13

Windows processes Scheduling algorithm (1) • Set the process (all threads) priority class • Set. Priority. Class() • the allowed values: real time, high, above normal, below normal, and idle • Set the relative priority of a thread within its own process • Set. Thread. Priority() • the allowed values: time critical, highest, above normal, below normal, lowest, and idle • The system has 32 priorities • the 42 possible priority classes are mapped onto the 32 system priories • real time (16 -31), user (1 -15), zero (0), and idle (-1) • Works with threads not processes and every thread has associated • base priority • current priority ( >= base priority ) Fall, 2007 TUCN. Operating Systems. Lecture 6 No. 14

Windows processes Scheduling algorithm (2) • The highest priority thread is chosen • Real-time priorities are fixed • User priorities are dynamic • A process get a boost when it is woken up because – an I/O operation completes » the amount of boost: 1 for disk, 2 for serial line, 6 for keyboard , 8 for sound card etc. – A semaphore is signaled (up) » the amount of boost is 2 • When a process consumes its entire quantum, it looses a point from its priority Fall, 2007 TUCN. Operating Systems. Lecture 6 No. 15

Windows processes Mapping of scheduling priorities Fall, 2007 TUCN. Operating Systems. Lecture 6 No. 16

Windows processes Scheduling priority classes Fall, 2007 TUCN. Operating Systems. Lecture 6 No. 17

Linux processes The support • Processes • an instance of a program in execution • a collection of data structure fully describes how far the execution has progressed • Lightweight processes • can share some resources (memory, file descriptors, etc. ) • Linux support for multithreading • Threads • many independent execution flows in the same process • examples of POSIX-compliant pthread libraries » Linux. Threads » Next Generation. Posix Threading Package (NGPT) Fall, 2007 TUCN. Operating Systems. Lecture 6 No. 18

Linux processes Processes description • Each process has a unique identifier (pid) • returned by getpid() • the maximum PID number allowed on Linux is 32, 767 • A process is created by another process • using the fork() system call => parent-child relationship • the child process is a copy of the parent, but with its own identity and resources • group of processes • Processes can communicate or cooperate • pipes, signals, shared memory, message queues, semaphores • Background processes = daemons Fall, 2007 TUCN. Operating Systems. Lecture 6 No. 19

Linux processes Processes information – ps command (1) Fall, 2007 TUCN. Operating Systems. Lecture 6 No. 20

Linux processes Processes information – ps command (2) Fall, 2007 TUCN. Operating Systems. Lecture 6 No. 21

Linux processes System calls for processes Fall, 2007 TUCN. Operating Systems. Lecture 6 No. 22

Linux processes The way the shell works Fall, 2007 TUCN. Operating Systems. Lecture 6 No. 23

Linux processes Process descriptor (1) • A data structure containing all the information related to a process • • • Process state Flags Scheduling information (priority etc. ) File descriptors Pointers to the allocated memory areas • Each process, even lightweight processes, has its own process descriptor Fall, 2007 TUCN. Operating Systems. Lecture 6 No. 24

Linux processes Process descriptor (2) Fall, 2007 TUCN. Operating Systems. Lecture 6 No. 25

Linux processes Process state • Running • currently executed on a CPU or waiting to be executed • Interruptible • suspended (sleeping) until some conditions become true • Uninterruptible • similar with the one above, but not responsive to signals • Stopped • process execution has been stopped • Zombie • process execution is terminated, but the parent process has not issued a wait() for the terminated child process Fall, 2007 TUCN. Operating Systems. Lecture 6 No. 26

Linux processes Process Usage Limits • Maximum CPU time • signal SIGXCPU sent when limit exceeded • Maximum file size allowed • signal SIGXFSZ sent when limit exceeded • • Maximum heap size Maximum number of processes a user can own Maximum number of open files Maximum size of process address space Fall, 2007 TUCN. Operating Systems. Lecture 6 No. 27

Linux processes Process creation • Traditional way • Resources own by the parent are duplicated, and a copy is granted to the child • Modern kernels • copy-on-write technique • clone() system call => lightweight processes » » » address space root and working directory file descriptors table signal handlers process identifier (pid) • vfork() system call » Processes share the same address space » Parent process is blocked until the child finishes Fall, 2007 TUCN. Operating Systems. Lecture 6 No. 28

Linux processes System calls for threads Fall, 2007 TUCN. Operating Systems. Lecture 6 No. 29

Linux processes Threads implementation • Based on lightweight processes • Thread group – A collection of lightweight processes that share the same pid • The fork() semantics • only the currently executed thread is activated in the child process • atfork() function can be used • Signal handling Fall, 2007 TUCN. Operating Systems. Lecture 6 No. 30

Linux processes Scheduling policy (1) • The set of rules used to determine when and how selecting a new process to run • Linux scheduling is based on • process preemption • time-sharing • ranking processes according to their priority • The value of processes quantum • a good compromise between efficient CPU use and good system responsiveness • choose a quantum duration as long as posible, while keeping good system response time Fall, 2007 TUCN. Operating Systems. Lecture 6 No. 31

Linux processes Scheduling policy (2) • Priorities • static – real time processes (between 1 - 99) • dynamic – conventional processes » Implicitly favor I/O-bound processes over CPU-bound ones • Scheduling classes • real-time FIFO • real-time Round Robin • conventional time-sharing • Always chooses the highest priority process to be executed Fall, 2007 TUCN. Operating Systems. Lecture 6 No. 32

Linux processes Scheduling algorithm – kernel 2. 4 (1) • Divide CPU time into epochs • an epoch is the time between all runnable processes begin with a new time slice and the time all runnable processes have used up their time slices • every process has a specified quantum whose duration is computed when the epoch begins • Each process has a base quantum (base priority) • the quantum assigned by the scheduler to the process if it has exhausted its quantum in previous epoch • about 210 ms • can be modified with nice() or setpriority() system calls • Dynamic priority of conventional processes • base priority + number of ticks of CPU time left to the process before its quantum expires in the current epoch Fall, 2007 TUCN. Operating Systems. Lecture 6 No. 33

Linux processes Scheduling algorithm – kernel 2. 4 (2) • At process creation • the number of CPU ticks left to the parent are split in two halves between it and its child • Direct invocation • The current process must be blocked • Lazy invocation • quantum expired • a process with a greater priority than the current process is woken up • sched_setscheduler() or sched_yield() system call is issued Fall, 2007 TUCN. Operating Systems. Lecture 6 No. 34

Linux processes Scheduling algorithm – kernel 2. 4 (3) • At the beginning of a new epoch • quantum = quantum/2 + base_priority • give preference to I/O bound processes • never become larger then 2*(base_priority) • How good is a runnable process (goodness) if (process_policy == real_time) process_goodness = 1000 + process_static_priority; if (process_policy == time_sharing && process_quantum==0) process_goodness = 0; if (process_policy == time_sharing && process_quantum>0) if (process was the previous process || share the address space with previous process) process_goodness = process_quantum + process_base_priority + 1; else process_goodness = process_quantum + process_base_priority ; Fall, 2007 TUCN. Operating Systems. Lecture 6 No. 35

Linux processes Performance of sched. alg. – kernel 2. 4 • The algorithm does not scale well – O(n) complexity • The predefined quantum is too large for high system loads • I/O-bound processes boosting strategy is not optimal • Support for real-time application is weak Fall, 2007 TUCN. Operating Systems. Lecture 6 No. 36

Linux processes Scheduling algorithm – kernel 2. 6 (1) • O(1) complexity • use a ready queue organized as a stack of priorities queues – called priority array • use two priority arrays – Active tasks –have not yet consumed entirely their time slice in the current epoch – Expired – have consumed their time slice in the current epoch • a bitmap is used to find quickly the highest priority thread – A bit 1 indicates a non empty priority list in the priority array – O(1) complexity • a new time slice for a task is computed when it is inserted in the expired priority array • switch between epochs = switch between the two priority arrays – O(1) complexity Fall, 2007 TUCN. Operating Systems. Lecture 6 No. 37

Linux processes Scheduling algorithm – kernel 2. 6 (2) • Range of priorities – 0 ÷ 140 – Real-time (RT) tasks : 0 ÷ 99 – Normal tasks: 100 ÷ 140 • Non RT tasks – Average time slice – 100 ms – All tasks have a static priority called nice value (-20 ÷ 19) – never changed by scheduler – Dynamic priority – add to or subtract from static priority • • Reward I/O tasks Punish CPU-bound tasks Maximum priority bonus is 5 Maximum priority penalty is 5 – Sleep_avg < MAX_SLEEP_AVG • add the total sleep time • subtract the total runtime Fall, 2007 TUCN. Operating Systems. Lecture 6 No. 38

Linux processes Scheduling algorithm – kernel 2. 6 (3) • Dynamic priority bonus(p) = CURRENT_BONUS(p) - MAX_BONUS / 2; prio(p) = static_prio(p) – bonus(p); CURRENT_BONUS(p) = sleep_avg * MAX_BONUS / MAX_SLEEP_AVG; MAX_BONUS = 10; • Time slice – calculated by simply scaling a task’s static priority onto the possible time slice range and making sure a certain minimum and maximum time-slice is enforced time_slice = (MIN_TIMESLICE + ((MAX_TIMESLICE - MIN_TIMESLICE) * (MAX_PRIO - 1 - static_prio) / (MAX_USER_PRIO - 1))) Fall, 2007 TUCN. Operating Systems. Lecture 6 No. 39

Linux processes Scheduling related system calls System call Description nice() Change the priority of a conventional process. getpriority() Get the max. priority of a group of conventional processes. setpriority() Set the priority of a group of conventional processes. sched_getscheduler() Get the scheduling policy of a process. sched_setscheduler() Set the scheduling policy and priority of a process. sched_getparam() Get the scheduling priority of a process. sched_setparam() Set the priority of a process. sched_yield() Relinquish the processor voluntarily without blocking. sched_get_priority_min() Get the minimum priority value for a policy. sched_get_priority_max() Get the maximum priority value for a policy. sched_rr_get_interval() Fall, 2007 Get the time quantum value for Round Robin policy. TUCN. Operating Systems. Lecture 6 No. 40

Bibliography [Tann 01] Andrew Tannenbaum, “Modern Operating Systems”, second edition, Prentice Hall, 2001, pgs. 690 – 708, pgs. 796 – 809. [BC 01] D. Bovet, M. Cesati, “Understanding Linux Kernel”, O’Reilly, 2001, pgs. 65 – 96, pgs. 277 – 298. [JA 05] Josh Aas, “Understanding the Linux 2. 6 CPU Scheduler”, Silicon Graphics, February 2005. Fall, 2007 TUCN. Operating Systems. Lecture 6 No. 41