Introduction to Operating Systems CS2301 System Programming for

Introduction to Operating Systems CS-2301, System Programming for Non-Majors (Slides include materials from The C Programming Language, 2 nd edition, by Kernighan and Ritchie and from C: How to Program, 5 th and 6 th editions, by Deitel and Deitel) CS-2301, B-Term 2009 Introduction to Operating Systems 1

Why an Intro to Operating Systems? • This is a System Programming Course • For people who are not CS majors • (Nearly) every programming task in real-life includes working with an OS • Inevitably will have to deal with principle OS features • Even if not knowledgeable in their designs CS-2301, B-Term 2009 Introduction to Operating Systems 2

Class Discussion What is an Operating System? (Laptops closed, please!) CS-2301, B-Term 2009 Introduction to Operating Systems 3

What Operating Systems have you Used? (Other than Windows, Linux, Mac-OS, Unix) CS-2301, B-Term 2009 Introduction to Operating Systems 4

de o c f so 9 line 8 – 10 x) u n S. 0 i 1 L O = e d e an tim l a Larg indows e ar r o f (W de o c f Characteristics 107 line o s • sm. Functions aller tem s y s – Large, complex. Sset – Creates abstractions omeof What is an Operating System? • programs – Long-lived, evolving – Worked on by many people for many years CS-2301, B-Term 2009 Introduction to Operating Systems – Multiplexes concurrent activities – Manages resources – Mediates access to hardware devices – Provides a variety of services to users and applications – … 5

Definition – Abstraction • The distillation of a complex mechanism into a simple, conceptual model he t n i n otio n ns l s i o i h t d t in C func ode a y h tion n of le m d a lre func lectio simp a ve n of col ng a a h itio s a ti e a i e n r W efi ion r c d act fo r a t t s a Ab nd d a • User of abstraction does not need to worry about details • Implementer of abstraction does not need to worry about how user will use it • within limits CS-2301, B-Term 2009 Introduction to Operating Systems 6

What is an Operating System? • Characteristics • Functions Large, complex set of W– hat ope hav programs rati e we ng s cou– Long-lived, rse used yste evolving ? alre m se rvic – Worked onadybyi many n th es people for many iyears s CS-2301, B-Term 2009 Introduction to Operating Systems – Creates abstractions – Multiplexes concurrent activities – Manages resources – Mediates access to hardware devices – Provides a variety of services to users and applications – … 7

Operating Systems • Typically – – Long-lived Frequently extended and updated Worked on by many developers Used and, maybe, abused by a variety of users with varying expertise and expectations • Essential to create an acceptable computing environment to create and execute other programs that achieve business or personal goals CS-2301, B-Term 2009 Introduction to Operating Systems 8

Kinds of operating systems • • • Mainframe Operating Systems Server Operating Systems Multiprocessor Operating Systems Personal Computer Operating Systems Mobile Phone Operating Systems Handheld Computer Operating Systems Embedded Operating Systems Sensor Node Operating Systems Real-time Operating Systems Smart-card Operating Systems … CS-2301, B-Term 2009 Introduction to Operating Systems E. g. : – Windows Linux Unix Important new category 9

Some operating systems you may (or may not) have heard about • • • z/OS VMS/Open VMS Vx. Works RT Linux QNX Neutrino e. COS Brick. OS/Le. Jos Tiny. OS Arduino … CS-2301, B-Term 2009 • • i. Phone OS Android Symbian Blackberry OS Palm. OS/Garnet Windows Mobile … Introduction to Operating Systems 10

OS and Hardware • OS mediates programs’ access to hardware – Computation – CPU – Storage – volatile (memory) and persistent (disk) – Networks – NIC, protocols – I/O devices – sound cards, keyboards, displays CS-2301, B-Term 2009 Introduction to Operating Systems 11

Four Fundamental Abstractions • Processes & threads • Multiplexing of processor(s) to create the illusion of many of them • Virtual memory • Multiplexing of physical memory and disk blocks to create illusion of own memory per process • Files – i. e. , persistent storage • Organizing principles about long-term data storage • Sockets & Connections • Organizing principles about network communication CS-2301, B-Term 2009 Introduction to Operating Systems 12

Four Fundamental Abstractions • Processes & threads • Multiplexing of processor(s) to create the illusion of many of them • Virtual memory • Multiplexing of physical memory and disk blocks to create illusion of own memory per process • Files – i. e. , persistent storage • Organizing principles about long-term data storage • Sockets & Connections • Organizing principles about network communication CS-2301, B-Term 2009 Introduction to Operating Systems 13

Definition – Process • A particular execution of a program • Different from all other executions of that program • Different from executions of other programs • The OS uses one or more CPUs to make it seem like each process has its own CPU • Can execute at same time! • Uses interrupts to manage and enforce multiplexing of CPU CS-2301, B-Term 2009 Introduction to Operating Systems 14

Definition – Interrupt • A mechanism by which the processor suspends execution of the current, running program and gives control to the OS • OS saves the state of the interrupted program so that it can be restarted later • OS then takes appropriate action CS-2301, B-Term 2009 Introduction to Operating Systems 15

Why Processes? • Enables programmers – to completely disengage from issues of concurrent execution of independent programs – to build applications with more than one concurrent activity • Enables independent applications to share a computing system – Safely! CS-2301, B-Term 2009 Introduction to Operating Systems 16

Why Processes (continued)? • Exploit modern processors – Capable of executing multiple, simultaneous, program executions – Interleaved at instruction level or even memory access level • Moore’s Law: – – wer o p to ue d n o Integrated circuit componentslimshrink in itati ern on. d o 50% every 18 months M issipati d – Double in speed every 18 months CS-2301, B-Term 2009 Introduction to Operating Systems 17 size by

Resources Assigned to a Process • Memory • Virtual or real • Processor time • Priorities • Deadlines for real-time activities • Privileges • Security, authentication, etc. • Files and file space • For long-term storage, temporary storage • Devices • For input and output activity, sensors, etc. CS-2301, B-Term 2009 Introduction to Operating Systems 18

Resources (continued) • Managed by OS • Protection and isolation from other processes • Allocation according to defined policies • Enforcement of limits, etc. • … CS-2301, B-Term 2009 Introduction to Operating Systems 19

Shell / Command Prompt • Linux Shell is a process • Windows Command Prompt is a process • Created when • you log on or connect to system (e. g. , via Pu. TTY) • Open Command Prompt, konsole, xterm , etc. , window – – Reads what you type (and displays it in your window) Interprets lines as commands and arguments Creates a process for each command, passes args (Typically) waits for process to complete before interpreting next line CS-2301, B-Term 2009 Introduction to Operating Systems 20

Window Manager • Window Manager is a process • Tracks mouse movements, key clicks, menu actions • “Open” an application means … • Create process for that application; give it a window • “Open” a document means … • If application is not open, create process for it • Pass document as argument to application CS-2301, B-Term 2009 Introduction to Operating Systems 21

Creating and Deleting Processes A process is created … – … at system boot time • The first process • Built-in processes – e. g. , in embedded systems OR – … by another process • Possibly in response to an action by a (human) user A process is deleted … – When its program exits – By another process – killed or paused (by debugger) – When system crashes or shuts down CS-2301, B-Term 2009 Introduction to Operating Systems 22

Questions about Processes? CS-2301, B-Term 2009 Introduction to Operating Systems 23

Four Fundamental Abstractions • Processes & threads • Multiplexing of processor(s) to create the illusion of many of them • Virtual memory • Multiplexing of physical memory and disk blocks to create illusion of separate memory per process • Files – i. e. , persistent storage • Organizing principles about long-term data storage • Sockets & Connections • Organizing principles about network communication CS-2301, B-Term 2009 Introduction to Operating Systems 24

Virtual Memory • Definition: – the illusion that a process has its own, independent memory • (Often) more memory than machine has installed • May be implemented using interrupts, pages, and disk blocks • Swapping fast enough so process is unaware • May be implemented by partitioning • Swapping not necessary for real-time activities CS-2301, B-Term 2009 Introduction to Operating Systems 25

Independence of Virtual Memories • A process cannot even see the virtual memory of another process • A process cannot even see the memory used by the OS • I. e. , no possible pointer value can point to something in a different virtual memory • Separate, parallel universes • Except where explicitly linked together CS-2301, B-Term 2009 Introduction to Operating Systems 26

Typical Virtual Memory for Process (Windows & Linux) 0 x. FFFF stack (dynamically allocated) SP address space heap (dynamically allocated) static data 0 x 0000 CS-2301, B-Term 2009 program code (text) Introduction to Operating Systems PC 27

Eve Ind ry p Typical Virtual Memory for Process epe n roce ss h (Windows & Linux) as o den ne o t of 0 x. FFFF f th all o ese. ther s. address space stack (dynamically allocated) SP heap (dynamically allocated) static data 0 x 0000 CS-2301, B-Term 2009 program code (text) Introduction to Operating Systems PC 28

Virtual Memory (continued) • Typical virtual memory size – 4 GBytes • Per process — even in a 1 GByte computer! • “Inactive” pages are not resident in RAM • Reference results in interrupt called a page fault • OS brings in page from disk, (maybe) swaps one out • Unused pages not filled in by OS • Dereferencing pointer results in Segmentation Fault • New pages created by OS as needed • When stack or heap grows or programs are loaded CS-2301, B-Term 2009 Introduction to Operating Systems 29

Virtual Memory in Embedded Systems • Implemented by partitioning RAM • No swapping in or out • May not even have a disk! CS-2301, B-Term 2009 Introduction to Operating Systems Also in smart-phone operating systems 30

Virtual Memories in Embedded System 0 x 0000 FFFF stack Process 3 stack Physical Process 2 memory stack Process 1 OS Kernel 0 x 0000 CS-2301, B-Term 2009 Introduction to Operating Systems 31

Questions about Processes or Virtual Memory? Not in Kernighan & Ritchie CS-2301, B-Term 2009 Introduction to Operating Systems 32

Threads • A refinement of concept of process • Short for “thread of control” • Concurrent execution of a function within the context of a process • Needs own stack in order to call other functions • Shares heap, static data, and code with other threads of same process • Reason • Application may need to manage concurrency of its own computation with external events CS-2301, B-Term 2009 Introduction to Operating Systems 33

Virtual Memory for Multiple Threads 0 x. FFFF thread 1 stack SP (T 1) thread 2 stack SP (T 2) thread 3 stack SP (T 3) Virtual address space heap static data 0 x 0000 CS-2301, B-Term 2009 PC (T 2) PC (T 1) PC (T 3) code (text) Introduction to Operating Systems 34

Why Threads? • To enable development of applications with concurrent activities inside them • Need to share data (difficult with separate processes) • Examples • • • Web server over common data pages Transaction processor over common data base Applications within a mobile phone or PDA Applications with different speeds of devices … CS-2301, B-Term 2009 Introduction to Operating Systems 35

Thread Interface – POSIX standard #include <pthread. h> • int pthread_create(pthread_t *thread, const pthread_attr_t *attr, void*(*start_routine) (void), void *arg) – creates a new thread of control – new thread begins executing at start_routine • pthread_exit(void *value_ptr) – terminates the calling thread • pthread_join(pthread_t thread, void **value_ptr) – blocks the calling thread until the thread specified terminates • pthread_t pthread_self() – Returns the calling thread's identifier CS-2301, B-Term 2009 Introduction to Operating Systems 36

– n o cti R n fu & a to of K r te 5. 7 n i Po see § Thread Interface – POSIX standard #include <pthread. h> • int pthread_create(pthread_t *thread, const pthread_attr_t *attr, void*(*start_routine) (void), void *arg) – creates a new thread of control – new thread begins executing at start_routine • pthread_exit(void *value_ptr) – terminates the calling thread • pthread_join(pthread_t thread, void **value_ptr) – blocks the calling thread until the thread specified terminates • pthread_t pthread_self() – Returns the calling thread's identifier CS-2301, B-Term 2009 Introduction to Operating Systems 37

Thread Interface – POSIX standard #include <pthread. h>ents at h t to um n g r • int pthread_create(pthread_t *thread, const A nctio fu void*(*start_routine) (void), pthread_attr_t *attr, void *arg) – creates a new thread of control – new thread begins executing at start_routine • pthread_exit(void *value_ptr) – terminates the calling thread • pthread_join(pthread_t thread, void **value_ptr) – blocks the calling thread until the thread specified terminates • pthread_t pthread_self() – Returns the calling thread's identifier CS-2301, B-Term 2009 Introduction to Operating Systems 38

Thread Example pthread_create(tp, &f, &args) f main function g main function f h main function f pthread_join(tp) CS-2301, B-Term 2009 Introduction to Operating Systems 39

Thread Example pthread_create(tp, &f, &args) Two threads run at the same time f main function g main function f h main function f pthread_join(tp) CS-2301, B-Term 2009 Introduction to Operating Systems 40

Why Threads? • To enable development of applications with concurrent activities inside them • Need to share data (difficult with separate virtual memories) • Examples • • • Web server over common data pages Transaction processor over common data base Applications within a mobile phone or PDA Applications with different speeds of devices … CS-2301, B-Term 2009 Introduction to Operating Systems 41

Example Thread Application • One or more threads to get data from sensor • 1000 times per second; cannot afford to miss a reading • Places data on queue • Another thread processes and displays data • Removes and processes each data item from queue • Displays system state on control panel • User thread • Allows operator to adjust system parameters • While other threads are still running CS-2301, B-Term 2009 Introduction to Operating Systems 42

Additional Functions & Data Required • pthread_mutex_t — mutual exclusion lock – Enables a thread to “lock” some data so that other threads do not touch in mid-operation • pthread_cond_t — condition variable – Enables a thread to wait for an event to happen – Signaled by another thread • See man pages for details CS-2301, B-Term 2009 Introduction to Operating Systems 43

Questions about Threads? Not in Kernighan & Ritchie CS-2301, B-Term 2009 Introduction to Operating Systems 44

Four fundamental Abstractions • Processes & threads • Multiplexing of processor(s) to create the illusion of many of them • Virtual memory • Multiplexing of physical memory and disk blocks to create illusion of own memory per process • Files & persistent storage • Organizing principles about long-term data storage • Sockets & connections • Organizing principles about network communication CS-2301, B-Term 2009 Introduction to Operating Systems 45

Definition – File • A (potentially) large amount of information or data that lives a (potentially) very long time • Often much larger than the memory of the computer • Often much longer than any computation • Sometimes longer than life of machine itself • (Usually) organized as a linear array of bytes or blocks • Internal structure is imposed by application • (Occasionally) blocks may be variable length • (Often) requiring concurrent access by multiple threads or processes • Even by processes on different machines! CS-2301, B-Term 2009 Introduction to Operating Systems 46

Implementations of Files • Usually on disks (or devices that mimic disks) • Magnetic – hard drive or floppy • Optical – CD, DVD • Flash drives – electronic memory, organized as disks • Requirement • Preserve data contents during power-off or disasters • Directory / Folder • Special kind of file that contains links pointing to other files • Associates names with files CS-2301, B-Term 2009 Introduction to Operating Systems 47

sed u o als per s em e, pa t s y ed s p h a r t c e n d c Ol gneti of pu ma trays Usually on disks (or devices that mimic disks) , tape , etc. s • Magnetic – hard drive or floppy card • Optical – CD, DVD • Flash drives – electronic memory, organized as disks Implementations of Files • • Requirement • Preserve data contents during power-off or disasters • Directory / Folder • Special kind of file that contains links pointing to other files • Associates names with files CS-2301, B-Term 2009 Introduction to Operating Systems 48

File Access in C • See Kernighan & Ritchie, Chapter 8 • Raw file access • Without simplifying stream functions – e. g. , – scanf, fscanf, printf, fgetc, etc. • read and write raw disk blocks • Seek to a file position – lseek, fseek — sets file pointer to specified location – Subsequent read, write, etc. , start there – ftell – returns file pointer CS-2301, B-Term 2009 Introduction to Operating Systems 49

Organizations of Files • Contiguous • Blocks stored contiguously on storage medium • E. g. , CD, DVD, some large database systems • Access time to any block is O(1) • Linked • Blocks linked together – File Allocation Table (FAT) • Access time is O(n) • Indexed • Blocks accessed via tree of index blocks (i-nodes) • Access time is O(log n) • However, base of logarithm may be very large (>100) CS-2301, B-Term 2009 Introduction to Operating Systems 50

Organizations of Files • Contiguous • Blocks stored contiguously on storage medium • E. g. , CD, DVD, some large database systems • Access time to any block is O(1) • Linked • ile es • Blocks linked together – File Allocation Table (FAT) f x iv nu i r d L nd • Access time is O(n) ard a h S n NTF stems o sy Indexed • Blocks accessed via tree of index blocks (i-nodes) • Access time is O(log n) • However, base of logarithm may be very large (>100) CS-2301, B-Term 2009 Introduction to Operating Systems 51

Organizations of Files • Contiguous • • Blocks stored contiguously on storage medium s, p i h • E. g. , CD, DVD, some large database systems a c pies r e cam s, flop l • Access time to any block is O(1) a ic p ive y r T d sh a l f Linked • Blocks linked together – File Allocation Table (FAT) • Access time is O(n) • Indexed • Blocks accessed via tree of index blocks (i-nodes) • Access time is O(log n) • However, base of logarithm may be very large (>100) CS-2301, B-Term 2009 Introduction to Operating Systems 52

Questions about Files? CS-2301, B-Term 2009 Introduction to Operating Systems 53

Four fundamental Abstractions • Processes & threads • Multiplexing of processor(s) to create the illusion of many of them • Virtual memory • Multiplexing of physical memory and disk blocks to create illusion of own memory per process • Files & persistent storage • Organizing principles about long-term data storage • Sockets & connections • Organizing principles about network communication CS-2301, B-Term 2009 Introduction to Operating Systems 54

Sockets and Connections • Connection: – a serial conversation over a network between two end points • e. g. , processes, threads, tasks on different computers – organized as a sequence of messages or datagrams – distinct from all other connections • Socket: – An end point of a connection – An abstraction that allows a process to send or receive only the information of that connection – Multiplexed on network with all other connections CS-2301, B-Term 2009 Introduction to Operating Systems 55

Sockets and Connections • Defer to another course – CS-4513, Distributed Systems – CS-4514, Computer Networks CS-2301, B-Term 2009 Introduction to Operating Systems 56

General Questions? CS-2301, B-Term 2009 Introduction to Operating Systems 57