TDC 561 Network Programming Review UNIX Architecture and

TDC 561 Network Programming Review UNIX Architecture and Programming Camelia Zlatea, Ph. D Email: czlatea@cs. depaul. edu Distributed Systems Frameworks (DS 520 -95 -103) DISC’ 99

References § W. Richard Stevens, Network Programming, Vol, I, 2 nd Ed, Prentice Hall PTR, NJ, 1998. § John Shapley Gray, Interprocess Communications in UNIX -- The Nooks and Crannies Prentice Hall PTR, NJ, 1998. § HP-UX man pages http: //docs. hp. com/hpux/onlinedocs/B 2355 -90682. html (system calls) Network Programming (TDC 561) Winter 2003 Page 2

Overview UNIX Architecture UNIX processes, threads UNIX program development System calls – fork, exec, etc. Distributed Systems Frameworks (DS 520 -95 -103) DISC’ 99

UNIX Architectural Overview Applications Shell’s Network Applications. DBMS Executable Programs pipe, filters Utilities Commands System Calls Interface Kernel UNIX Hardware Architecture Network Programming (TDC 561) Winter 2003 Page 4

UNIX Features § § § § Portability Multi-process architecture (multitasking) Multi-user capability Ability to initiate asynchronous processes A hierarchical file system Device independent I/O operations User interface: Shell; selectable per user basis Network Programming (TDC 561) Winter 2003 Page 5

UNIX Standards § System V Interface Definition (SVID), AT&T § Portable Operating System Interface for Computer Environments (POSIX), based on SVID, IEEE § ANSI C, American National Standard Institute § ANSI/ISO C++ Standard (draft) Network Programming (TDC 561) Winter 2003 Page 6

UNIX Implementations § Solaris - Sun Microsystems – Sun. OS (later called Solaris), Solaris 2. x based on SVR 4 § § HP-UX, Hewlett-Packard, SVR 2 Linux, Linus Torvalds, free distribution, PC-based AIX, IBM, similar to SVR 4 IRIX, Silicon Graphics, SVR 4 Network Programming (TDC 561) Winter 2003 Page 7

UNIX Architectural Overview Applications Shell’s Network DBMS Utilities Applications Executable Commands pipe, filters Programs System Calls Interface Kernel UNIX Hardware Architecture Network Programming (TDC 561) Winter 2003 Page 8

UNIX Features § § § § Portability Multi-process architecture (multitasking) Multi-user capability Ability to initiate asynchronous processes A hierarchical file system Device independent I/O operations User interface: Shell; selectable per user basis Network Programming (TDC 561) Winter 2003 Page 9

Multi-process/Multi-user architecture § Virtual Machine – timesharing OS – process, process quantum, process states § Kernel, base OS – manages all HW dependent functions – users have no direct access to it § System Calls Interface – service routine performing user requests Network Programming (TDC 561) Winter 2003 Page 10

UNIX Processes § A process may create sub-processes – fork(); § A process may terminate – exit(); § A process may put itself to sleep temporarily – sleep(20); – pause(); – wait(); § Processes – synchronization mechanisms – communication mechanisms Network Programming (TDC 561) Winter 2003 Page 11

UNIX Threads § Multiple Processes - concurrency at OS level § Multiple Threads - concurrency at process level – thread = flow of control in a process – multiple threads (stream of instructions) are executed within the same process – threads share code & data (address space) – threads have their own execution stack, PC, register set and states – context switches are avoided – efficient mapping on multi-processor machines Network Programming (TDC 561) Winter 2003 Page 12

UNIX Kernel - model User Processes Space System Call Interface (Library Routines) Process Memory File System I/O Services Mgmt. Scheduler Device Drivers I/O Buffers Kernel Space Hardware Network Programming (TDC 561) Winter 2003 Page 13

Apps. /Utilities System Calls UNIX KERNEL Hardware Interrupts Hardware Network Programming (TDC 561) Winter 2003 Page 14

UNIX Kernel - low level § § § Process representation, scheduling, dispatching Memory allocation and de-allocation Interrupt handling Low level device control Disk Mgmt. , data buffering Process synchronization and IPC Network Programming (TDC 561) Winter 2003 Page 15

Kernel - services level § Maps user-level requests with device driver actions § A user system call is translated to a call of the kernel routine, providing that requested service § Type of Services: – – process creation and termination I/O services UNIX file system services terminal handling services Network Programming (TDC 561) Winter 2003 Page 16

Kernel - services level User program: system(“ps -e”); System calls actions: fork(); //process creation service execl(“/bin/ps”, “ps -e”); //code execution Network Programming (TDC 561) Winter 2003 Page 17

System Call Interface level § A user mode process is translated into a protected kernel mode process § Now, program can call kernel routines Network Programming (TDC 561) Winter 2003 Page 18

User Processes level § User processes running: – – shells Unix commands utilities application programs Network Programming (TDC 561) Winter 2003 Page 19

UNIX and POSIX API § UNIX API - system calls § UNIX API are called by – C library functions and – C++ standard classes Example: iostream class § Program confirms POSIX. 1? #define _POSIX_SOURCE or % CC -D_POSIX_SOURCE *. C Network Programming (TDC 561) Winter 2003 Page 20

UNIX and POSIX API § API set to perform: – determine system configuration and user information – file management – process creation and management – inter-process communication – network communication Network Programming (TDC 561) Winter 2003 Page 21

UNIX and POSIX API User Process (User Mode of Execution) an API is invoked API execution completed UNIX API’s level Kernel mode of execution API executed in protected mode Network Programming (TDC 561) Winter 2003 Page 22

UNIX and POSIX API § Context Switch from user to kernel mode – more overhead than library functions, for the same task – I/O lib. functions are buffered Network Programming (TDC 561) Winter 2003 Page 23

Some Unix Terms § TRAP INSTRUCTION – switches a system call from user to kernel mode – handles error conditions – synchronous execution – Ex: division by zero/overflow § INTERRUPT – used by peripherals to request services from the devices handles – asynchronous execution Network Programming (TDC 561) Winter 2003 Page 24

Some Unix Terms § SIGNALS software notification of an event § example: – when typing “ctrl-c” an interrupt is generated for the keyboard driver, which notifies the process by sending a signal Network Programming (TDC 561) Winter 2003 Page 25

Processes § Process - a program in execution – process - active entity – program - passive entity (binary file) § Address Space - list of memory locations from where a process reads/writes (code/data/stack) § Set of registers (PC, SP, . . . ) § Process Table - linked list of structures associates w/ processes § System Calls - interface between OS and User process Network Programming (TDC 561) Winter 2003 Page 26

Process Control Block (process attributes) § Process State – new, ready, running, blocked, terminated § Process Image Map Pointer § Process ID – assigned at process creation § Program Counter (PC) – address of next instruction to be executed § § CPU Registers (saved process context) List of Open File Descriptors I/O Devices Attached CPU Scheduling Info (priority) Network Programming (TDC 561) Winter 2003 Page 27

Process Image Map Process Table Proc. 1 Process Image Text/Code Segment Proc. n Data Segment Stack Segment Process Control Block Network Programming (TDC 561) Winter 2003 Page 28

Network Programming (TDC 561) Winter 2003 Page 29

Example: /* Display Segment Address Information */ #include <stdio. h> extern int etext, edata, end; void main(void) { printf(“etext: %6 Xt edata: %6 X t end: %6 X n”, &etext, &edata, &end); } Network Programming (TDC 561) Winter 2003 Page 30

Process States § New - process created ( Ex: fork(); ) § Ready process is waiting to be assigned to processor (inserted in ready queue) § Running - instructions are being executed § Blocked - wait for events to occur (inserted in queue) Ex: wait(); pause(); § Terminated - normal/abnormal termination (exit(); ) Network Programming (TDC 561) Winter 2003 Page 31

Process Model New wakeup Blocked/ Suspended Terminated Network Programming (TDC 561) created Ready Quantum Expired dispatch Running sleep User Mode exit System Call Return Running Interrupt Kernel Mode Interrupt Winter 2003 return Page 32

§ Context of a Process – – – process state (defined by it’s code) value of u-area values of registers the process uses contents of user and kernel stacks is associated with process image map § Context Switching – system executes a process in the context of the process – when the kernel decides to execute another process, it does context switching – kernel saves enough information such that it can later switch back to the first process and resumes its execution § Mode Switching – moving from user to kernel mode – kernel save information to return to user mode Network Programming (TDC 561) Winter 2003 Page 33

§ User mode – processes in use mode can access their own instructions and data; NOT kernel or other process’s code or data § Kernel mode – process can access system(kernel) code and data and user addresses – Kernel is part of each process – Kernel executes on behalf of the process HW Kernel Mode OS User Mode Network Programming (TDC 561) P 1 P 2 P 3 P 4 K K U U Winter 2003 Page 34

Context Switching P 1 P 2 OS Save state in PCB 1 Reload state from PCB 2 Save state in PCB 2 Reload state from PCB 1 Network Programming (TDC 561) Winter 2003 Page 35

Context Switching § Switching the CPU to another process by saving the state of the old process (PCB) and load the state of the new process (PCB) § Pure Overhead § Performance Bottleneck § Avoid Overhead of Context Switching by introducing new structures: THREADS Network Programming (TDC 561) Winter 2003 Page 36

Context Switching Multitasking Sequential Execution Network Programming (TDC 561) Winter 2003 Page 37

Compilation cc -o file. c file Man Pages man cc man sys_call man shell_cmd Network Programming (TDC 561) Winter 2003 Page 38

UNIX PROCESSES § Process - system support § Fork system call § Process states § Exec system call § Exit function § Background processes § Parent-Child Synchronization (wait) § File Sharing § Examples Network Programming (TDC 561) Winter 2003 Page 39

A fork system call: pid_t pid; pid = fork(); if (pid ==-1) {/* fork failure, no more entries in process table*/} else if (pid==0){/*child process*/} else {/* parent process */} Network Programming (TDC 561) Winter 2003 Page 40

Fork - System Call § The user process calls fork(); § Store system call parameters – arguments, return address, local variables) into the user stack. § Call corresponding kernel service – execution of a trap instruction § In kernel space save: – parameters, return address and local variables for kernel routine § Execute kernel routine § Normal Return – cleans-up kernel stack – switch from kernel to user mode Network Programming (TDC 561) Winter 2003 Page 41

Fork - System Call Successful return from system call: § parent and child processes: – – share same text segment identical data segments identical user & kernel stack segments same user structure § parent normal return – restores return address from user stack § child “pseudo return” – restores same return address as the parent from its user stack Network Programming (TDC 561) Winter 2003 Page 42

Parent Process text data stack Open Files Current Directory File Table I-node Table Child Process text data stack Open Files Current Directory Network Programming (TDC 561) Winter 2003 Page 43

Process information #include <sys/types. h> #include <unistd. h> pid_t getpid(void); /* get current process ID */ pid_t getppid(void); /* get ID of the parent of the process */ Network Programming (TDC 561) Winter 2003 Page 44

pid_t pid; static int x; x=1; pid = fork(); if (pid < 0 ) {perror(“fork failure”)} else if (pid==0) x++; else x--; printf(“process %d: x=%dn”, getpid(), x); § What value(s) are printed for variable x? Network Programming (TDC 561) Winter 2003 Page 45

#include <stdio. h> #include <sys/types. h> #include <unistd. h> void main(void) { fork(); printf(“An”); fork(); printf(“Bn”); fork(); printf(“Cn”); } § Comment on the above program output. Network Programming (TDC 561) Winter 2003 Page 46

#include <stdio. h> #include <sys/types. h> #include <unistd. h> void main(void) { int i; for (i=1; i<=3; i++) { fork(); printf(“PID=%d i=%dn”, getpid(), i); } § Comment on the above program output. Network Programming (TDC 561) Winter 2003 Page 47

#include <stdio. h> #include <sys/types. h> #include <unistd. h> void main(void) { int i; for (i=1; i<=3; i++) { if (fork()==0) break; printf(“PID=%d i=%dn”, getpid(), i); } § Comment on the above program output. Network Programming (TDC 561) Winter 2003 Page 48

#include <stdio. h> #include <sys/types. h> #include <unistd. h> void main(void) { int i; for (i=1; i<==3; i++) { if (fork()>0) break; printf(“PID=%d i=%dn”, getpid(), i); } § Comment on the above program output. Network Programming (TDC 561) Winter 2003 Page 49

Process States § Initiated - fork() § Ready-to-Run – in ready queue for CPU access § Running – process quantum § Blocked – sleep(n); /* deterministic delay */ – pause(); /* non-deterministic delay */ § Terminated – exit(int status); § Shell command: %ps [options] Network Programming (TDC 561) Winter 2003 Page 50

#include <stdio. h> #include <sys/types. h> #include <unistd. h> #include <string. h> void main(void) { static char buf[2]; if (fork()==0) strcpy(buf, ”An”); else strcpy(buf, ”Bn”); sleep(10); write(1, buf, sizeof(buf)); } § Comment on the above program execution Network Programming (TDC 561) Winter 2003 Page 51

Exec - System Call #include <unistd. h> int execl(const char *file, const char arg 0, . . , const argn); int execv(const char *file, char *argv[]); § the process executes the binary code from file § if failure return -1 § if success does not return Network Programming (TDC 561) Winter 2003 Page 52

§ File test 1. c #include <stdio. h> #include <sys/types. h> #include <unistd. h> void main(void) { int i; for (i=1; i<=3; i++) if (fork()==0) { /* process child */ execv(“/bin/ps”, ”ps -el”); } else { /* process partent */ printf(“Parent: %dn”, getpid()); } } § %cc -o test 1. c § % test 1 Network Programming (TDC 561) Winter 2003 Page 53

§ File test 2. c #include <stdio. h> #include <sys/types. h> #include <unistd. h> void main(void) { int i; for (i=1; i<=3; i++) if (fork()==0) { /* process child */ execv(“child”, ”child”); } else { /* process partent */ printf(“Parent: %dn”, getpid()); } } § %cc -o test 2. c Network Programming (TDC 561) Winter 2003 Page 54

§ File child. c #include <stdio. h> #include <sys/types. h> #include <unistd. h> void main(void) { pid_t pid; printf(“Parent: %dn”, getpid()); sleep(30); } %cc -o test 2. c %cc -o child. c %test 2 Network Programming (TDC 561) Winter 2003 Page 55

Exit - Function Call #include <stdlib. h> void exit(int status); § § status - parameter returned to parent process status=0 - normal termination status# - abnormal termination Effects: – – close all open file descriptors notifies the parent (by signal) return status info to parent if parent no longer exists then PPD: =1 adopted by process init) Network Programming (TDC 561) (is Winter 2003 Page 56

_exit - System Call #include <unistd. h> void _exit(int status); § § status - process exit status code status=0 - normal termination status#0 - abnormal termination Effects: – – – terminates a process closes all open file descriptors deallocates all process data and stack segment process becomes a zombie (it is no longer scheduled to run) init process clean up process table slot Network Programming (TDC 561) Winter 2003 Page 57

_exit - System Call #include <iostream. h> #include <unistd. h> int main() { cout << "Test for _exit" << endl; _exit(0); return 0; } % CC -o test. c; test Test for _exit % echo $status 0 Network Programming (TDC 561) Winter 2003 Page 58

Wait - System Call #include <sys/types. h> #include <sys/wait. h> pid_t wait(int *status); § returns -, 1 if failure (check errno) – no unwaited-for child process – interrupt signal § returns the child PID, is success § synchronization: – parent waits for a child to terminate – if more child-processes, waits for any Network Programming (TDC 561) Winter 2003 Page 59

Waitpid - System Call #include <sys/types. h> #include <sys/wait. h> pid_t waitpid(pid_t p, int *status, int opt); § Return value - a child process ID or -1 § Argument pid_t p - a child PID -1 waits for any child (same as wait) 0 waits for any child in same group w/ parent >0 waits for child process with this PID=p <-1 waits for any child with GID = |p| Network Programming (TDC 561) Winter 2003 Page 60

Waitpid - System Call #include <sys/types. h> #include <sys/wait. h> pid_t waitpid(pid_t p, int *status, int opt); § Options values WNOHANG - do not block if status cannot be obtained WNOWAIT - keep process in wait status WUNTRACED - wait any stopped child process Network Programming (TDC 561) Winter 2003 Page 61

Parent-Child Process synchronization #include <stdio. h> #include <sys/types. h> #include <unistd. h> #include <sys/wait. h> void main(void) { pid_t pid; int i, status; for (i=1; i<=3; i++) if (fork()==0) execv(“child”, ”child”); else printf(“Parent: %dn”, getpid()); while (pid = wait(&status) && pid!=-1) printf(“Child %d is donen”, getpid()); exit(0); } Network Programming (TDC 561) Winter 2003 Page 62

Parent-Child Process synchronization (by priorities) #include <stdio. h> #include <sys/types. h> #include <unistd. h> #include <sys/wait. h> void main(void) { pid_t pid[3], p; int i, status; for (i=0; i<=3; i++) if ((pid[i]=fork())==0) else printf(“Parent: %dn”, getpid()); execv(“child”, ”child”); for(i=0; (p=waitpid(pid[i], &status, 0)&&p!=-1; i++) if (p!=-1) && (errno==EINTR) printf(“Signal Caught: %sn”, strerrno(errno)); exit(0); } Network Programming (TDC 561) Winter 2003 Page 63

Emulate the C “system” function #include <iostream. h> #include <stdio. h> #include <stdlib. h> #include <errno. h> #include <sys/types. h> #include <sys/wait. h> #include <unistd. h> int My. System( const char *cmd) { pid_tpid; int status; switch (pid=fork()) { case -1: return -1; case 0: execl("/bin/sh", "-c", cmd, 0); perror("execl"); exit(errno); } if (waitpid(pid, &status, 0)==pid && WIFEXITED(status)) return WEXITSTATUS(status); return -1; } Network Programming (TDC 561) Winter 2003 Page 64

Emulate the C “system” function (cont. ) int main() { int char do { rc = 0; buf[256]; printf("sh> "); fflush(stdout); if (!gets(buf)) break; rc = My. System(buf); } while (!rc); return (rc); } /* C library function */ int system(const char* cmd); Network Programming (TDC 561) Winter 2003 Page 65

Processes Hierarchy Pid=0 swapper Pid=1 init Pid gtty fork - by init exec gtty Network Programming (TDC 561) Pid sh sh fork - by init exec gtty exec login exec sh Pid ls -l fork - by sh exec ls Winter 2003 Page 66

Shell Processes § Execution of a Shell command – [fork] new process – [exec] command – [wait] shell waits for child process to terminate § % ps -al Network Programming (TDC 561) Winter 2003 Page 67

Background Processes § Shell does not wait for child process to complete § Returns prompter immediately % sleep 30 & § shell: [fork] [exec] – shell child exits immediately after the second [fork], returns prompter – shell grand-child [exec] the command in background – it is adopted by process init Network Programming (TDC 561) Winter 2003 Page 68