Interprocess Communication Process Synchronization Fall 09 1 Cooperating

Interprocess Communication & Process Synchronization Fall 09 1

Cooperating Processes Independent process cannot affect or be affected by the execution of another process Cooperating process can affect or be affected by the execution of another process Advantages of process cooperation • • • – – Information sharing Computation speed-up Modularity Convenience Cooperating processes need interprocess communication (IPC) and process synchronization – 2

Communications Models Message passing 3 Shared memory

Producer-Consumer Problem A paradigm for cooperating processes Producer process produces information that is consumed by a consumer process E. g. web server and browser If shared memory is available 4 Producer write to the buffer, client read from the buffer unbounded-buffer places no practical limit on the size of the buffer bounded-buffer assumes that there is a fixed buffer size

Bounded-Buffer – Shared-Memory Solution Shared data: implemented as a circular array #define BUFFER_SIZE 10 typedef struct {. . . // information to be shared } item; out item buffer[BUFFER_SIZE]; int in = 0; int out = 0; 5 in 0

Bounded-Buffer Producer while (true) { /* Produce an item */ while (( (in + 1) % BUFFER_SIZE) == out) ; /* do nothing -- no free buffers */ buffer[in] = new. Produced. Item; in = (in + 1) % BUFFER SIZE; } out while (true) { while (in == out) ; // do nothing -- nothing to consume Consumer // remove an item from the buffer item. To. Consume = buffer[out]; out = (out + 1) % BUFFER SIZE; return item. To. Comsume; } 6 Solution is correct, but can only use BUFFER_SIZE-1 elements in

POSIX Shared Memory – Process first creates shared memory segment_id= shmget(IPC_PRIVATE, size, S_IRUSR | S_ IWUSR); shmget - allocates a shared memory segment SYNOPSIS #include <sys/ipc. h> #include <sys/shm. h> Int shmget(key_t key, int size, int shmflg) shmget() returns the identifier of the shared memory segment associated to the value of the argument key. A new shared memory segment, with size equal to the round up of size to a multiple of PAGE_SIZE, is created if key has value IPC_PRIVATE or key isn't IPC_PRIVATE, no shared memory segment is associated to key, and IPC_CREAT is asserted in shmflg (i. e. shmflg&IPC_CREAT isn't zero). 7

POSIX Shared Memory Process wanting access to that shared memory must attach to it shared_memory = (char *) shmat(id, NULL, 0); – SYNOPSIS #include <sys/types. h> #include <sys/shm. h> void * shmat(int shmid, const void *shmaddr, int shmflg); The function shmat attaches the shared memory segment identified by shmid to the address space of the calling process. The attaching address is specified by shmaddr with one of the following criteria: 8 if shmaddr is NULL, system chooses a suitable (unused) address at which to attach the segment if shmaddr isn’t NULL, ….

POSIX Shared Memory - Now the process could write to the shared memory sprintf(shared_memory, "Writing to shared memory"); When done a process must detach shared memory from its address space shmdt(shared memory); – shmdt detaches the shared memory segment located at the address specified by shmaddr from the address space of the calling process. The to-be-detached segment must be currently attached with shmaddr equal to the value returned by the its attaching shmat call. 9

Sample code studies Handout for share memory examples Two unrelated processes communicate through shared memory Access shared memory is simpler between related processes, i. e. , parent and child process is a copy of parent process, shared-memory attached to parent is also attached to child’s address space

Case studies: IPC with shared memory Calculating and displaying Fibonacci sequence(P. 145, 3. 19) Use command line argument to specify how many elements to calculate & output Create a child process to calculate the sequence elements Parent process displays the result to standard output

Producer/Consumer as processes They have their different address space Use shared memory to implement the circular buffer struct circular_buffer{ item buffer[BUFFER_SIZE]; int in; int out; }; circular_buffer * shared_memory; segment_id = shmget(key, sizeof(circular_buffer), S_IRUSER|S_IWUSER); shared_memory = (circular_buffer *)shmat (segment_id, NULL, 0); …

Message Passing • Message system – processes communicate with each other without resorting to shared variables • • Mechanism for processes to communicate and to synchronize their actions If proceses P and Q wish to communicate, they need to: – – establish a communication link between them exchange messages via send/receive – – – send(message) – message size fixed or variable receive(message) Implementation of communication link – – 13 physical link: how is message passing link is implemented ? e. g. , shared memory, hardware bus, network connection … logical link: focus on logical properties of the link

Communication link: Design Questions • How are links established? • • Direct Communication: process P send message to Q Indirect Communication: through a mailbox, message queue, … Can a link be associated with more than two processes? How many links can there be between every pair of communicating processes? What is the capacity of a link? Is the size of a message that the link can accommodate fixed or variable? Is a link unidirectional or bi-directional? 14

Direct Communication Processes must name each other explicitly: send (P, message) – send a message to process P receive(Q, message) – receive a message from process Q Properties of direct communication 15 Links are established automatically A link is associated with exactly one pair of communicating processes Between each pair there exists exactly one link Link may be unidirectional, but is usually bidirectional

Indirect Communication Messages are directed and received from mailboxes (also referred to as ports), or queues Properties of communication link 16 Each mailbox has a unique id Processes can communicate only if they share a mailbox Link established only if processes share a common mailbox A link may be associated with many processes Each pair of processes may share several communication links Link may be unidirectional or bi-directional

Indirect Communication Operations create a new mailbox send and receive messages through mailbox destroy a mailbox Primitives are defined as: send(A, message) – send a message to mailbox A receive(A, message) – receive a message from mailbox A 17

Indirect Communication • Mailbox sharing – – – • P 1, P 2, and P 3 share mailbox A P 1, sends; P 2 and P 3 receive Who gets the message, P 2, P 3 or both? Solutions – – – 18 Allow a link to be associated with at most two processes Allow only one process at a time to execute a receive operation Allow the system to select arbitrarily the receiver. Sender is notified who the receiver was.

Synchronization Message passing may be either blocking or non -blocking Blocking is considered synchronous Blocking send has the sender block until message is received Blocking receive has the receiver block until a message is available Blocking message passing is a way to synchronize two processes Non-blocking is considered asynchronous 19 Non-blocking send has the sender send message and continue Non-blocking receive return immediately, with a valid message or null (if no message is available)

Asynchrony (Synchrony): from Wikipedia The state of not being synchronized Digital logic and physical layer of communication asynchronous process does not require a clock signal Data link layer of communication asynchrony is synonym of statistical multiplexing: information transmission may or may not start immediately as requested by sender, additional delay being caused by medium congestion Programming asynchronous events are those occurring independently of main program flow E. g. Unix signal is asynchronous event to a process asynchronous actions are actions executed in a non-blocking scheme, allowing main program flow to continue processing Asynchronous I/O: Interrupt I/O

Buffering Queue of messages attached to the link; implemented in one of three ways 1. Zero capacity – 0 messages Sender must wait for receiver (rendezvous) 2. Bounded capacity – finite length of n messages Sender must wait if link full 3. Unbounded capacity – infinite length Sender never waits 21

Code studies: POSIX Message Queue POSIX messaging: Two or more processes can exchange information via access to a common system message queue. Processes must share a common key to access the queue Initialize queue: msgget() Send/receive message: msgsnd, msgrcv() functions, can be blocking or non-blocking Handout

Pipes first-in-first-out queue, written by one process and read by another. Design choices Unidirectional or bidirectional communication? If bidirectional, is it half duplex (data travel in one way at a time) or full duplex? Must the processes be related to use the pipe? Must the processes reside in same machine ?

Unix Unnamed Pipe Unidirectional: info. flow from one end to another end Allow parent-child processes to communicate with each other int pipe(int fd[2]); // a system call you pass it an array of two integers. on success, zero is returned, fd contains two file descriptors, fd[0], fd[1] On error, -1 is returned, and errno is set appropriately. Routine perror() produces a message on standard error output, describing last error encountered during a system call or library function. fd[0]: reading end fd[1]: writing end

Unix Unnamed Pipe int fd[2]; if (pipe(fd)!=0){ fd[1]: writing end fd[0]: reading end perror (“pipe”); exit 1; } else { // on success, fd contains two file descriptors, fd[0], fd[1] // fd[0] is for reading, fd[1] is for writing // Anything that is written on fd[1] can be read from fd[0]. … } a file descriptor is an index for an entry in a kernel data structure containing details of all open files. Pipes are treated as files You can use system calls read(), write(), close() … on pipes

Simple example #include <stdio. h> main() { int pipefd[2]; int i; char s[1000]; char *s 2; if (pipefd) < 0) { write() to a write end of a pipe: asks perror("pipe"); operating system to hold those bytes in a exit(1); buffer until some process requests for } them by performing a read() on the read s 2 = “Hello World!"; end of the pipe. write(pipefd[1], s 2, strlen(s 2)); i = read(pipefd[0], s, 1000); s[i] = ''; printf("Read %d bytes from the pipe: '%s'n", i, s); }

Visualize pipes program Code, Globals stacks heap File descriptors 0 1 2 pipefd[0] pipefd[1] Operating system buffer Hello World!

#include <stdio. h> #include <unistd. h> #include <sys/types. h> #include <string. h> #define BUFFER_SIZE 25 #define READ_END 0 #define WRITE_END 1 int main(void) { char write_msg[BUFFER_SIZE] = "Greetings"; char read_msg[BUFFER_SIZE]; pid_t pid; int fd[2]; /** create the pipe */ if (pipe(fd) == -1) { fprintf(stderr, "Pipe failed"); return 1; } /** now fork a child process */ pid = fork(); if (pid < 0) { fprintf(stderr, "Fork failed"); return 1; }

if (pid > 0) { /* parent process */ /* close the unused end of the pipe */ close(fd[READ_END]); /* write to the pipe */ write(fd[WRITE_END], write_msg, strlen(write_msg)+1); /* close the write end of the pipe */ close(fd[WRITE_END]); } else { /* child process */ /* close the unused end of the pipe */ close(fd[WRITE_END]); /* read from the pipe */ read(fd[READ_END], read_msg, BUFFER_SIZE); printf("child read %sn", read_msg); } } /* close the write end of the pipe */ close(fd[READ_END]); return 0;

dup, dup 2 system calls How to implement command line pipeline, or standard input/output redirection? dup and dup 2 create a copy of file descriptor oldfd. Old and new descriptors may be used interchangeably, and they share locks, file position pointers and flags. int dup(int oldfd); int dup 2(int oldfd, int newfd); dup uses lowest-numbered unused descriptor for new descriptor dup 2 makes newfd be copy of oldfd, closing newfd first if necessary. 30 CSRU 3130: Unix System Programming

Code studies: Command Line Pipeline

Communications for Distributed Systems (Computing Environment) Client-Server Computing: computing server, file server, web server …. Peer-to-peer systems: all nodes are both client and server; or sometimes acts as client, sometimes as server Building distributed systems involves communications between processes running on different systems 32 Sockets Remote Procedure Calls Remote Method Invocation (Java)

Sockets A socket is defined as an endpoint for communication Identified by IP address and port With IP address, networking protocol routes the packet to the destined host Different servers listens for requests on well known port # E. g. the socket 161. 25. 19. 8: 1625 refers to port 1625 on host 161. 25. 19. 8 Communication consists between a pair of sockets 33 TCP socket: reliable stream based communication UDP socket: unreliable datagram communication

Socket Communication 34

Remote Procedure Calls • • Remote procedure call (RPC) abstracts procedure calls between processes on networked systems Stubs – client-side proxy for the actual procedure on the server • • • Client-side stub locates server and marshalls parameters Parameter marshalling: packaging parameters into a form to be transmitted over network Server-side stub (daemon) receives this message, unpacks marshalled parameters, and performs requested procedure on server • 35 Value is returned (if needed) to client in similar way

Marshalling Parameters 36

Execution of RPC 37

Examples of IPC Systems – Windows XP • Message-passing centric via local procedure call (LPC) facility – – – Only works between processes on the same system Uses ports (like mailboxes) to establish and maintain communication channels Communication works as follows: • • 38 The client opens a handle to the subsystem’s connection port object The client sends a connection request The server creates two private communication ports and returns the handle to one of them to the client The client and server use the corresponding port handle to send messages or callbacks and to listen for replies

Local Procedure Calls in Windows XP 39

Remote Method Invocation (RMI) is a Java mechanism similar to RPCs RMI allows a Java program on one machine to invoke a method on a remote object 40

File Access: typical use • • • #include <stdio. h> FILE *fp; To open a file – • To read a character from a file: – • fp = fopen(“~/tmp. txt”, “r+”); int c; c = getc(fp); To write a character to a file: – putc(c, fp); • Read/write using fscanf/fprintf, fgets/fputs, . . • Finally, fclose(fp) is used to close a file 41 CSRU 3130 Unix Programming Spring 2008

fopen() routine (1) #include <stdio. h> FILE *fopen(const char *path, const char *mode); • path: a relative or full path name of the file • mode: access mode – – – 42 “r”: open the file to read “w”: open the file to write (existing content will be discarded) “a”: open file to append (write starts at end of file) “rw”: open file to read and write … CSRU 3130 Unix Programming Spring 2008

fopen routine (2) FILE *fopen(const char *path, const char *mode); FILE : a data structure containing info. needed to perform input or output operations on it, including: a file descriptor (will study in low-level file access) current stream position an end-of-file indicator , an error indicator a pointer to the stream's buffer, if applicable Note: 43 read/write: at current stream position Buffered I/O: not every write is applied to the disk immediately CSRU 3130 Unix Programming Spring 2008

Example • FILE * fp = fopen(“~/tmp. txt”, “r+”); – – • takes a filename, does some housekeeping and negotiation with the kernel Returns pointer to the FILE data structure on success; return NULL on failure Always check for error after the call if (fp==NULL){ printf (“failed to open file ~/tmp. txtn”); exit(1); } 44 CSRU 3130 Unix Programming Spring 2008

Meaningful error message • errno – integer variable, set by system calls and some library functions in event of an error to indicate what went wrong # include <errno. h> … if (fp==NULL){ Use “man errno” to switch (errno) { all error code. case EACCES: printf (“You don’t have permissionn”); break; … default: printf (“Something went wrong in fopenn”); } exit(1); } 45 CSRU 3130 Unix Programming Spring 2008 find out

Using perror() #include <stdio. h> void perror(const char *s); perror() produces a message on standard error output, describing the last error encountered, i. e. , errno. FILE * fp = fopen(“~/tmp. txt”, “r+”); if (fp==NULL){ perror(“open ~/tmp. txt”); exit(1); } 46 CSRU 3130 Unix Programming Spring 2008

Three special files Whenever a program is started, three files are automatically opened, with file pointers stdin, stdout, stderr. getchar() is same as getc(stdin) putchar(c) is same as putc(stdout, c). printf(s, …) is same as fprintf (stdout, s, …); scanf(s, …) is same as fscanf (stdin, s, …); 47 CSRU 3130 Unix Programming Spring 2008

Other standard I/O Functions feof(FILE *): return non-zero when end of file is reached ferror(FILE *): return non-zero when any error fflush (FILE *): flush any buffered output to the file 48 CSRU 3130 Unix Programming Spring 2008

New vis: handling files int main(int argc, char *argv[]) { int strip = 0; int i; FILE *fp; while (argc > 1 && argv[1][0] == '-'){ switch (argv[1][1]) { case 's': /* -s: strip funny characters */ strip = 1; break; default: fprintf(stderr, "%s: unknown arg %sn", argv[0], argv[1]); return 1; } argc--; argv++; } 49 CSRU 3130 Unix Programming Spring 2008

main(): cont’d if (argc == 1) vis(stdin, strip); else for (i = 1; i < argc; i++) if ((fp = fopen(argv[i], "r")) == NULL) { fprintf(stderr, "%s: can't open %sn", argv[0], argv[1]); return 1; } else { vis(fp, strip); fclose(fp); } return 0; 50 CSRU 3130 Unix Programming Spring 2008 }

Now vis void vis(FILE *fp, int strip) { int c; while ((c = getc(fp)) != EOF) if (isprint(c) || isspace(c)) putchar(c); else if (!strip) printf("\%03 o", c); } 51 CSRU 3130 Unix Programming Spring 2008