Advanced Operating Systems n Main Text n n

Advanced Operating Systems n Main Text: n n Project Reference Text: n n “Advanced Concepts in Operating Systems: Distributed, Database, & Multiprocessor Operating Systems” by Mukesh Singhal and Niranjan G. Shivaratri. Mc. Graw Hill publishers. Unix Network programming: Networking APIs Sockets and XTI (2 nd edition), Prentice Hall. Reference Papers: n Recommend reading appropriate reference papers given in each chapter end. B. Prabhakaran 1

Contact Information B. Prabhakaran Department of Computer Science University of Texas at Dallas Mail Station EC 31, PO Box 830688 Richardson, TX 75083 Email: praba@utdallas. edu Fax: 972 883 2349 URL: http: //www. utdallas. edu/~praba/cs 6378. html Phone: 972 883 4680 Office: ES 3. 706 Office Hours: 11. 45 am-12. 45 pm, Mon/Wednesdays Other times by appointments through email Announcements: Made in class and on course web page. TA: TBA. B. Prabhakaran 2

Course Outline Proposed Outline. Might be modified based on time availability: n Introduction to operating systems, inter-process communication. (Chapters 1 & 2). n Distributed Operating Systems n n n Architecture (Chapter 4) Clock Synchronization, Ordering (Chapter 5) Distributed Mutual Exclusion (Chapter 6) Distributed Deadlock Detection (Chapter 7) Agreement Protocols (Chapter 8) Distributed Resource Management n n n Distributed File Systems (Chapter 9) Distributed Shared Memory (Chapter 10) Distributed Scheduling (Chapter 11) B. Prabhakaran 3

Course Outline. . . n Recovery & Fault Tolerance n n Chapters 12 and 13 Concurrency Control/ Security n Depending on time availability Discussions will generally follow the main text. However, additional/modified topics might be introduced from other texts and/or papers. References to those materials will be given at appropriate time. B. Prabhakaran 4

Evaluation n n 1 Mid-term: in class. 75 minutes. Mix of MCQs (Multiple Choice Questions) & Short Questions. 1 Final Exam: 75 minutes or 2 hours (depending on class room availability). Mix of MCQs and Short Questions. 2 - 3 Quizzes: in class. 5 -6 MCQs or very short questions. 15 -20 minutes each. Homeworks/assignments: 3 or 4 spread over the semester. Programming Projects: 3 planned over the semester B. Prabhakaran 5

Grading n n n Home works: 5% Quizzes: 10% (15% - if 3 rd Quiz) Mid-term & Final: 55% Project 1: 5% Projects 2 & 3: 25% B. Prabhakaran 6

Schedule n n n Quizzes: Dates announced in class & web, a week ahead. Mostly just before midterm and final. Mid-term: February 26, 2007 Final Exam: Last day of class (April 23 rd) OR 8. 30 am, April 30, 2007 (As per UTD schedule) Subject to minor changes Quiz, projects, and homework schedules will be announced in class and course web page, giving sufficient time for submission. Likely project deadlines: end January, Middle of March, End of semester (April). B. Prabhakaran 7

Programming Projects n n n n No copying/sharing of code/results will be tolerated. Any instance of cheating in projects/homeworks/exams will be reported to the University. No copying code from the Internet. 2 individual students copying code from Internet independently: still considered copying in the project !! Individual projects. Projects might involve Unix, C/C++/Java programming, network programming. Deadlines will be strictly followed for projects and homeworks submissions. Projects submissions through Web CT. Demo may be needed B. Prabhakaran 8

Web CT n n Go to: http: //webct 6. utdallas. edu Web CT has a discussion group that can be used for project and other course discussions. B. Prabhakaran 9

Cheating n n n Academic dishonesty will be taken seriously. Cheating students will be handed over to Head/Dean for further action. Remember: home works/projects (exams too !) are to be done individually. Any kind of cheating in home works/ projects/ exams will be dealt with as per UTD guidelines. Cheating in any stage of projects will result in 0 for the entire set of projects. B. Prabhakaran 10

Projects n n n 3 projects planned. Involves exercises such as ordering, deadlock detection, load balancing, message passing, and implementing distributed algorithms (e. g. , for scheduling, etc. ). Platform: Linux, C/C++/Java. Network programming will be needed. Multiple systems might be used. Specific details and deadlines will be announced in class and course webpage. Suggestion: Learn network socket programming and threads, if you do not know already. Try simple programs for file transfer, talk, etc. Sample programs and tutorials available at: n http: //www. utdallas. edu/~praba/projects. html B. Prabhakaran 11

Homeworks n n Each homework will be for 10 marks. Homeworks Submission: n Submit on paper to TA/Instructor. B. Prabhakaran 12

Basic Computer Organization n n n Input Unit Output Unit CPU Memory ALU (Arithmetic & Logic Unit) Secondary Storage Memory ALU Disk CPU I/O Display Keyboard B. Prabhakaran 13

Simplified View of OS OS Kernel Physical OS Tools Memory User Processes User i Process j Code Data Code Data User Virtual Memory Processes. . Data Tools ++ Memory Space B. Prabhakaran 14

Distributed View of the System hardware Process hardware B. Prabhakaran 15

Inter-Process Communication n n Need for exchanging data/messages among processes belonging to the same or different group. IPC Mechanisms: n Shared Memory: Designate and use some data/memory as shared. Use the shared memory to exchange data. n n Message Passing: Use “higher” level primitives to “send” and “receive” data. n n Requires facilities to control access to shared data. Requires system support for sending and receiving messages. Operation oriented language constructs n n n Request-response action Similar to message passing with mandatory response Can be implemented using shared memory too. B. Prabhakaran 16

IPC Examples n Parallel/distributed computation such as sorting: shared memory is more apt. n n Client-server type: message passing or RPC may suit better. n n Using message passing/RPC might need an array/data manager of some sort. Shared memory may be useful, but the program is more clear with the other types of IPCs. RPC vs. Message Passing: if response is not a must, atleast immediately, simple message passing should suffice. B. Prabhakaran 17

Shared Memory Writers/ Producers Only one process can write at any point in time. No access to readers. Shared Memory Multiple readers can access. No access to Writers. Readers/ Consumers B. Prabhakaran 18

Shared Memory: Possibilities n n n Locks (unlocks) Semaphores Monitors Serializers Path expressions B. Prabhakaran 19

Message Passing n n n Blocked Send/Receive: Both sending and receiving process get blocked till the message is completely received. Synchronous. Unblocked Send/Receive: Both sender and receiver are not blocked. Asynchronous. Unblocked Send/Blocked Receive: Sender is not blocked. Receiver waits till message is received. Blocked Send/Unblocked Receive: Useful ? Can be implemented using shared memory. Message passing: a language paradigm for human ease. B. Prabhakaran 20

Un/blocked n Blocked message exchange n n n Easy to: understand, implement, verify correctness Less powerful, may be inefficient as sender/receiver might waste time waiting Unblocked message exchange n n n More efficient, no waste on waiting Needs queues, i. e. , memory to store messages Difficult to verify correctness of programs B. Prabhakaran 21

Message Passing: Possibilities receiver i sender receiver j receiver i sender receiver k receiver j receiver k B. Prabhakaran 22

Message Passing: Possibilities. . . sender i sender j sender k sender i receiver sender j receiver sender k B. Prabhakaran 23

Naming n Direct Naming n n Port Naming n n n Specify explicitly the receiver process-id. Simple but less powerful as it needs the sender/receiver to know the actual process-id to/from which a message is to be sent/received. Not suitable for generic client-server models receiver uses a single port for getting all messages, good for client-server. more complex in terms of language structure, verification Global Naming (mailbox) n n suitable for client-server, difficult to implement on a distributed network. complex for language structure and verification B. Prabhakaran 24

Communicating Sequential Processes (CSP) process reader-writer OKtoread, OKtowrite: integer (initially = value); busy: boolean (initially = 0); *[ busy = 0; writer? request() -> busy : = 1; writer!OKtowrite; � busy = 0; reader? request() -> busy : = 1; reader!OKtoread; � busy = 1; reader? readfin() -> busy : = 0; � busy = 1; writer? writefn() -> busy : = 0; ] B. Prabhakaran 25

CSP: Drawbacks n n Requires explicit naming of processes in I/O commands. No message buffering; input/output command gets blocked (or the guards become false) -> Can introduce delay and inefficiency. B. Prabhakaran 26

Operation oriented constructs Remote Procedure Call (RPC): Task A Service declarations …… RPC: abc(xyz, ijk); Task B Service declarations send xyz; wait for result return ijk ……. • • Service declaration: describes in and out parameters Can be implemented using message passing Caller: gets blocked when RPC is invoked. Callee implementation possibilities: • Can loop “accepting” calls • Can get “interrupted” on getting a call • Can fork a process/thread for calls B. Prabhakaran 27

RPC: Issues n n Pointer passing, global variables passing can be difficult. If processes on different machines, data size (number of bits for a data type) variations need to be addressed. n n n Abstract Data Types (ADTs) are generally used to take care of these variations. ADTs are language like structures that specify how many bits are being used for integer, etc… What does this imply? Multiple processes can provide the same service? Naming needs to be solved. Synchronous/blocked message passing is equivalent to RPC. B. Prabhakaran 28

Ada task [type] <name> is entry specifications end Task body <name> is Declaration of local variables begin list of statements …. . accept <entry id> (<formal parameters> do body of the accept statement end<entry id> exceptions Exception handlers end; B. Prabhakaran task proc-buffer is entry store(x: buffer); remove(y: buffer); end; task body proc-buffer is temp: buffer; begin loop when flag accept store(x: bu temp : = x; flag : =0; end store; When !flag accept remove(y: y : = temp; flag : =1; end remove; end loop end proc-buffer. 29

Ada Message Passing Task A entry store(. . . ) …. accept Store(. ) …. . . Task B xyz is sent from …. Task B to A result Store(xyz); …. . Somewhat similar to executing procedure call. Parameter value for the entry procedure is supplied by the calling task. Value of Result, if any, is returned to the caller. B. Prabhakaran 30

RPC Design n Structure n n n Binding n n n Caller: local call + stub Callee: stub + actual procedure Where to execute? Name/address of the server that offers a service Name server with inputs from service specifications of a task. Parameter & results n n Packing: Convert to remote machine format Unpacking: Convert to local machine format B. Prabhakaran 31

RPC Execution Binding Server Local Proc. Local call Stub Receive Query binding server Return Server Address Params packing Register Services Stub Remote Proc. Unpack Execute procedure Local call Wait Return Pack results Unpack result Caller Return Callee B. Prabhakaran 32

RPC Semantics n At least once n n n Exactly once n n n A RPC results in zero or more invocation. Partial call, i. e. , unsuccessful call: zero, partial, one or more executions. Only one call maximum Unsuccessful? : zero, partial, or one execution At most once n Zero or one. No partial executions. B. Prabhakaran 33

RPC Implementation n Sending/receiving parameters: n n n Use reliable communication? : Use datagrams/unreliable? Implies the choice of semantics: how many times a RPC may be invoked. B. Prabhakaran 34

RPC Disadvantage n Incremental results communication not possible: (e. g. , ) response from a database cannot return first few matches immediately. Got to wait till all responses are decided. B. Prabhakaran 35

Distributed Operating Systems Issues : n n n n n Global Knowledge Naming Scalability Compatibility Process Synchronization Resource Management Security Structuring Client-Server Model B. Prabhakaran 36

DOS: Issues. . n Global Knowledge n n n Lack of global shared memory, global clock, unpredictable message delays Lead to unpredictable global state, difficult to order events (A sends to B, C sends to D: may be related) Naming n n Need for a name service: to identify objects (files, databases), users, services (RPCs). Replicated directories? : Updates may be a problem. Need for name to (IP) address resolution. Distributed directory: algorithms for update, search, . . . B. Prabhakaran 37

DOS: Issues. . n Scalability n n n System requirements should (ideally) increase linearly with the number of computer systems Includes: overheads for message exchange in algorithms used for file system updates, directory management. . . Compatibility n n n Binary level: Processor instruction level compatibility Execution level: same source code can be compiled and executed Protocol level: Mechanisms for exchanging messages, information (e. g. , directories) understandable. B. Prabhakaran 38

DOS: Issues. . n Process Synchronization n n Resource Management n n n Distributed shared memory: difficult. Data/object management: Handling migration of files, memory values. To achieve a transparent view of the distributed system. Main issues: consistency, minimization of delays, . . Security n Authentication and authorization B. Prabhakaran 39

DOS: Issues. . n Structuring n n Monolithic Kernel: Not needed (e. g. , ) file management not needed fully on diskless workstations. Collective kernel: distributed functionality on all systems. n n Micro kernel + set of OS processes Micro kernel: functionality for task, memory, processor management. Runs on all systems. OS processes: set of tools. Executed as needed. Object-oriented system: services as objects. n n Object types: process, directory, file, … Operations on the objects: encapsulated data can be manipulated. B. Prabhakaran 40

DOS: Communication Computer Switch B. Prabhakaran 41

ISO-OSI Reference Model Application Presentation Session Communication Network Transport Session Transport Network Datalink Physical B. Prabhakaran 42

Un/reliable Communication n Reliable Communication n n Virtual circuit: one path between sender & receiver. All packets sent through the path. Data received in the same order as it is sent. TCP (Transmission Control Protocol) provides reliable communication. Unreliable communication n Datagrams: Different packets are sent through different paths. Data might be lost or out of sequence. UDP (User datagram Protocol) provides unreliable communication. B. Prabhakaran 43