Duke Systems CPS 210 Software Architecture and OS

Duke Systems CPS 210 Software Architecture and OS Platforms Jeff Chase Duke University http: //www. cs. duke. edu/~chase/cps 210

Friday • Recitations – Fridays: 3: 05 -4: 20 PM – Soc Sci 139: see maps. duke. edu – TA: Vamsi Thummala – This week: important info for P 1 A: heap manager • Monday – Prof. Landon Cox on Unix – Reading: flashback to 1968

“Software Architecture” User Applications Software architecture Physics stops here. Computer architecture Operating System(s) Substrate / Architecture Comparative architecture: what works Reusable / recurring design patterns • Used in OS • Supported by OS

Abstraction(s) • A means to organize knowledge – Capture what is common and essential – Generalize and abstract away the details – Specialize as needed – Concept hierarchy • A design pattern or element – Templates for building blocks – Instantiate as needed • E. g. : class, subclass, and instance

Platform abstractions • Platforms provide “building blocks”… • …and APIs to use them. – Instantiate/create/allocate – Manipulate/configure – Attach/detach – Combine in uniform ways – Release/destroy • Abstractions are layered. – What to expose? What to hide? The choice of abstractions reflects a philosophy of how to build and organize software systems.

[Garlan/Shaw 94] Components and connectors Abstract Data Type (ADT) object/class/module Connector pipes/filters objects / ADTs events layering repository interpreter Component Garlan and Shaw, An Introduction to Software Architecture, 1994.

“Components in context” Context (Domain) Thread Program For our purposes, an operating system is a platform that supports protection and isolation: every component runs within a context. Program, context and thread are OS abstractions.

Isolated contexts (domains) On most platforms, contexts are isolated. They cannot interfere with one another, even if they are running the same program. I. e. , each context is its own “sandbox”. The programs may fail independently. In classical OS, a context is called a process. Each has a private virtual address space. Butler Lampson’s definition: “I am isolated if anything that goes wrong is my fault (or my program’s fault). ” Isolation might not be absolute. E. g. , the program instances may choose to interact, or may access shared data.

Running a program code constants initialized data imports/exports symbols types/interfaces data Program When a program launches, the OS platform allocates memory to store its code and data. It may establish a new context and/or thread.

The Operating System • An operating system: – sets up the contexts – enforces isolation – mediates interactions • Applications trust the OS to do these things. • Any software platform for running application programs can be called an OS.

The Trusted Computing Base (TCB) • Programs may incorporate (link to) libraries packaged with the OS or written by third parties. • These libraries may even define a system API that applications use. (e. g. , heap manager malloc and free). • But they run within the program’s context, and so are not isolated from the rest of the program at runtime. Any failure in the library can damage the program, and any failure of the rest of the program can damage the library. • These libraries have no more privilege or power than any other part of the program. • The part of a platform that is responsible for the integrity of application contexts is called its TCB. • Trusted software is often called the kernel in classical OS.

Operating Systems: The Classical View Programs run as independent processes. data Protected system calls Protected OS kernel mediates access to shared resources. Each process has a private virtual address space and one or more threads. . and upcalls (e. g. , signals) Threads enter the kernel for OS services. The kernel code and data are protected from untrusted processes.

Key Concepts for Classical OS • kernel • The software component that controls the hardware directly, and implements the core privileged OS functions. • Modern hardware has features that allow the OS kernel to protect itself from untrusted user code. • thread • An executing instruction path and its CPU register state. • virtual address space • An execution context for thread(s) defining a name space for executing instructions to address data and code. • process • An execution of a program, consisting of a virtual address space, one or more threads, and some OS kernel state.

Protection Systems 101 Reference monitor Example: OS platform Isolation boundary Principles of Computer System Design © Saltzer & Kaashoek 2009

Post-note • This slide introduces some new terminology that we will hear more about later: • Principal. Some entity that requests an action. It is often (but not always) associated with a real-world identity, such as a user or account. E. g. , in Android, each application is a separate principal. • Authentication. When data arrives through a channel, or a channel is established, we generally need to determine something about the principal controlling the other endpoint of the channel. I. e. , “to whom am I speaking? ”. (See slides on “channels” later in this deck. ) • Authorization. Checks to determine if a given authenticated caller (a subject or principal) is permitted to perform the requested operation, according to an access control policy or procedure (the guard). • Reference monitor. Any component that has an API that is externally visible (through a channel) must check authorization for operations on that API before performing the op. The component must be isolated from its callers, so it must run in a context that is distinct from the callers. We can use the term reference monitor to refer to the context, or the component, or the guard.

Threads: a familiar metaphor 1 Page links and back button navigate a “stack” of pages in each tab. 2 Each tab has its own stack. One tab is active at any given time. You create/destroy tabs as needed. You switch between tabs at your whim. 3 Similarly, each thread has a separate stack. The OS switches between threads at its whim. One thread is active per CPU core at any given time

OS Platform: A Better Model Applications Libraries: shared by multiple applications Platform: same for all applications OS platform mediates access to shared resources. [RAD Lab]

[Garlan/Shaw 94] Components and connectors Abstract Data Type (ADT) object/class/module Connector pipes/filters objects / ADTs events layering repository interpreter Component Garlan and Shaw, An Introduction to Software Architecture, 1994.

GS 1. Pipes and filters Garlan and Shaw, An Introduction to Software Architecture, 1994.

GS 2. Objects and invocation Garlan and Shaw, An Introduction to Software Architecture, 1994.

GS 3. Events and implicit invocation publish notify subscribe register Event producers {publish/raise/announce/signal} events to notify {consumers/receivers/subscribers} of occurrences relating to a {subject/topic/category}. The system invokes a registered handler method/procedure in each receiver. Delivery is synchronous or asynchronous.

Communication: endpoints and channels endpoint port channel pipe binding connection data transfer stream flow messages request/reply RPC events operations advertise (bind) listen connect (bind) close write/send read/receive If one side advertises a named endpoint, we call it a server. If one side initiates a channel to a named endpoint, we call it a client.

Pipes parent operations close write read pipe Unix supports IPC channels called pipes between pairs of processes. § Unidirectional byte stream (upstream, downstream). § Brokered by a common parent that creates the pair. § The child processes need not create/initiate the pipe or even know where it comes from or where it goes. § A pipe may be the “standard” input or output.

Example: Unix Pipes [http: //www. bell-labs. com/history/unix/philosophy. html]

Map. Reduce/Hadoop Map. Reduce is a filter-and-pipe model for data-intensive cluster computing

We stopped here We discussed the need for protected contexts (“sandboxes”), even on single-user systems like your smartphone. The discussion touched on various dimensions of isolation for protected contexts (e. g. , processes): • Fault isolation. One app or app instance (context or process) can fail independently of others. If it fails, the OS can kill the process and reclaim all of its memory, etc. • Performance isolation. The OS manages resources (“metal and glass”: computing power, memory, disk space, I/O throughput capacity, network capacity, etc. ). Each instance needs the “right amount” of resources to run properly. The OS prevents apps from impacting the performance of other apps. E. g. , the OS can prevent a program with an endless loop from monopolizing the processor. • Security. An app may contain malware that tries to corrupt the system, steal data, or otherwise compromise the integrity of the system. The OS uses protected contexts and a reference monitor to check and authorize all accesses to data or objects outside of the context, including channel setup. )

OS protection The discussion touched on how a classical OS uses the hardware to protect itself. (More on this later. ) • In particular, machines support events. The OS kernel registers handlers for various machine events when it boots (starts up). These events include machine exceptions (faults), which may be caused by errant code, interrupts from the clock or external devices (e. g. , network packet arrives), and deliberate kernel calls (traps) caused by programs requesting service from the kernel through its API. • All of these machine events make safe control transfers into the registered kernel handler for the named event. Applications run in sandboxes that prevent them from calling procedures in the kernel or accessing kernel data directly (unless the kernel chooses to allow it). • In fact, after the system is booted, these events are the only ways to ever enter the kernel, i. e. , to run code in the kernel.

GS 4. Layered systems Garlan and Shaw, An Introduction to Software Architecture, 1994.

Post-note • Note that an operating system is a layered system. – The platform API encapsulates all the functionality below. – Other platforms may be layered on top (“nested dolls”) – Applications above the platform API provide other useful functions. – These application-layer programs include utilities (e. g. , program development tools) and servers (e. g. , window system). – See the layering diagrams for Android and Unix. • Also, an OS uses layering below the platform API. – e. g. , network protocol stack (later) – device driver software, etc.

Unix defines uniform, modular ways to combine programs to build up more complex functionality. Other application programs sh nroff cpp who a. out Kernel comp date Hardware cc wc as ld vi ed grep Other application programs

GS 5. Shared state repository Garlan and Shaw, An Introduction to Software Architecture, 1994.

State storage object segment file table object access attach/detach mount/unmount map/unmap open/close put/get fetch read/write push commit store filesystem repository volume database target bit, byte block page row fragment stripe object

GS 6. Interpreter Garlan and Shaw, An Introduction to Software Architecture, 1994.

Interpreter: example An interpreter controls how a program executes and what it sees. An interpreter can “sandbox” a program for isolation.

ECMAscript Some languages and interpreters are safer than others.

Architectural styles: tradeoffs • constraints on composition – give rise to general properties of compositions • generality of connectors – data type signatures • coupling and control • concurrency • state (data) partitioning and sharing – less sharing better scalability – standardizing interfaces vs. schema

The Birth of a Program (C/Ux) myprogram. c int j; char* s = “hellon”; myprogram. o assembler data object file int p() { j = write(1, s, 6); return(j); } data p: compiler …. . libraries and other objects linker store this store that push jsr _write ret etc. myprogram. s data program myprogram (executable file)