Preliminaries l Mailing list u u l List

Preliminaries l Mailing list u u l List of students u l Questions about papers, doing the presentations? Evals u l Finding partners Office hour? u l Send me your email address if you haven’t already Even if you are just going to sit in Great! Put them online? Anonymized? Slides u Online also? January 13, 2000 CSE 291: Web. OS & Legion 1

Web. OS Operating System Services for Wide Area Applications Amin Vahdat, Eshwar Belani, Paul Eastham, Chad Yoshikawa, Thomas Anderson, David Culler, and Michael Dahlin

Vision l l Framework for supporting wide-area, distributed, highly available, incrementally scalable, reconfigurable services and applications Components u u Resource discovery – which server do I use? Persistent storage – wide-area file system Remote process execution – fork on a remote machine Authentication and security – provide trust January 13, 2000 CSE 291: Web. OS & Legion 3

Example App: Rent-A-Server l Set of generic servers distributed across Internet u u u l Transparent, automatic replication of HTTP service (CNN) Spawn new servers when demand is high Do not have to statically allocate resources to match peak demand Advantages of using Web. OS to build Rent-A-Server u u u Better availability – transparently mask server failures Better cost-performance – do not have to provide peak capacity for peak Better burst behavior (handle peak demand) » Efficient use of a collection of resources January 13, 2000 CSE 291: Web. OS & Legion 4

Resource Discovery l How to choose the best server to send request? u u l HTTP redirect requires two round-trips DNS is round-robin (performance insensitive) Web. OS uses Smart Clients u u u Applet chooses server based upon distance and server load Servers periodically piggy-back load info back to applet at client to update load info Load info grows stale, defaults to random January 13, 2000 CSE 291: Web. OS & Legion 5

Smart Clients l Algorithm u u u u User requests a service (e. g. , “service: //chat”) Meta-applet in browser handles “service: ” URLs, contacts a “service: ” server (e. g. , by using Alta. Vista) “Service” server returns a list of available hosts supporting the service (e. g. , chat servers) + info (location, load) Meta-applet chooses a host, downloads Smart Client applet runs in client browser Smart Client chooses chat server based upon location, load Smart Client updates its info as user continues to make requests to server January 13, 2000 CSE 291: Web. OS & Legion 6

Smart Proxies l l l Similar to Smart Clients, but run inside of Proxies Aggregate client requests to servers, much better picture of server load Get around Java applet restrictions (making network connections to arbitrary sites) January 13, 2000 CSE 291: Web. OS & Legion 7

Wide Area File System l Web. FS u Persistent files named using URLs » Location dependent u u u Can use standard Web servers (backwards compatible) Authenticated access when talking to other Web. FS’s Consistent » Application controlled; e. g. , invalidation as in AFS u Caches data » Services using Web. FS get caching for free u Implemented to the Vnode interface » Looks like another file system to applications January 13, 2000 CSE 291: Web. OS & Legion 8

Process Control l Fork processes on remote servers u l Resource Manager controls Web. OS jobs u u l As easily as forking on own machine Does admission control Authenticates the source of the job Jobs run in restricted virtual machine (Janus) u u u Privileged system calls are intercepted Configuration files determine whether calls are allowed to continue (c. f. Java security manager) Resources limited using Unix resource controls January 13, 2000 CSE 291: Web. OS & Legion 9

Security and Authentication l Based on CRISIS [Belani 98] u l Entities u u u l Not going to go into details, we’re going to read it later on Principles – sources of trust Objects – resources Reference monitors – grant requests Statements of trust represented by certifications u u u Authenticate principles Validate privileges (access controls to resources) Transfer privileges (very useful) January 13, 2000 CSE 291: Web. OS & Legion 10

Applications l Internet Chat u u l Remote Compute Engine u u l Web. OS server on supercomputers Applications are spawned inside Web. OS VMs Cooperative Web Caching u l Each room is implemented as a consistent, shared file Writes are appended to the file, updates multicast to clients Smart Clients to choose best cache, Web. FS to distribute data Internet Weather u u Central server maintains Internet performance database Access server using an applet; use also contributes data January 13, 2000 CSE 291: Web. OS & Legion 11

Rent-A-Server Redux l Web. OS killer app u u u l Smart Clients choose best server to use Process control allows dynamic spawning of servers Web. FS allows replication of data among servers CRISIS allows trust among servers Load balancer spawns/reaps servers depending upon load How did you like this service? u Billing… January 13, 2000 CSE 291: Web. OS & Legion 12

Discussion l l Questions about the paper, system? Would you want to use Web. OS to build wide-area services? January 13, 2000 CSE 291: Web. OS & Legion 13

What did I like? l Services motivated by application needs u u l Leverage existing services u u l URLs for names, Java to implement client functionality, SSL for secure connections Don’t try to reinvent the world Implemented u u l Application requirements well understood Design to meet those requirements Web. OS services implemented 5 applications implemented Good systems paper January 13, 2000 CSE 291: Web. OS & Legion 14

The Core Legion Object Model Mike Lewis and Andrew Grimshaw

Vision l Provide a single virtual machine model across all hosts connected by the Internet u u u l Location and distribution are transparent to users and applications “System boundaries, data location, faults are invisible” Applications are run on Legion, not on a host Contrast with systems where only the services are well defined u u SMTP, MPI, HTTP, DCE-RPC, etc. Web. OS January 13, 2000 CSE 291: Web. OS & Legion 16

Target Applications l Compute intensive u u l l Highly parallel Coarse grained u l Marshal CPU resources of many machines Workstations, supercomputers (toasters? ) Has to be if wide-area Applications? u u u Funny, not specified… Large scale scientific apps – supercomputing++ Highly parallel apps w/ minimal communiction » Cracking crypto keys January 13, 2000 CSE 291: Web. OS & Legion 17

Objectives (The Kitchen Sink) 1. 2. 3. 4. 5. 6. 7. 8. 9. 10. Site autonomy Extensible core (*) Scalable architecture Seamless computing environment Highly parallel Global name space (*) Security for users and resources Heterogeneous – many platforms Multiple languages, interoperability Fault tolerant January 13, 2000 CSE 291: Web. OS & Legion 18

Legion Objects l l Interface specified using an IDL (CORBA, Mentat) Implementation can be done using any language u l Legion compiler wraps it up into a Legion Object Coarse grained u u Not fine-grained like C++, Java, Smalltalk Execute in disjoint address spaces Think of an object as being a process For an app, probably one object per host January 13, 2000 CSE 291: Web. OS & Legion 19

Object Communication l Objects communicate via method calls u l Example methods mostly deal with control u l Non-blocking (not sure how to synchronize? ) Locating, creating, deleting, etc. , objects Not sure if methods are intended for data transport u u Use Legion comm to locate and establish links with other Legion Objects Once all objects are connected, use, e. g. , MPI to shuttle data around? January 13, 2000 CSE 291: Web. OS & Legion 20

Core Objects l Core classes in Legion implement essential services u u u Legion. Object Legion. Class Legion. Host Legion. Vault Legion. Binding. Agent January 13, 2000 CSE 291: Web. OS & Legion 21

Legion. Object l Root of type and instance hierarchy u u l All objects derive from Legion. Object Classes are objects, so they derive, too Interface for essential methods u u Access – may_I() Persistence – save_state(), restore_state() January 13, 2000 CSE 291: Web. OS & Legion 22

Legion. Class l Responsible for managing object instances u l Subclasses u l derive(), inherit_from() Locating objects u l create(), delete() get_binding() Each class object can construct the address of all of its instances (including subclasses)…more later January 13, 2000 CSE 291: Web. OS & Legion 23

Legion. Host l l Represents a host Subclasses represent different kinds of hosts u l All objects are created on a Legion. Host u l Unix. Host, Unix. SMMP, CM-5, etc. They must live somewhere and have backing store allocated for them Used for resource control u u Legion. Host determines whether an object can execute on it “Site autonomy” January 13, 2000 CSE 291: Web. OS & Legion 24

Legion. Vault l l l Interface to persistent storage Objects have two states – Active, Inert objects stored as Object Persistent Representations (OPRs) u l Legion. Vaults store OPRs u l Objects save and restore state from their OPRs E. g. , on disk Objects are migrated by storing to OPR, migrating OPR from one Vault to another, and restoring u Very coarse-grained migration January 13, 2000 CSE 291: Web. OS & Legion 25

Legion. Binding. Agent l Binds object identifiers to addresses u u l Think of it as a DNS server of sorts …or something that maps URIs to URLs Need to talk about LOIDs and Object Addresses… January 13, 2000 CSE 291: Web. OS & Legion 26

Legion Object IDs (LOIDs) Format l l l Instance # Public Key (Additional Fields) Universal, unique name of an object u l Class ID Does not specify location Format – defines the fields in the LOID Class ID – class of object (from Legion. Object) Instance # – instance identifier (from Legion. Class) Public Key – used for security and authentication Additional Fields – depends on format; extensible January 13, 2000 CSE 291: Web. OS & Legion 27

Object Addresses l l l Programs name objects using LOIDs, but you can’t find an object Object Addresses (OA) define the location of an object To use an object, its LOID must be mapped to an OA u u l OAs contain Object Address Elements u l Similar in spirit to binding process used in RPC No session semantics, though List of addresses, can be used to provide multicast Practically, an OA is an IP address and port # January 13, 2000 CSE 291: Web. OS & Legion 28

Binding l Legion. Binding. Agents (LBAs) provide the service that binds LOIDs to OAs (get_binding ()) u l A wants to get a binding for B (has class C) u u l …with the help of Legion. Class A queries an LBA with LOID(B) Recall classes know the OAs of their instances LBA contacts class C to find OA for B LBA can find C by contacting Legion. Class (since C is an instance of Legion. Class) Caching (in object A, LBA, and class C) makes it scalable January 13, 2000 CSE 291: Web. OS & Legion 29

Discussion l Questions about the paper, system? l Move into discussion… u u Single virtual machine model Universal names January 13, 2000 CSE 291: Web. OS & Legion 30

Single Virtual Machine l Is a single virtual machine the best model? u u l The VM model provides transparency Transparency is extremely useful…but also seductive May not want transparency for the wide-area u RPC aims to be transparent » Is RPC successful? Butler Lampson says NO. u u l Can’t make faults transparent Can’t make performance transparent Well-defined service interfaces, protocols enough u u E. g. , HTTP Difference between distributed systems, parallel systems? January 13, 2000 CSE 291: Web. OS & Legion 31

Universal Names l In favor of a universal naming mechanism u l Orthogonal to single virtual machine model u l Can be its own service, not tied to a particular system architecture (again, DNS) One problem universal names should solve is object mobility u l DNS is very successful Legion LOIDs do not seem suited for this What do you think? u u Universal names a good idea? LOIDs and OAs a good scheme? January 13, 2000 CSE 291: Web. OS & Legion 32

What did I like? l Like the goal of unique universal names u l Funny way to justify system architecture, though u l A service we should have “Our system is more ambitious than other systems, so it must be better” Btw, what are the apps again? January 13, 2000 CSE 291: Web. OS & Legion 33

Web. OS/Legion l What do you think about Web. OS vs. Legion? l Programming model vs. services? January 13, 2000 CSE 291: Web. OS & Legion 34