Intro to DFS assignment Announcements DFS Part PA

  • Slides: 33
Download presentation
Intro to DFS assignment

Intro to DFS assignment

Announcements • DFS • Part (PA 4) Part 1 will be out over the

Announcements • DFS • Part (PA 4) Part 1 will be out over the weekend 1 Due 10/30 11: 59 pm 2 12/7 11: 59 pm 3 12/14 11: 59 pm 4 (optional extra credit) 12/23 11: 59 pm

You build a DFS! 1. Lock server, at-most-once RPC semantics 2. Implement extent server;

You build a DFS! 1. Lock server, at-most-once RPC semantics 2. Implement extent server; create/lookup/readdir FUSE ops 3. Implement read/write/open/setattr 4. Implement mkdir/unlink, integrate locks! (optional)

Programming Assignment: YFS • Yet-another File System. : ) • Simpler version of what

Programming Assignment: YFS • Yet-another File System. : ) • Simpler version of what we just talked about: only one extent server (you don’t have to implement Petal; single lock server)

Part 1: lock server • Implementing the lock server • Provide mutual exclusion to

Part 1: lock server • Implementing the lock server • Provide mutual exclusion to ‘lock resources’ • Using pthread mutexes, condition variables! • Implementing at-most-once RPC semantics

Providing mutual exclusion over the net • In the assignment: lock is a 64

Providing mutual exclusion over the net • In the assignment: lock is a 64 -bit number • Client might: • acquire(lock_a); do work; release(lock_a); • You must create the lock if it doesn’t yet exist on the server

Example tester %. /lock_tester 3772 simple lock client acquire a release a acquire b

Example tester %. /lock_tester 3772 simple lock client acquire a release a acquire b release a test 2: client 0 acquire a release a test 2: client 2 acquire a release a. . .

What’s provided • Simple RPC framework, skeleton for lockserver code • What it does:

What’s provided • Simple RPC framework, skeleton for lockserver code • What it does: handles setting up sockets, marshalling/unmarshalling basic data types, sending over TCP (whew) • What it doesn’t: does not keep track of RPC request state; duplicate RPCs are invoked twice! : (

Example RPC call

Example RPC call

Your job for lock server

Your job for lock server

Lossy • RPC_LOSSY: drops, duplicates, delays. • Run lock_tester with RPC_LOSSY=0 • Now try

Lossy • RPC_LOSSY: drops, duplicates, delays. • Run lock_tester with RPC_LOSSY=0 • Now try running it with RPC_LOSSY=5 • Simulates network failure • it should hang or error. FAIL!

Why? • No at-most-once RPC semantics • implemented. . . yet. • If reply

Why? • No at-most-once RPC semantics • implemented. . . yet. • If reply dropped, duplicate sent: acquire(a), acquire(a). . . .

Strawman • Remember every RPC call • Client sends a unique RPC identifier •

Strawman • Remember every RPC call • Client sends a unique RPC identifier • Remember every RPC reply • To avoid invoking the actual function • What’s the problem with this approach?

Sliding window RPCs • Assume the following sent by client: marshall m 1 <<

Sliding window RPCs • Assume the following sent by client: marshall m 1 << clt nonce // client id << svr nonce // server id << proc // procedure number (bind, acquire, etc). << myxid // unique request id for this RPC << xid_rep_window. front() // Last out-of-order RPC reply received << req. str(); // Data • But we need some additional info from the client

At the end % export RPC_LOSSY=5 %. /lock_server 3772 & %. /lock_tester 3772. .

At the end % export RPC_LOSSY=5 %. /lock_server 3772 & %. /lock_tester 3772. . . all tests passed %. /rpctest simple_tests. . . all tests passed

Helper functions • Method_thread

Helper functions • Method_thread

Code reading

Code reading

Part 2 • Extent server • FUSE! • Semantics of filesystem calls

Part 2 • Extent server • FUSE! • Semantics of filesystem calls

Programming Assignment: YFS • Part 1: lock server • Part 2: extent server

Programming Assignment: YFS • Part 1: lock server • Part 2: extent server

The extent server • Don’t have to deal with storing data on disk: you

The extent server • Don’t have to deal with storing data on disk: you will store contents in memory • map<inode, std: : string> • All yfs_client operations synchronize with same extent server

Getattr (given)

Getattr (given)

Your job • Extend extent_server to support • put(extentid, string, . . . )

Your job • Extend extent_server to support • put(extentid, string, . . . ) • get(extentid, string&) • remove(extentid, . . . ) • See extent_protocol. h and extent_smain. cc • Later you will likely add more! • Must properly deal with ctime/atime/mtime

Data formats at extent server • Directories maintain a mapping of filenames to inode

Data formats at extent server • Directories maintain a mapping of filenames to inode numbers • if root (inode 1) has two files “file 1”, “file 2”: get(1) might return • file 1: 3983293923 • file 2: 3384927553 • You can choose how you store this information as a string

Metadata time mgmt • atime: access time • Updated whenever file contents accessed •

Metadata time mgmt • atime: access time • Updated whenever file contents accessed • Set to 0 on file creation • mtime: modification time • Updated when file contents modified • ctime: change time • Updated whenever metadata modified

FUSE • http: //fuse. sourceforge. net/

FUSE • http: //fuse. sourceforge. net/

Mapping FUSE functions

Mapping FUSE functions

Example • Fuse. cc • void • fuseserver_getattr(

Example • Fuse. cc • void • fuseserver_getattr(

CREATE/MKNOD • Generate inode number (rand()) • This is called on files: make sure

CREATE/MKNOD • Generate inode number (rand()) • This is called on files: make sure inode has MSB set to 1 • Directories have MSB set to 0 • Create the file at the extent server • Add the name to inode mapping to parent info at extent server

Lookup • Given: filename, parent inode • Look through parent directory list • Find

Lookup • Given: filename, parent inode • Look through parent directory list • Find inode that maps from filename • Getattr on that inode • Fill in return structure (fuse_entry_param)

Readdir • Given: inode number for a directory • Get filename: inode mapping for

Readdir • Given: inode number for a directory • Get filename: inode mapping for that dir • For each filename: inode pair call dirbuf_add(. . . ) function provided

 • You will be able to create empty files in the root directory

• You will be able to create empty files in the root directory and do an ls on the root • You’re not yet storing files. . .

Testing • . /start. sh • . /test-lab-2. pl • . /stop. sh •

Testing • . /start. sh • . /test-lab-2. pl • . /stop. sh • test-lab-2. pl creates a bunch of files, tests whether the files the script created are there

Parts 3 and 4 • Part 3: You implement open, read, write, setattr •

Parts 3 and 4 • Part 3: You implement open, read, write, setattr • Part 4: You implement mkdir, unlink, and locking