Project 3 File System Design COS 318 Fall

Project 3: File System Design COS 318 Fall 2002

Last Time • Web Server u/ vivek/ public_html/ server client request response index. html cos 318_02/ • Extensive use of a file system on server machine without actually worrying about how it was implemented.

Outline • • • Introduction Design requirement Helper functions Implementation of file system calls Hints Reference

Introduction • Why do we need a file system? – Abstraction – Protected access – Security • Simple Unix-like file system – i-node – directory tree – links(symbolic/physical)

What does my file server need to accomplish • A New Client Program with • New Methods: – GET <dir name | file name> • Fetch a file or dir listing and show them – CREATE <dir name> • Create a dir in server disk – DELETE <file name | dir name> • Delete a file or dir in server disk • populated directories should not be deleted – PUT <file name> <local file name> <startpos> <endpos> • Create a new file in server disk, or append to an exsiting file with contents from local file [Notice: name uses absolute path]

More on file server functionality • Use our client program to create a populated directory tree inside of server disk, and then use your web server to serve this content. • More test programs: – Randomly requests – Maybe not just be PUT or GET: after some file deletions, things will become more interesting. – Coming soon…

So what do you have • A Virtual Disk: a data structure – Disk_name: dev_318 – Disk_pointer: used to access the disk(in fact a file descriptor) – Sector_size: 512 bytes – Capacity: M, G … • Rather than write to a real disk, you'll be building a disk inside of a large file • All development and debugging are in user space

Helper Functions: not tuxedo • • • Get. Existing. Disk(char *pathname) Create. New. Disk(void) Close. Disk(void) Read. Sector(int sector. Num, char *buf) Write. Sector(int sector. Num, char *buf) Notice: – You are allowed to access disk only with those function calls;

Helper Functions(cont. ) • Get. Existing. Disk(char *pathname) – returns a pointer to previously created disk in directory pathname, or exits if no such disk • Create. New. Disk(void) – returns a pointer to a new disk • Close. Disk(void) – closes the virtual disk(dev_318)

Helper Functions(cont. ) • Read. Sector(int sector. Num, char *buf) – reads 512 bytes from disk • Write. Sector(int sector. Num, char *buf) – writes 512 bytes from disk • Notice: – If any other user space function calls can facilitate your work, feel free to use them. • Using file system functions to build disk is relatively easy compared to using really disk to build file system

What do I need to do • You should build on the code you wrote for the previous assignments – Good thing: we have a reference server • The first step is to understand the default file system provided by the base code(helper functions) • You'll have to understand at least one new header with one of the new methods.

What do I need to do(cont. ) • You'll have to write versions of various system calls that you used in Project 1 – mkfs, open, close, read, and write – Additionally, you'll also have to implement versions of stat, mkdir, and a few others, also function calls to getting file metadata and directory information in order to properly build directory listings • No need to handle all of the various flags and parameters, but should implement enough to get the project working correctly

One example-open() • For example, the open() system call can be used to try to test the existence of a file in addition to creating it for writing • O_CREATE: the file creation flag – To properly handle “PUT" method • Not always true: need other flags – “GET” a non-existent file

mkfs • File system creation – Dos: format – Linux: mkfs/mke 2 fs/mkfs. msdos – Solaris: newfs • Structure of a typical file system(unixlike) – Superblock • Describes layout of i-Nodes, bitmaps, blocks, root directory, etc. – i-Nodes • Describes a file/dir – Link counter, list of blocks, size, etc. (owner, permission, …) – Block Allocation Bitmap – Data blocks

mkfs(cont. ) • Creates the file system – – Set up the Superblock Flag all i-Nodes as type free (f) Zero out the Alloc Bitmap Create the root directory • • Flag an i-Node as type directory (D) Store it’s number in Superblock Alloc one block in the bitmap Initialize the ‘. ’ and ‘. . ’ directory entries and set the others as being INVALID Block Bootblock Super Alloc Kernel Block i-Nodes Bitmap* Procs Data Blocks

Super. Block • Describes the layout of the file system – – – – – signature – A magic number size – Total size of fs in blocks root_inode – The root directory inode_start – 1 st block for i-Nodes inode_blocks – Total i-Nodes blocks bitmap_start – 1 st block of bitmap_blocks – Total bitmap blocks alloc_start – 1 st block for data num_blocks – Total data blocks free_blocks – Count of unallocated blocks

Bitmap • Bitmap – Indicates which data blocks are free, and which are used – Maps the data blocks only (not i-Nodes)

I-node • Describes a file/directory – – – type – File, directory opens – Count of open file handles links – Count of links to this i-Node size – Total byte count direct[10] – List of data blocks indirect[3] – List of indirect blocks • Advantages – Fast access to small files – Supports very large files – Support sparse files

Hints • Break this project into two components expansion of the main server itself, and implementation of the underlying disk • Within the file system: – Start with the creation of files and try to get that working properly; – Then build directories on top of that mechanism; – Finally work on directory listing support. • For files: direct blocks first

More Hints • Keep the high level interface as much as possible – By having wrappers that can either call the real call or your replacement, you'll be able to do testing quickly and efficiently • Standard interface: – Stat(const char *pathname, struct stat *buf)

stat • Returns statistics for given a filename – – – inode. No – The file’s i-Node number Type – File or directory Opens – The open count Size – The byte count to EOF num. Blocks – Number of allocated blocks

Reference • Check the manual page of the function calls – What they do – What’s their interface – What’s the most useful parameters

Wrap Up • • Fun again? Probably…. Not too difficult. But a little bit more coding. Questions?