Lecture 4 FileSystem Interface Operating System Concepts Essentials

Lecture 4: File-System Interface Operating System Concepts Essentials – 2 nd Edition Silberschatz, Galvin and Gagne © 2013

File-System Interface n File Concept n Access Methods n Disk and Directory Structure n File Sharing n Protection Operating System Concepts Essentials – 2 nd Edition 10. 2 Silberschatz, Galvin and Gagne © 2013

File Concept n Contiguous logical address space n Types: l Data 4 numeric 4 character 4 binary l Program n Contents defined by file’s creator l Many types 4 Consider text file, source file, executable file Operating System Concepts Essentials – 2 nd Edition 10. 3 Silberschatz, Galvin and Gagne © 2013

File Attributes n Name – only information kept in human-readable form n Identifier – unique tag (number) identifies file within file system n Type – needed for systems that support different types n Location – pointer to file location on device n Size – current file size n Protection – controls who can do reading, writing, executing n Time, date, and user identification – data for protection, security, and usage monitoring n Information about files are kept in the directory structure, which is maintained on the disk Operating System Concepts Essentials – 2 nd Edition 10. 4 Silberschatz, Galvin and Gagne © 2013

File info Window on Mac OS X Operating System Concepts Essentials – 2 nd Edition 10. 5 Silberschatz, Galvin and Gagne © 2013

File Operations n File is an abstract data type n Create n Write – at write pointer location n Read – at read pointer location n Reposition within file - seek n Delete n Truncate n Open(Fi) – search the directory structure on disk for entry Fi, and move the content of entry to memory n Close (Fi) – move the content of entry Fi in memory to directory structure on disk Operating System Concepts Essentials – 2 nd Edition 10. 6 Silberschatz, Galvin and Gagne © 2013

Open Files n Several pieces of data are needed to manage open files: l Open-file table: tracks open files l File pointer: pointer to last read/write location, per process that has the file open l File-open count: counter of number of times a file is open – to allow removal of data from open-file table when last processes closes it l Disk location of the file: cache of data access information l Access rights: per-process access mode information Operating System Concepts Essentials – 2 nd Edition 10. 7 Silberschatz, Galvin and Gagne © 2013

Open File Locking n Provided by some operating systems and file systems l Similar to reader-writer locks l Shared lock similar to reader lock – several processes can acquire concurrently l Exclusive lock similar to writer lock Operating System Concepts Essentials – 2 nd Edition 10. 8 Silberschatz, Galvin and Gagne © 2013

File Types – Name, Extension Operating System Concepts Essentials – 2 nd Edition 10. 9 Silberschatz, Galvin and Gagne © 2013

Sequential-access File Operating System Concepts Essentials – 2 nd Edition 10. 10 Silberschatz, Galvin and Gagne © 2013

Access Methods n Sequential Access read next write next reset no read after last write (rewrite) n Direct Access – file is fixed length logical records read n write n position to n read next write next rewrite n n = relative block number Operating System Concepts Essentials – 2 nd Edition 10. 11 Silberschatz, Galvin and Gagne © 2013

Other Access Methods n Can be built on top of base methods n General involve creation of an index for the file n Keep index in memory for fast determination of location of data to be operated on n If too large, index (in memory) of the index (on disk) Operating System Concepts Essentials – 2 nd Edition 10. 12 Silberschatz, Galvin and Gagne © 2013

Directory Structure n A collection of nodes containing information about all files Directory Files F 1 F 2 F 3 F 4 Fn Both the directory structure and the files reside on disk Operating System Concepts Essentials – 2 nd Edition 10. 13 Silberschatz, Galvin and Gagne © 2013

Disk Structure n Disk can be subdivided into partitions n Disks or partitions can be RAID protected against failure n Entity containing file system known as a volume n Each volume containing file system also tracks that file system’s info in device directory or volume table of contents n As well as general-purpose file systems there are many special-purpose file systems, frequently all within the same operating system or computer Operating System Concepts Essentials – 2 nd Edition 10. 14 Silberschatz, Galvin and Gagne © 2013

A Typical File-system Organization Operating System Concepts Essentials – 2 nd Edition 10. 15 Silberschatz, Galvin and Gagne © 2013

Operations Performed on Directory n Search for a file n Create a file n Delete a file n List a directory n Rename a file Operating System Concepts Essentials – 2 nd Edition 10. 16 Silberschatz, Galvin and Gagne © 2013

Directory Organization The directory is organized logically to obtain n Efficiency – locating a file quickly n Naming – convenient to users l Two users can have same name for different files l The same file can have several different names n Grouping – logical grouping of files by properties, (e. g. , all Java programs, all games, …) Operating System Concepts Essentials – 2 nd Edition 10. 17 Silberschatz, Galvin and Gagne © 2013

File Sharing n Sharing of files on multi-user systems is desirable n Sharing may be done through a protection scheme n On distributed systems, files may be shared across a network n Network File System (NFS) is a common distributed file-sharing method n If multi-user system l User IDs identify users, allowing permissions and protections to be per-user Group IDs allow users to be in groups, permitting group access rights l Owner of a file / directory l Group of a file / directory Operating System Concepts Essentials – 2 nd Edition 10. 18 Silberschatz, Galvin and Gagne © 2013

File Sharing – Remote File Systems n Uses networking to allow file system access between systems l Manually via programs like FTP l Automatically, seamlessly using distributed file systems l Semi automatically via the world wide web n Client-server model allows clients to mount remote file systems from servers l Server can serve multiple clients l Client and user-on-client identification is insecure or complicated l NFS is standard UNIX client-server file sharing protocol l CIFS is standard Windows protocol Operating System Concepts Essentials – 2 nd Edition 10. 19 Silberschatz, Galvin and Gagne © 2013

Protection n File owner/creator should be able to control: l what can be done l by whom n Types of access l Read l Write l Execute l Append l Delete l List Operating System Concepts Essentials – 2 nd Edition 10. 20 Silberschatz, Galvin and Gagne © 2013

Windows 7 Access-Control List Management Operating System Concepts Essentials – 2 nd Edition 10. 21 Silberschatz, Galvin and Gagne © 2013

File System Implementation Operating System Concepts Essentials – 2 nd Edition Silberschatz, Galvin and Gagne © 2013

File System Implementation n File-System Structure n File-System Implementation n Directory Implementation n Allocation Methods n Efficiency and Performance n Recovery Operating System Concepts Essentials – 2 nd Edition 10. 23 Silberschatz, Galvin and Gagne © 2013

File-System Structure n File structure l Logical storage unit l Collection of related information n File system resides on secondary storage (disks) l Provided user interface to storage, mapping logical to physical l Provides efficient and convenient access to disk by allowing data to be stored, located retrieved easily n Disk provides in-place rewrite and random access l I/O transfers performed in blocks of sectors (usually 512 bytes) n File control block – storage structure consisting of information about a file n Device driver controls the physical device n File system organized into layers Operating System Concepts Essentials – 2 nd Edition 10. 24 Silberschatz, Galvin and Gagne © 2013

Layered File System Operating System Concepts Essentials – 2 nd Edition 10. 25 Silberschatz, Galvin and Gagne © 2013

File System Layers n Device drivers manage I/O devices at the I/O control layer l Given commands like “read drive 1, cylinder 72, track 2, sector 10, into memory location 1060” outputs low-level hardware specific commands to hardware controller n Basic file system given command like “retrieve block 123” translates to device driver n Also manages memory buffers and caches (allocation, freeing, replacement) l Buffers hold data in transit l Caches hold frequently used data n File organization module understands files, logical address, and physical blocks n Translates logical block # to physical block # n Manages free space, disk allocation Operating System Concepts Essentials – 2 nd Edition 10. 26 Silberschatz, Galvin and Gagne © 2013

File System Layers (Cont. ) n Logical file system manages metadata information l Translates file name into file number, file handle, location by maintaining file control blocks (inodes in UNIX) l Directory management l Protection n Layering useful for reducing complexity and redundancy, but adds overhead and can decrease performance. Translates file name into file number, file handle, location by maintaining file control blocks (inodes in UNIX) l Logical layers can be implemented by any coding method according to OS designer Operating System Concepts Essentials – 2 nd Edition 10. 27 Silberschatz, Galvin and Gagne © 2013

File System Layers (Cont. ) n Many file systems, sometimes many within an operating system l Each with its own format : l Unix has UFS, FFS; l Windows has FAT, FAT 32, NTFS as well as floppy, CD, DVD l Linux has more than 40 types, with extended file system ext 2 and ext 3 leading; plus distributed file systems, etc. ) Operating System Concepts Essentials – 2 nd Edition 10. 28 Silberschatz, Galvin and Gagne © 2013

File-System Implementation n We have system calls at the API level, but how do we implement their functions? l On-disk and in-memory structures n Boot control block contains info needed by system to boot OS from that volume l Needed if volume contains OS, usually first block of volume n Volume control block (superblock, master file table) contains volume details l Total # of blocks, # of free blocks, block size, free block pointers or array n Per-file File Control Block (FCB) contains many details about the file Operating System Concepts Essentials – 2 nd Edition 10. 29 Silberschatz, Galvin and Gagne © 2013

Directory Implementation n Linear list of file names with pointer to the data blocks l Simple to program l Time-consuming to execute 4 Linear search time 4 Could keep ordered alphabetically via linked list or use B+ tree n Hash Table – linear list with hash data structure l Decreases directory search time l Collisions – situations where two file names hash to the same location Operating System Concepts Essentials – 2 nd Edition 10. 30 Silberschatz, Galvin and Gagne © 2013

Allocation Methods - Contiguous n An allocation method refers to how disk blocks are allocated for files: n Contiguous allocation – each file occupies set of contiguous blocks l Best performance in most cases l Simple – only starting location (block #) and length (number of blocks) are required l Problems include finding space for file, knowing file size, external fragmentation. Operating System Concepts Essentials – 2 nd Edition 10. 31 Silberschatz, Galvin and Gagne © 2013

Contiguous Allocation n Mapping from logical to physical Q LA/512 R Block to be accessed = Q + starting address Displacement into block = R Operating System Concepts Essentials – 2 nd Edition 10. 32 Silberschatz, Galvin and Gagne © 2013

Allocation Methods - Linked n Linked allocation – each file a linked list of blocks l File ends at nil pointer l No external fragmentation l Each block contains pointer to next block l No compaction, external fragmentation l Free space management system called when new block needed l Improve efficiency by clustering blocks into groups but increases internal fragmentation l Locating a block can take many I/Os and disk seeks Operating System Concepts Essentials – 2 nd Edition 10. 33 Silberschatz, Galvin and Gagne © 2013

Allocation Methods – Linked (Cont. ) n FAT (File Allocation Table) variation l Beginning of volume has table, indexed by block number l Much like a linked list, but faster on disk and cacheable l New block allocation simple Operating System Concepts Essentials – 2 nd Edition 10. 34 Silberschatz, Galvin and Gagne © 2013

Linked Allocation Operating System Concepts Essentials – 2 nd Edition 10. 35 Silberschatz, Galvin and Gagne © 2013

File-Allocation Table Operating System Concepts Essentials – 2 nd Edition 10. 36 Silberschatz, Galvin and Gagne © 2013

Allocation Methods - Indexed n Indexed allocation l Each file has its own index block(s) of pointers to its data blocks n Logical view Operating System Concepts Essentials – 2 nd Edition 10. 37 Silberschatz, Galvin and Gagne © 2013

Example of Indexed Allocation Operating System Concepts Essentials – 2 nd Edition 10. 38 Silberschatz, Galvin and Gagne © 2013

Performance n Best method depends on file access type l Contiguous great for sequential and random n Linked good for sequential, not random n Declare access type at creation -> select either contiguous or linked n Indexed more complex l Single block access could require 2 index block reads then data block read l Clustering can help improve throughput, reduce CPU overhead Operating System Concepts Essentials – 2 nd Edition 10. 39 Silberschatz, Galvin and Gagne © 2013

Recovery n Consistency checking – compares data in directory structure with data blocks on disk, and tries to fix inconsistencies l Can be slow and sometimes fails n Use system programs to back up data from disk to another storage device (magnetic tape, other magnetic disk, optical) n Recover lost file or disk by restoring data from backup Operating System Concepts Essentials – 2 nd Edition 10. 40 Silberschatz, Galvin and Gagne © 2013

Log Structured File Systems n Log structured (or journaling) file systems record each metadata update to the file system as a transaction n All transactions are written to a log l A transaction is considered committed once it is written to the log (sequentially) n The transactions in the log are asynchronously written to the file system structures l When the file system structures are modified, the transaction is removed from the log n If the file system crashes, all remaining transactions in the log must still be performed n Faster recovery from crash, removes chance of inconsistency of metadata Operating System Concepts Essentials – 2 nd Edition 10. 41 Silberschatz, Galvin and Gagne © 2013

Mass-Storage Systems Operating System Concepts Essentials – 2 nd Edition Silberschatz, Galvin and Gagne © 2013

Overview of Mass Storage Structure n Magnetic disks provide bulk of secondary storage of modern computers l Drives rotate at 60 to 250 times per second l Transfer rate is rate at which data flow between drive and computer l Positioning time (random-access time) is time to move disk arm to desired cylinder (seek time) and time for desired sector to rotate under the disk head (rotational latency) l Head crash results from disk head making contact with the disk surface -- That’s bad n Disks can be removable n Drive attached to computer via I/O bus Operating System Concepts Essentials – 2 nd Edition 10. 43 Silberschatz, Galvin and Gagne © 2013

Moving-head Disk Mechanism Operating System Concepts Essentials – 2 nd Edition 10. 44 Silberschatz, Galvin and Gagne © 2013

Solid-State Disks n Nonvolatile memory used like a hard drive l Many technology variations n Can be more reliable than HDDs n More expensive per MB n Maybe have shorter life span n Less capacity n But much faster n Busses can be too slow -> connect directly to PCI for example n No moving parts, so no seek time or rotational latency Operating System Concepts Essentials – 2 nd Edition 10. 45 Silberschatz, Galvin and Gagne © 2013

Magnetic Tape n Was early secondary-storage medium n Relatively permanent and holds large quantities of data n Access time slow n Random access ~1000 times slower than disk n Mainly used for backup, storage of infrequently-used data, transfer medium between systems n Once data under head, transfer rates comparable to disk l 140 MB/sec and greater n 200 GB to 1. 5 TB typical storage Operating System Concepts Essentials – 2 nd Edition 10. 46 Silberschatz, Galvin and Gagne © 2013

Disk Scheduling n The operating system is responsible for using hardware efficiently — for the disk drives, this means having a fast access time and disk bandwidth n Minimize seek time n Seek time seek distance n Disk bandwidth is the total number of bytes transferred, divided by the total time between the first request for service and the completion of the last transfer Operating System Concepts Essentials – 2 nd Edition 10. 47 Silberschatz, Galvin and Gagne © 2013

Disk Scheduling (Cont. ) n There are many sources of disk I/O request l OS l System processes l Users processes n I/O request includes input or output mode, disk address, memory address, number of sectors to transfer n OS maintains queue of requests, per disk or device n Idle disk can immediately work on I/O request, busy disk means work must queue n Note that drive controllers have small buffers and can manage a queue of I/O requests n Several algorithms exist to schedule the servicing of disk I/O requests Operating System Concepts Essentials – 2 nd Edition 10. 48 Silberschatz, Galvin and Gagne © 2013

What is RAID? l Stands for Redundant Array of Independent Disks. l It’s a technology that enables greater levels of performance, reliability and/or large volumes when dealing with data. l How? ? By concurrent use of two or more ‘hard disk drives’. l How Exactly? ? Mirroring, Stripping (of data) and Error correction techniques combined with multiple disk arrays give you the reliability and performance. Operating System Concepts Essentials – 2 nd Edition 10. 49 Silberschatz, Galvin and Gagne © 2013

RAID flavors Commonly used ones: 1. RAID 0 2. RAID 1 3. RAID 5 4. RAID 1+0 Other types used…but rarely: RAID 2, 3, 4, 6, 50…… Operating System Concepts Essentials – 2 nd Edition 10. 50 Silberschatz, Galvin and Gagne © 2013

RAID 0 § It splits data among two or more disks. § Provides good performance. § Lack of data redundancy means there is no fail over support with this configuration. § In the diagram to the right, the odd blocks are written to disk 0 and the even blocks to disk 1 such that A 1, A 2, A 3, A 4, … would be the order of blocks read if read sequentially from the beginning. Operating System Concepts Essentials – 2 nd Edition 10. 51 Silberschatz, Galvin and Gagne © 2013

RAID 1 • RAID 1 is ‘data mirroring’. • Two copies of the data are held on two physical disks, and the data is always identical. • Twice as many disks are required to store the same data when compared to RAID 0. Operating System Concepts Essentials – 2 nd Edition 10. 52 Silberschatz, Galvin and Gagne © 2013

RAID 5 § RAID 5 is an ideal combination of good performance, good fault tolerance and high capacity and storage efficiency. § An arrangement of parity and CRC to help rebuilding drive data in case of disk failures. Operating System Concepts Essentials – 2 nd Edition 10. 53 Silberschatz, Galvin and Gagne © 2013

RAID 1+0 § Combines RAID 1 and RAID 0. § Which means having the pleasure of both - good performance and good failover handling. § Also called ‘Nested RAID’. Operating System Concepts Essentials – 2 nd Edition 10. 54 Silberschatz, Galvin and Gagne © 2013