Disks Computer Center CS NCTU Outline q Interfaces

Disks

Computer Center, CS, NCTU Outline q Interfaces q Geometry q Add new disks • Installation procedure • Filesystem check • Add a disk q RAID • GEOM 2

Computer Center, CS, NCTU Disk Interfaces Expensive! • Small Computer Systems Interface SCSI Card ~ 10 k • High performance and reliability Low Price! q IDE (or ATA) q SCSI • Integrated Device Electronics (or Advanced Technology Attachment) • Low cost • Become acceptable for enterprise with the help of RAID technology q SATA • Serial ATA Enhancement q SAS • Serial Attached SCSI q USB • Universal Serial Bus • Convenient to use 3 Speeds up!

Computer Center, CS, NCTU Disk Interfaces – ATA & SATA q ATA (AT Attachment) • ATA 2 Ø PIO, DMA Ø LBA (Logical Block Addressing) • ATA 3, Ultra DMA/33/66/100/133 • ATAPI (ATA Packet Interface) Ø CDROM, TAPE • Only one device can be active at a time Ø SCSI support overlapping commands, command queuing, scattergather I/O Primary Master (0)/Slave(1) • Master-Slave • 40 -pin ribbon cable Secondary Master(2)/Slave(3) q SATA 4 • Serial ATA • SATA-1 1. 5 Gbit/s, SATA-2 3 Gbit/s, SATA-3 6 GBit/s • SATA 3. 1, SATA 3. 2 16 Gbit/s, SATA 3. 3, e. SATA, m. SATA

Computer Center, CS, NCTU 5 Disk Interfaces – ATA & SATA Interfaces q ATA interface and it’s cable Data Power q SATA interface and it’s cable Data Power

Computer Center, CS, NCTU 6 Disk Interfaces – USB q IDE/SATA to USB Converters

Computer Center, CS, NCTU 7 Disk Geometry (1) q sector • Individual data block q track • circle q cylinder • circle on all platters q Position • CHS: Cylinder, Head (0, 1, …), Sector Like CDs. .

Computer Center, CS, NCTU Disk Geometry (2) q 40 G HD • 4866 cylinders, 255 heads • 63 sectors per track, 512 bytes per sector • 512 * 63 * 4866 * 255 = 40, 024, 212, 480 bytes G M K • 1 KB = 1024 bytes • 1 MB = 1024 KB = 1, 048, 576 bytes • 1 GB = 1024 MB = 1, 073, 741, 824 bytes Why? • 40, 024, 212, 480 / 1, 073, 741, 824 ≒ 37. 275 GB 10^3 vs. 2^10… 8

Disk Installation Procedure (in BSD…)

Computer Center, CS, NCTU Disk Installation Procedure (1) q The procedure involves the following steps: • Connecting the disk to the computer Ø IDE: master/slave Ø SATA Ø SCSI: ID, terminator Ø power • Creating device files Ø Auto created by devfs • Formatting the disk Ø Low-level format Please do it offline… Meta data a HD data Format (metadata + data) vs. fast format (data only) – Manufacturer diagnostic utility – Kill address information and timing marks on platters – Repair bad sectors mark the bad sectors and don’t use them! 10

Computer Center, CS, NCTU Disk Installation Procedure (2) • Partitioning (and Labeling) the disk) Ø Allow the disk to be treated as a group of independent data area Ø e. g. root, home, swap partitions Ø Former Suggestions: – /var, /tmp separate partition (for backup issue) – Make a copy of root filesystem for emergency • Establishing logical volumes Ø Combine multiple partitions into a logical volume Ø Related to RAID Ø Software RAID technology – GEOM: geom(4)、geom(8) – ZFS: zpool(8)、zfs(8)、zdb(8) 11

Computer Center, CS, NCTU 12 Disk Installation Procedure (3) • Creating UNIX filesystems within disk partitions Ø Use “newfs” to install a filesystem for a partition Ø Establish all filesystem components – – – A set of inode storage cells A set of data blocks A set of superblocks A map of the disk blocks in the filesystem A block usage summary

Computer Center, CS, NCTU Disk Installation Procedure (4) Ø Superblock contents – – – The length of a disk block Inode table’s size and location Disk block map Usage information Other filesystem’s parameters Ø sync – The sync() system call forces a write of dirty (modified) buffers in the block buffer cache out to disk. – The sync utility can be called to ensure that all disk writes have been completed before the processor is halted in a way not suitably done by reboot(8) or halt(8). 13

Computer Center, CS, NCTU Disk Installation Procedure (5) • mount Ø Bring the new partition to the filesystem tree Ø mount point can be any directory (empty) Ø # mount /dev/ad 1 s 1 e /home 2 ad 1 • Setting up automatic mounting s 1 Ø Automount at boot time – /etc/fstab – % mount -t ufs /dev/ad 2 s 1 a /backup – % mount -t cd 9600 -o ro, noauto /dev/acd 0 c /cdrom liuyh@NASA: /etc> cat fstab # Device Mountpoint /dev/ad 0 s 1 b none /dev/ad 2 s 1 b none /dev/ad 0 s 1 a / /dev/acd 0 /cdrom /dev/ad 2 s 1 a /backup csduty: /bsdhome 14 Fstype swap ufs cd 9660 ufs nfs Options sw sw rw ro, noauto rw, noauto Mount from the network; talk about it in “NFS”… d e f partition, newfs Mount CD Also for ISO image file Dump Pass# 0 0 1 1 0 0 2 2 0 0 Usually: 2, 1 for root; No write = 0

Computer Center, CS, NCTU 15 Disk Installation Procedure (6) • Setting up swapping on swap partitions Ø swapon, swapoff, swapctl – # swapon –a » mount all partitions for swap usage Ø swapinfo, pstat nctucs [~] -wangth- swapinfo Device 1 K-blocks Used Avail Capacity /dev/da 0 p 2 2097152 42772 2054380 2%

Computer Center, CS, NCTU fsck – check and repair filesystem (1) q System crash will cause • Inconsistency between memory image and disk contents q fsck • Examine all local filesystem listed in /etc/fstab at boot time. (fsck -p) • Automatically correct the following damages: Ø Unreferenced inodes Ø Inexplicably large link counts Ø Unused data blocks not recorded in block maps Ø Data blocks listed as free but used in file Ø Incorrect summary information in the superblock Ø fsck(8)、fsck_ffs(8) Ø ffsinfo(8): dump metadata Check if filesystem is clean… 1: clean (ro) 0: dirty (rw) 16

Computer Center, CS, NCTU fsck – check and repair filesystem (2) q Run fsck in manual to fix serious damages • • • Blocks claimed by more than one file Blocks claimed outside the range of the filesystem Link counts that are too small Blocks that are not accounted for Directories that refer to unallocated inodes Other errors q fsck will suggest you the action to perform • Delete, repair, … 17 No guarantee on fully recover you HD…

Computer Center, CS, NCTU Adding a disk to Free. BSD (1) 1. Check disk connection > ada 3: 238475 MB <Hitachi HDS 722525 VLAT 80 V 36 OA 6 MA> at ata 1 -slave UDMA 100 2. Use gpart(8) to create a partition on the new HD > > 3. 4. # gpart create -s GPT ada 3 # gpart add -t freebsd-ufs -a 1 M ada 3 # newfs -U /dev/ada 3 p 1 Make mount point and mount it > > > 4. Line, speed Use newfs(8) to construct new UFS file system > 18 Look system boot message # mkdir /home 2 # mount -t ufs /dev/ada 3 p 1 /home 2 # df Edit /etc/fstab • https: //www. freebsd. org/doc/handbook/disks-adding. html

Computer Center, CS, NCTU Adding a disk to Free. BSD (2) q If you forget to enable soft-update when you add the disk • • % umount /home 2 % tunefs -n enable /dev/ada 3 p 1 % mount -t ufs /dev/ada 3 p 1 /home 2 % mount /dev/ada 0 p 2 on / (ufs, local, soft-updates) /dev/ada 1 p 1 on /home (ufs, local, soft-updates) procfs on /proc (procfs, local) /dev/ada 3 p 1 on /home 2 (ufs, local, soft-updates) • https: //www. freebsd. org/doc/handbook/configtuning-disk. html 19

GEOM Modular Disk Transformation Framework

Computer Center, CS, NCTU GEOM – (1) q Support • • • ELI – geli(8): cryptographic GEOM class JOURNAL – gjournal(8): journaled devices LABEL – glabel(8): disk labelization MIRROR – gmirror(8): mirrored devices STRIPE – gstripe(8): striped devices … Journalize (logs) before write Software RAID 1 Software RAID 0 • http: //www. freebsd. org/doc/handbook/geom. html 21

Computer Center, CS, NCTU GEOM – (2) q GEOM framework in Free. BSD • Major RAID control utilities • Kernel modules (/boot/kernel/geom_*) • Name and Prodivers devices Logical volumes Ø “manual” or “automatic” Ø Metadata in the last sector of the providers q Kernel support • {glabel, gmirror, gstripe, g*} load/unload Ø device GEOM_* in kernel config Ø geom_*_enable="YES" in /boot/loader. conf 22 (1) On demand load/unload kernel modules • load automatically at booting (2) Build-in kernel and recompile

Computer Center, CS, NCTU GEOM – (3) q LABEL Why use it? bundle by name instead of bundle by provider • Used for GEOM provider labelization • Kernel Ø device GEOM_LABEL Ø geom_label_load="YES" • glabel (for new storage) e. g. ad 0 s 1 d usr glabel … Create permanent labels glabel create … Create transient labels Ø # glabel -v usr da 2 /dev/label/usr Ø # newfs /dev/label/usr Ø # mount /dev/label/usr Ø # glabel stop usr Stop using the name Ø # glabel clear da 2 Clear metadata on provider • UFS label (for an using storage) Ø # tunefs -L data /dev/da 4 s 1 a Ø # mount /dev/ufs/data /mnt/data 23 “data” is a name

Computer Center, CS, NCTU 24 GEOM – (4) q MIRROR • Kernel Ø device GEOM_MIRROR Ø geom_mirror_load="YES" • gmirror Ø # gmirror label -v -b round-robin data da 0 Ø # newfs /dev/mirror/data logical volume called “data”, using HD: da 0, … Ø # mount /dev/mirror/data /mnt Ø # gmirror insert data da 1 Add in HD Ø # gmirror forget data Kill inexist HDs Ø # gmirror insert data da 1 Ø # gmirror stop data Ø # gmirror clear da 0

Computer Center, CS, NCTU 25 GEOM – (5) q STRIPE • Kernel Ø device GEOM_STRIPE Ø geom_stripe_load="YES" • gstripe Ø # gstripe label -v -s 131072 data da 0 da 1 da 2 da 3 Create logical volume “data”, Ø # newfs /dev/stripe/data which stripe da 0~da 3 HDs Ø # mount /dev/stripe/data /mnt Ø # gstripe stop data Ø # gstripe clear da 0

RAID

Computer Center, CS, NCTU RAID – (1) q Redundant Array of Inexpensive Disks • A method to combine several physical hard drives into one logical unit e. g. HD 1, HD 2 D: in windows q Depending on the type of RAID, it has the following benefits: RAID 1 - RAID 0 • Fault tolerance • Higher throughput • Real-time data recovery - HD - RAID 0 - HD q RAID Level • RAID 0, 1, 0+1, 2, 3, 4, 5, 6 • Hierarchical RAID 0 27 RAID 1 RAID 0

Computer Center, CS, NCTU RAID – (2) q Hardware RAID • There is a dedicate controller to take over the whole business • RAID Configuration Utility after BIOS Ø Create RAID array, build Array q Software RAID Ø GEOM – CACHE、CONCAT、ELI、JOURNAL、LABEL、MIRROR、 MULTIPATH、NOP、PART、RAID 3、SHSEC、STRIPE、 VIRSTOR Ø ZFS – JBOD、STRIPE – MIRROR – RAID-Z、RAID-Z 2、RAID-Z 3 28

Computer Center, CS, NCTU RAID 0 (normally used) (500 GB+500 GB=1 TB) q Stripped data intro several disks q Minimum number of drives: 2 q Advantage e. g. HD 1 (500 GB), HD 2 (500 GB) D: in windows (1 TB) • Performance increase in proportional to n theoretically • Simple to implement parallel file io from/to different HDs q Disadvantage • No fault tolerance q Recommended applications • Non-critical data storage • Application requiring high bandwidth (such as video editing) 29

Computer Center, CS, NCTU 30 RAID 1 (normally used) (500 GB+500 GB=500 B) q Mirror data into several disks q Minimum number of drives: 2 q Advantage • 100% redundancy of data q Disadvantage • 100% storage overage • Moderately slower write performance q Recommended application Cause by double check mechanisms on data… • Application requiring very high availability (such as home)

Computer Center, CS, NCTU 31 RAID 0+1 (normally used) [(500 GB+500 GB)+(500 GB+500 GB)]=1 TB) q Combine RAID 0 and RAID 1 q Minimum number of drives: 4 RAID 1, RAID 1 Them RAID 0 above it

Computer Center, CS, NCTU RAID 2 q Hamming Code ECC Each bit of data word q Advantages: Read, check if correct, then read • "On the fly" data error correction q Disadvantages: • Inefficient • Very high ratio of ECC disks to data disks q Recommended Application • No commercial implementations exist / not commercially viable 32

Computer Center, CS, NCTU RAID 3 RAID 1 if two HDs Save parity q Parallel transfer with Parity q Minimum number of drives: 3 q Advantages: • Very high data transfer rate q Disadvantages: • Transaction rate equal to that of a single disk drive at best q Recommended Application • Any application requiring high throughput 33

Computer Center, CS, NCTU 34 RAID 4 q Similar to RAID 3 q RAID 3 V. S RAID 4 • Byte Level V. S Block Level • Block interleaving Block normally 512 bytes (4 k for WD HDs) Ø Small files (e. g. 4 k)

Computer Center, CS, NCTU RAID 5 (normally used) q Independent Disk with distributed parity blocks q Minimum number of drives: 3 Origin from RAID 3 q Advantage Parallel file I/O • Highest read data rate • Medium write data rate q Disadvantage • Disk failure has a medium impact on throughput • Complex controller design • When one disk failed, you have to rebuild the RAID array Can tolerate only 1 HD failure 35

Computer Center, CS, NCTU 36 RAID 6 (normally used) q Similar to RAID 5 q Minimum number of drives: 4 q 2 parity checks, 2 disk failures tolerable. Slower than RAID 5 because of storing 2 parities…