DISK STORAGE BASIC FILE STRUCTURES HASHING AND MODERN
DISK STORAGE, BASIC FILE STRUCTURES, HASHING, AND MODERN STORAGE ARCHITECTURES 160517 tabata 1
outline • 16. 1 Introduction • 16. 1. 1 Memory Hierarchies and Storage Devices 1 • 16. 1. 2 Storage Organization of Databases • 16. 2 Secondary Storage Devices • • 16. 2. 1 Hardware Description of Disk Devices 16. 2. 2 Making Data Access More Efficient on Disk 16. 2. 3 Solid. State Device (SSD) Storage 16. 2. 4 Magnetic Tape Storage Devices • 16. 3 Buffering of Blocks • 16. 3. 1 Buffer Management • 16. 3. 2 Buffer Replacement Strategies: • 16. 4 Placing File Records on Disk • • • 16. 4. 1 Records and Record Types 16. 4. 2 Files, Fixed-Length Records, and Variable-Length Records 16. 4. 3 Record Blocking and Spanned versus Unspanned Records 16. 4. 4 Allocating File Blocks on Disk 16. 4. 5 File Headers 2
outline 16. 5 Operations on Files 16. 6 Files of Unordered Records (Heap Files) 16. 7 Files of Ordered Records (Sorted Files) 16. 8 Hashing Techniques • 16. 8. 1 Internal Hashing • 16. 8. 2 External Hashing for Disk Files • 16. 8. 3 Hashing Techniques That Allow Dynamic File Expansion • 16. 9 Other Primary File Organizations • 16. 9. 1 Files of Mixed Records • 16. 9. 2 B-Trees and Other Data Structures as Primary Organization • 16. 10 Parallelizing Disk Access Using RAID Technology • 16. 10. 1 Improving Reliability with RAID • 16. 10. 2 Improving Performance with RAID • 16. 10. 3 RAID Organizations and Levels • • 3
outline • 16. 11 Modern Storage Architectures • • • 16. 11. 1 Storage Area Networks 16. 11. 2 Network-Attached Storage 16. 11. 3 i. SCSI and Other Network-Based Storage Protocols 16. 11. 4 Automated Storage Tiering 16. 11. 5 Object-Based Storage • 16. 12 Summary 4
outline • 16. 1 Introduction • 16. 2 Secondary Storage Devices • 16. 3 Buffering of Blocks • 16. 4 Placing File Records on Disk • 16. 5 Operations on Files • 16. 6 Files of Unordered Records (Heap Files) • 16. 7 Files of Ordered Records (Sorted Files) • 16. 8 Hashing Techniques • 16. 9 Other Primary File Organizations • 16. 10 Parallelizing Disk Access Using RAID Technology • 16. 11 Modern Storage Architectures • 16. 12 Summary 5
outline • 16. 1 Introduction • 16. 2 Secondary Storage Devices • 16. 3 Buffering of Blocks • 16. 4 Placing File Records on Disk • 16. 5 Operations on Files • 16. 6 Files of Unordered Records (Heap Files) • 16. 7 Files of Ordered Records (Sorted Files) • 16. 8 Hashing Techniques • 16. 9 Other Primary File Organizations • 16. 10 Parallelizing Disk Access Using RAID Technology • 16. 11 Modern Storage Architectures • 16. 12 Summary 6
16. 1. 1 Memory Hierarchies and Storage Devices • 一時記憶装置 • キャッシュメモリ(cache memory): SRAM(Static RAM) • プログラム実行の高速化(speed up execution of program) • メインメモリ(main memory): DRAM(Dynamic RAM) • プログラム命令とデータの保持 • SRAMよりも安いが遅くて揮発性である 9
outline • 16. 1 Introduction • 16. 2 Secondary Storage Devices • 16. 3 Buffering of Blocks • 16. 4 Placing File Records on Disk • 16. 5 Operations on Files • 16. 6 Files of Unordered Records (Heap Files) • 16. 7 Files of Ordered Records (Sorted Files) • 16. 8 Hashing Techniques • 16. 9 Other Primary File Organizations • 16. 10 Parallelizing Disk Access Using RAID Technology • 16. 11 Modern Storage Architectures • 16. 12 Summary 16
Disk Pack 20
Disk Pack 21
Interfacing Disk Drives to Computer Systems • Seek time • Disk controllerがread/write headを正しいtrackに配置するのにかかる時間。 • Rotational delay(latency) • Magnetic headにblockの先頭が回転してくるまでの時間。 • Block transfer time • データを送るのにかかる時間 • 上記の三つの時間を合計がblockを見つけてから送るまでの時間。 • Seek time, Rotational delay > Block transfer time • 同じtrackやcylinderに連続したブロックをまとめる。 • 早くなる • bulk transfer rate • 連続したblockの送信にかかる時間の測定の手助けになる。 24
16. 2. 2 Making Data Access More Efficient on Disk • HDDにより効率的にアクセスする方法を扱う 1. Buffering of data 2. Proper organization of data on disk 3. Reading data ahead of request 4. Proper scheduling of I/O requests 5. Use of log disks to temporarily hold writes 6. Use of SSDs or flash memory for recovery purposes 25
outline • 16. 1 Introduction • 16. 2 Secondary Storage Devices • 16. 3 Buffering of Blocks • 16. 4 Placing File Records on Disk • 16. 5 Operations on Files • 16. 6 Files of Unordered Records (Heap Files) • 16. 7 Files of Ordered Records (Sorted Files) • 16. 8 Hashing Techniques • 16. 9 Other Primary File Organizations • 16. 10 Parallelizing Disk Access Using RAID Technology • 16. 11 Modern Storage Architectures • 16. 12 Summary 30
16. 3 Buffering of Blocks 32
outline • 16. 1 Introduction • 16. 2 Secondary Storage Devices • 16. 3 Buffering of Blocks • 16. 4 Placing File Records on Disk • 16. 5 Operations on Files • 16. 6 Files of Unordered Records (Heap Files) • 16. 7 Files of Ordered Records (Sorted Files) • 16. 8 Hashing Techniques • 16. 9 Other Primary File Organizations • 16. 10 Parallelizing Disk Access Using RAID Technology • 16. 11 Modern Storage Architectures • 16. 12 Summary 37
16. 4. 3 Record Blocking and Spanned versus Unspanned Records • Variable-length • SpannedでもunspannedでもOK • ただし、レコードが平均的に長い場合はspanningがよい。 • Fixed-length • Unspannedが良い。 • 開始位置が特定しやすくなる。 44
16. 4. 4 Allocating File Blocks on Disk • contiguous allocation と linked allocation • 連続したブロックを一つのclusterとする。 • このclusterをポインタでつなぐ。 • このclusterはfile segmentsやextentsと呼ばれる。 • indexed allocation • index blocksが実際のブロックへのポインタを持つ。 46
outline • 16. 1 Introduction • 16. 2 Secondary Storage Devices • 16. 3 Buffering of Blocks • 16. 4 Placing File Records on Disk • 16. 5 Operations on Files • 16. 6 Files of Unordered Records (Heap Files) • 16. 7 Files of Ordered Records (Sorted Files) • 16. 8 Hashing Techniques • 16. 9 Other Primary File Organizations • 16. 10 Parallelizing Disk Access Using RAID Technology • 16. 11 Modern Storage Architectures • 16. 12 Summary 48
16. 5 Operations on Files • File organization • Storageへの保存方法などの情報を参照する • Access method • 操作のセットを与える。 • File organizationにAccess methodに適用できる。ただし特定の access methodしか使えない場合もある。 55
outline • 16. 1 Introduction • 16. 2 Secondary Storage Devices • 16. 3 Buffering of Blocks • 16. 4 Placing File Records on Disk • 16. 5 Operations on Files • 16. 6 Files of Unordered Records (Heap Files) • 16. 7 Files of Ordered Records (Sorted Files) • 16. 8 Hashing Techniques • 16. 9 Other Primary File Organizations • 16. 10 Parallelizing Disk Access Using RAID Technology • 16. 11 Modern Storage Architectures • 16. 12 Summary 56
outline • 16. 1 Introduction • 16. 2 Secondary Storage Devices • 16. 3 Buffering of Blocks • 16. 4 Placing File Records on Disk • 16. 5 Operations on Files • 16. 6 Files of Unordered Records (Heap Files) • 16. 7 Files of Ordered Records (Sorted Files) • 16. 8 Hashing Techniques • 16. 9 Other Primary File Organizations • 16. 10 Parallelizing Disk Access Using RAID Technology • 16. 11 Modern Storage Architectures • 16. 12 Summary 59
16. 7 Files of Ordered Records (Sorted Files) • Ordered records • 特定のフィールド(ordering field)の値sortされている。 60
outline • 16. 1 Introduction • 16. 2 Secondary Storage Devices • 16. 3 Buffering of Blocks • 16. 4 Placing File Records on Disk • 16. 5 Operations on Files • 16. 6 Files of Unordered Records (Heap Files) • 16. 7 Files of Ordered Records (Sorted Files) • 16. 8 Hashing Techniques • 16. 9 Other Primary File Organizations • 16. 10 Parallelizing Disk Access Using RAID Technology • 16. 11 Modern Storage Architectures • 16. 12 Summary 64
Extendible Hashing 71
Dynamic hashing 73
16. 8. 3 Hashing Techniques That Allow Dynamic File Expansion • Linear hashing • ディレクトリを使用しないで動的なハッシュ法を実現する。 74
outline • 16. 1 Introduction • 16. 2 Secondary Storage Devices • 16. 3 Buffering of Blocks • 16. 4 Placing File Records on Disk • 16. 5 Operations on Files • 16. 6 Files of Unordered Records (Heap Files) • 16. 7 Files of Ordered Records (Sorted Files) • 16. 8 Hashing Techniques • 16. 9 Other Primary File Organizations • 16. 10 Parallelizing Disk Access Using RAID Technology • 16. 11 Modern Storage Architectures • 16. 12 Summary 77
16. 9. 2 B-Trees and Other Data Structures as Primary Organization • B-Tree • Record sizeとnumber of recordsが多くなかったら可能。 • B−TreeとはAVL木の各ノードが複数の値を持てるバージョン • Chapter 17で扱います。 79
outline • 16. 1 Introduction • 16. 2 Secondary Storage Devices • 16. 3 Buffering of Blocks • 16. 4 Placing File Records on Disk • 16. 5 Operations on Files • 16. 6 Files of Unordered Records (Heap Files) • 16. 7 Files of Ordered Records (Sorted Files) • 16. 8 Hashing Techniques • 16. 9 Other Primary File Organizations • 16. 10 Parallelizing Disk Access Using RAID Technology • 16. 11 Modern Storage Architectures • 16. 12 Summary 80
Striping 82
16. 10. 3 RAID Organizations and Levels • 複数のRAIDを組み合わせることも可能 • 0と 1の組み合わせ • 最小 4つのdiskでmirroringとstripingが可能になる。 • 他のRAID • RAID 1. 5, RAID 7, RAID-DP, RAID S or Parity RAID, Matrix RAID, RAID-K, RAID-Z, RAIDn, Linux MD RAID 10, IBM Serve. RAID 1 E, un. RAID • この本では扱いません 87
outline • 16. 1 Introduction • 16. 2 Secondary Storage Devices • 16. 3 Buffering of Blocks • 16. 4 Placing File Records on Disk • 16. 5 Operations on Files • 16. 6 Files of Unordered Records (Heap Files) • 16. 7 Files of Ordered Records (Sorted Files) • 16. 8 Hashing Techniques • 16. 9 Other Primary File Organizations • 16. 10 Parallelizing Disk Access Using RAID Technology • 16. 11 Modern Storage Architectures • 16. 12 Summary 88
16. 11 Modern Storage Architectures • 最近のstorage systemを扱う。 89
outline • 16. 1 Introduction • 16. 2 Secondary Storage Devices • 16. 3 Buffering of Blocks • 16. 4 Placing File Records on Disk • 16. 5 Operations on Files • 16. 6 Files of Unordered Records (Heap Files) • 16. 7 Files of Ordered Records (Sorted Files) • 16. 8 Hashing Techniques • 16. 9 Other Primary File Organizations • 16. 10 Parallelizing Disk Access Using RAID Technology • 16. 11 Modern Storage Architectures • 16. 12 Summary 95
- Slides: 95