The Three Cs of Misses Compulsory Misses The

The Three C’s of Misses • Compulsory Misses • The first time a memory location is accessed, it is always a miss • Also known as cold-start misses • Only way to decrease miss rate is to increase the block size • Capacity Misses • Occur when a program is using more data than can fit in the cache • Some misses will result because the cache isn’t big enough • Increasing the size of the cache solves this problem • Conflict Misses • Occur when a block forces out another block with the same index • Increasing Associtivity reduces conflict misses • Worst in Direct-Mapped, non-existent in Fully Associative 7. 5

The cost of a cache miss • For a memory access, assume: • 1 clock cycle to send address to memory This actually • 25 clock cycles for each DRAM access depends on the bus speed (clock cycle 2 ns, 50 ns access time) • 1 clock cycle to send each resulting data word • Miss access time (4 -word block) • 4 x (Address + access + sending data word) • 4 x (1 + 25 + 1) = 108 cycles for each miss 7. 2

Memory Interleaving 4 bytes CPU Bus Cache Bus 4 bytes Default Must finish accessing one word before starting the next access CPU Bus Cache Interleaving Begin accessing one word, and while waiting, start accessing other three words (pipelining) 1 (1+25+1)x 4 = 108 cycles 1 25 25 1 1 1 30 cycles Bus Bus Memory 3 Memory 2 Memory 1 Memory 0 Spread out addresses among the memories Requires 4 separate memories, each 1/4 size Interleaving works perfectly with caches Sophisticated DRAMs (EDO, SDRAM, etc. ) provide support for this 7. 2

Too little Memory • You’re running a huge program that requires 32 MB • Your PC has only 16 MB available. . . • Rewrite your program so that it implements overlays • Execute the first portion of code (fit it in the available memory) • When you need more memory. . . • Find some memory that isn’t needed right now • Save it to disk • Use the memory for the latter portion of code • So on. . . • The memory is to disk as registers are to memory • We’re using the disk as an extension of memory • Can we use the same techniques as caching? 7. 4

A Memory Hierarchy • Extend the hierarchy • Main memory acts like a cache for the disk • Cache: About $20/Mbyte <2 ns access time • 512 KB typical • Memory: About $0. 15/MBtye, 50 ns access time • 256 MB typical • Disk: About $0. 0015/MByte, 15 ms (15, 000 ns) access time • 40 GB typical CPU Registers Store Load or I-Fetch Cache Main Memory (DRAM) Disk 7. 4

Virtual Memory • Idea: Keep only the portions of a program (code, data) that are currently needed in Main Memory • Currently unused data is saved on disk, ready to be brought in when needed • Appears as a very large virtual memory (limited only by the disk size) • Advantages: • Programs that require large amounts of memory can be run (As long as they don’t need it all at once) • Multiple programs can be in virtual memory at once • Disadvantages: • The memory a program needs may all be on disk • The operating system has to manage virtual memory 7. 4

The Virtual Memory Concept Virtual Memory Space: All possible memory addresses (4 GB in 32 -bit systems) All that can be conceived. Virtual Memory Space Disk Swap Space Main Memory Disk Swap Space: Area on hard disk that can be used as an extension of memory. (Typically 100 MB) All that can be used. Main Memory: Physical memory. (Typically 64 MB) All that physically exists. 7. 4

The Virtual Memory Concept This address can be conceived of, but doesn’t correspond to any memory. Accessing it will produce an error. Virtual Memory Space Error Disk Swap Space Disk Address: 58984 Not in main memory Main Memory Physical Address: 883232 Disk Address: 322321 This address can be accessed. However, it currently is only on disk and must be read into main memory before being used. A table maps from its virtual address to the disk location. This address can be accessed immediately since it is already in memory. A table maps from its virtual address to its physical address. There will also be a back-up location on disk. 7. 4

The Process • The CPU deals with Virtual Addresses • Steps to accessing memory with a virtual address 1. Convert the virtual address to a physical address • A special table makes this easy • The table may indicate that the desired address is on disk, but not in physical memory • Read the location from the disk into memory (this may require moving something else out of memory to make room) 2. Do the memory access using the physical address • Check the cache first (note: cache uses only physical addresses) • Update the cache if needed 7. 4

Making Virtual Memory Work VM systems typically have a miss (page fault) rate of 0. 00001 0. 0001% • V. M. is like a cache system • Main memory is a cache for virtual memory • Differences • The miss penalty is huge (7, 000 cycles) • Increase the block size to be about 8 KB • Disk transfers have a large startup time, but data transfer is relatively fast after started • Blocks in V. M. are called pages • Even on misses, V. M. must provide info on the disk location • V. M. system must have an entry for all possible locations • When there’s a hit, the V. M. system provides the physical address in main memory, not the actual data • Saves room (one address rather than 8 KB of data) 7. 4

Example Virtual to Physical Mapping 31 (index) No tag - All entries are unique Virtual Page Number 13 12 Page Offset 0 • 4 GB (32 -bit) Virtual Address Space • 32 MB (25 -bit) Physical Address Space • 8 KB (13 -bit) page size (block size) Translation Note: may involve reading from disk 24 13 12 Physical Page Number Page Offset 0 • A 32 -bit virtual address is given to the V. M. hardware • The virtual page number (index) is derived from this by removing the page (block) offset • The Virtual Page Number is looked up in a page table • When found, entry is either: • The physical page number, if in memory • The disk address, if not in memory (a page fault) • If not found, the address is invalid 7. 4

Virtual Memory: 8 KB page size, 16 MB Mem Index 0 13 12 31 19 219=512 K Page offset 13 Virtual Address 4 GB / 8 KB = 512 K entries Virt. Pg. # V Phys. Page # 0 1 2. . . Disk Address . . . 512 K 11 23 13 12 0 Physical Address 7. 4

Virtual memory example System with 20 -bit V. A. , 16 KB pages, 256 KB of physical memory Page offset takes 14 bits, 6 bits for V. P. N. and 4 bits for P. P. N. Page Table: Virtual Page # (index) 0000001 000010 000011 000100 000101 000110 000111 Valid Bit 1 0 1 1 0 0 1 1 Physical Page #/ Disk address 1001 sector 5000. . . 0010 sector 4323… 1011 1010 sector xxxx. . . sector 1239. . . 1010 0001 Access to: 0000 1100 1010 PPN = 0010 Physical Address: 00 1000 1100 1010 Access to: 0001 1001 0011 1100 0000 PPN = Page Fault to sector 1239. . . Pick a page to “kick out” of memory (use LRU). Read data from sector 1239 Assume LRU is VPN 000101 for this example. into PPN 1010 7. 4

Virtual Memory Issues • A slot in the page table for every possible virtual address? • With 8 KB (213) pages, there are 2(32 -13) (512 K) entries for a 32 -bit V. A. • Each entry takes around 15 bits (round up to 16 bits…) That’s 1 MB for the page table. • Solutions • Put the page table itself in main memory rather than on the CPU chip • Make the page table only big enough for the amount of memory being used 7. 4

Write issues • Write Though - Update both disk and memory • + Easy to implement • - Requires a write buffer • - Requires a separate disk write for every write to memory • - A write miss requires reading in the page first, then writing back the single word • Write Back - Write only to main memory. Write to the disk only when block is replaced. • + Writes are fast • + Multiple writes to a page are combined into one disk write • - Must keep track of when page has been written (dirty bit) • - A Read miss may cause page to be replaced and written back 7. 5

Individual Space • Every process wants its own space • Ideally, it would like the entire computer to itself • Sharing the computer’s memory creates problems • Sometimes a program will be at location 4000, sometimes at location 5888820, etc. • Use Virtual Memory to fool each process to thinking that it starts at location 0 • CPU uses virtual addresses - Start the program at virtual page 0, even if it’s not physical page 0 • Each process must have an individual page table to make this work • Every virtual page 0 must point to a different physical page, and so on 7. 4

Example Page Table for process A: VPN (index) 0000001 000010 000011 000100 000101 000110 000111 Valid Bit 1 0 1 PPN/ Disk address 1001 sector 5000. . . 1100 sector 4323… 1011 0101 sector 1239. . . 0001 Page Table for process B: VPN (index) 0000001 000010 000011 000100 000101 000110 000111 Valid Bit 1 1 0 0 0 1 PPN/ Disk address 0010 0000 0011 1100 sector 2311. . . sector 158. . . sector 555. . . 0100 Virtual page 000000 (process A) points to physical page 1001. Virtual page 000000 (process B) points to physical page 0010. The processes both can start at location 000000, but have different data. Note: Physical page 1100 is shared. 7. 4

Protection Using Virtual Memory • We want to protect different processes from each other • Can’t read or write to any other process’s memory, unless specifically allowed • Providing separate page tables fixes this problem • Each process can only access pages through its page table • As long as the page table doesn’t point to pages belonging to other processes, no problem • Since only the OS can write the page tables, the system is safe 7. 4

Protection Example Page Table for process A: VPN (index) 0000001 000010 000011 000100 000101 000110 000111 Valid Bit 1 0 1 PPN/ Disk address 1001 sector 5000. . . 1100 sector 4323… 1011 0101 sector 1239. . . 0001 Page Table for process B: VPN (index) 0000001 000010 000011 000100 000101 000110 000111 Valid Bit 1 1 0 0 0 1 PPN/ Disk address 0010 0000 0011 1100 sector 2311. . . sector 158. . . sector 555. . . 0100 How can process A access process B’s V. P. 000010? None of process A’s V. P. point to Physical page 0011 - Impossible to to access it! Note: Since physical page 1100 is shared, protection is violated. 7. 4

Shooting Ourselves in the Foot • Virtual Memory Access • Look up page number in page table • Access memory • Each memory access becomes two accesses • Even for addresses stored in the cache • Solution: Cache the page table entries in a special cache • The Translation Lookaside Buffer (TLB) is just a cache that holds recently accessed page table entries • A TLB hit means that we don’t have to actually look in the page table 7. 4

TLB Design • We want the TLB to have a high hit rate • Fortunately, pages are huge, providing super-high locality • TLB usually only has a small number of entries (i. e. 64) and is fully-associative • Typical hit rates are 98. 0 to 99. 9% • The TLB should store anything needed from the page table • Physical page number • Valid bit, Dirty bit • Warning: TLB can violate protection after a process switch • Flush the TLB on each process switch 7. 4

Virtual Memory Benefits • Virtual Memory frees the programmer from many issues • Large programs can run in smaller systems • It doesn’t matter what else is running on the system, all programs start at a virtual address of zero and can access the entire address space • Virtual memory protects different processes from each other 7. 4

Evidence of Virtual Memory at Work • Thrashing • If a program is just too big, it will constantly page fault to read in new pages (and throw out ones it needs) • Paging Out • If a program has been sitting idle for a long time, it is likely that it will be completely paged out to disk • When you return to the program, it will start out slow as it pages all of the memory back in • Loading • Bringing in a new program may require writing pages for an old one out to disk 7. 4

• • • Three processes A, B and C use the following address translation tables in a system with a 20 -bit address bus and a page size of 8 KBytes. For the following accesses determine whethere is a) A fault and what kind b) A valid memory access c) A valid fetch from hard disk to memory and the new contents of the address translation table i) Process A: 103 B 4 ii) Process B: 12789 iii) Process A: 03278 iv) Process C: 0 A 1 A 2 Are there any other potential memory protection faults with the address translation table contents? Process A Process B Process C VPN index V PPN/ disk addr Last used (time) 000000 1 0010 10 000000 0 Sector 3248 90 000000 1 0100 2 000001 1 0001 30 000001 0 Sector 4352 77 000001 0 Sector 2814 48 000010 0 Sector 3452 52 000010 1 0100 32 000010 1 0000 23 000011 1 0110 28 000011 1 0101 56 000011 1 0011 31 000100 0 Sector 2240 43 000100 1 0011 5 000100 0 Sector 1112 68