14 332 331 Computer Architecture and Assembly Language
14: 332: 331 Computer Architecture and Assembly Language Spring 2005 Week 13 Basics of Cache [Adapted from Dave Patterson’s UCB CS 152 slides and Mary Jane Irwin’s PSU CSE 331 slides] 331 Lec 20. 1 Spring 2005
Head’s Up q This week’s material l Basics of caches - Reading assignment – PH 7. 2 q Reminders 331 Lec 20. 2 Spring 2005
Review: A Typical Memory Hierarchy q By taking advantage of the Principle of Locality: l Present the user with as much memory as is available in the cheapest technology at the access speed offered by the fastest technology. On-Chip Components Control Instr Data Cache 2 ns 10 ns 50 ns 1, 000 ns 128 B 64 KB 256 KB 4 GB TB’s Cost: 331 Lec 20. 3 ITLB DTLB 1 ns Reg. File Size: Secondary Memory (Disk) Second Level Cache (SRAM) Datapath Speed: e. DRAM highest Main Memory (DRAM) lowest Spring 2005
Review: Principle of Locality q Temporal Locality l q Spatial Locality l q Keep most recently accessed data items closer to the processor Move blocks consisting contiguous words upper levels Hit Time << Miss Penalty l To Processor Upper Level Memory From Processor Lower Level Memory of to the Blk X Blk Y Hit: data appears in some block in the upper level (Blk X) - Hit Rate: the fraction of accesses found in the upper level - Hit Time: Time to access the upper level = RAM access time + Time to determine hit/miss l Miss: data needs to be retrieve from a lower level block (Blk Y) - Miss Rate = 1 - (Hit Rate) - Miss Penalty: Time to replace a block in the upper level with a block from the lower level + Time to deliver this block to the processor q In general, Average Access Time: l = Hit Time + Miss Penalty x Miss Rate 331 Lec 20. 4 Spring 2005
Review: How is the Hierarchy Managed? q registers <-> memory l q cache <-> main memory l q 331 Lec 20. 5 by compiler (programmer? ) by the hardware main memory <-> disks l by the hardware and operating system (virtual memory) l by the programmer (files) Spring 2005
Cache q Two questions to answer (in hardware): l l q Q 1: How do we know if a data item is in the cache? Q 2: If it is, how do we find it? First method: l Direct mapped - For each item of data at the lower level, there is exactly one location in the cache where it might be (i. e. , lots of items at the lower level share locations in the upper level) l l 331 Lec 20. 6 Block size is one word of data Mapping: (word address) modulo (# of words in the cache) Spring 2005
Caching: A Simple First Example Cache Valid Tag Data 00 01 10 11 Q 1: Is it there? Compare the cache tag to the high order 2 memory address bits to tell if the memory block is in the cache 331 Lec 20. 7 0000 0001 0010 0011 0100 0101 0110 0111 1000 1001 1010 1011 1100 1101 1110 1111 Main Memory Q 2: How do we find it? Use low order 2 memory address bits to determine which cache block (i. e. , modulo the number of blocks in the cache) Spring 2005
Direct Mapped Cache q Consider the main memory reference string Start with an empty cache - all blocks marked as not valid 0 miss 00 01 Mem(0) 1 miss 00 Mem(0) 00 Mem(1) 4 miss 00 00 331 Lec 20. 8 4 Mem(0) Mem(1) Mem(2) Mem(3) 0 1 2 3 4 14 2 miss 00 00 00 3 hit 01 00 00 00 Mem(4) Mem(1) Mem(2) Mem(3) Mem(0) Mem(1) Mem(2) 4 01 00 00 00 3 miss 00 00 hit Mem(4) Mem(1) Mem(2) Mem(3) Mem(0) Mem(1) Mem(2) Mem(3) 14 miss 01 00 11 00 00 Mem(4) Mem(1) Mem(2) 14 Mem(3) Spring 2005
Another Reference String Mapping q Now consider the main memory reference string Start with an empty cache - all blocks marked as not valid 0 miss 00 00 01 l 331 Lec 20. 9 Mem(0) 0 miss 0 Mem(4) 01 00 0 4 0 4 4 miss 4 Mem(0) 00 01 0 miss 0 Mem(4) 01 00 4 miss Mem(0)4 4 miss 4 Mem(0) Ping pong effect due to conflict misses - two memory locations that map into the same cache block Spring 2005
Sources of Cache Misses q Compulsory (cold start or process migration, first reference): first access to a block l l q Conflict (collision): l Multiple memory locations mapped to the same cache location l Solution 1: increase cache size Solution 2: increase associativity l q “Cold” fact of life, not a whole lot you can do about it If you are going to run “billions” of instruction, Compulsory Misses are insignificant Capacity: l l 331 Lec 20. 10 Cache cannot contain all blocks accessed by the program Solution: increase cache size Spring 2005
MIPS Direct Mapped Cache Example q One word/block, cache size = 1 K words 31 30 Hit Tag . . . 13 12 11 20 . . . 2 1 0 Byte offset Data 10 Index Valid Tag Data 0 1 2. . . 1021 1022 1023 20 331 Lec 20. 11 32 Spring 2005
Handling Cache Misses q Handling hit is trivial q Handling misses needs to stall the processor q Upon an instruction cache miss l Send the original PC value (current PC – 4) to the memory l Instruct main memory to perform a read and wait for the memory to complete its access l Write the cache entry, putting the data from memory in the data portion of the entry, writing the upper bits of the address (from the ALU) into the tag field, and turning the valid bit on Restart the instruction execution at the first step, which will re-fetch the instruction, this time finding it in the cache l q Similar for data cache miss 331 Lec 20. 12 Spring 2005
Handling Writes q The cache and memory are inconsistent when their values (of the same data) are different q A simple solution: write through q l Write to both the cache and the memory at the same time l Poor performance. Every store instruction needs to stall the processor (a memory access can take 100 CPU cycles) Alternative: write back l 331 Lec 20. 13 Write to the cache; write to the memory when the cache block is replaced later. Spring 2005
Cache Summary q The Principle of Locality: l Program likely to access a relatively small portion of the address space at any instant of time - Temporal Locality: Locality in Time - Spatial Locality: Locality in Space q Three Major Categories of Cache Misses: l Compulsory Misses: sad facts of life. Example: cold start misses l Conflict Misses: increase cache size and/or associativity Nightmare Scenario: ping pong effect! Capacity Misses: increase cache size l q Cache Design Space l l l total size, block size, associativity (replacement policy) write-hit policy (write-through, write-back) write-miss policy (write allocate, write buffers) 331 Lec 20. 14 Spring 2005
Memory Systems that Support Caches q The off-chip interconnect and memory architecture can affect overall system performance in dramatic ways. One word wide organization on-chip CPU (one word wide bus and one word wide memory) q 32 -bit data & 32 -bit addr per cycle Assume Cache 1. 1 clock cycle (2 ns) to send the address bus 2. 25 clock cycles (50 ns) for DRAM cycle time, 8 clock cycles (16 ns) access time Memory 3. 1 clock cycle (2 ns) to return a word of data q Memory-Bus to Cache bandwidth l 331 Lec 20. 15 number of bytes accessed from memory and transferred to cache/CPU per clock Spring 2005 cycle
One Word Wide Memory Organization q on-chip CPU Cache bus If the block size is one word, then for a memory access due to a cache miss, the pipeline will have to stall the number of cycles required to return one data word from memory cycle to send address 1 25 1 cycles to read DRAM 27 total clock cycles miss penalty cycle to return data Memory q Number of bytes transferred per clock cycle (bandwidth) for a single miss is 4/27 = 0. 148 bytes per clock 331 Lec 20. 16 Spring 2005
One Word Wide Memory Organization, con’t q on-chip What if the block size is four words? 1 cycle to send 1 st address 4 x 25 = 100 cycles to read DRAM 1 cycles to return last data word CPU 102 total clock cycles miss penalty Cache 25 cycles bus 25 cycles Memory 25 cycles q Number of bytes transferred per clock cycle (bandwidth) for a single miss is (4 x 4)/102 = 0. 157 bytes per clock 331 Lec 20. 17 Spring 2005
Interleaved Memory Organization q For a block size of four words on-chip CPU 1 cycle to send 1 st address 25 + 3 = 28 cycles to read DRAM 1 cycles to return last data word Cache 30 total clock cycles miss penalty 25 cycles bus 25 cycles Memory bank 0 bank 1 bank 2 bank 3 25 cycles Number of bytes transferred per clock cycle (bandwidth) for a single miss is q (4 x 4)/30 = 0. 533 bytes per clock 331 Lec 20. 18 Spring 2005
DRAM Memory System Summary q Its important to match the cache characteristics l q with the DRAM characteristics l q caches access one block at a time (usually more than one word) use DRAMs that support fast multiple word accesses, preferably ones that match the block size of the cache with the memory-bus characteristics l l 331 Lec 20. 19 make sure the memory-bus can support the DRAM access rates and patterns with the goal of increasing the Memory-Bus to Cache bandwidth Spring 2005
- Slides: 19