Disadvantage of direct mapping The directmapped cache is

  • Slides: 16
Download presentation
Disadvantage of direct mapping § The direct-mapped cache is easy: indices and offsets can

Disadvantage of direct mapping § The direct-mapped cache is easy: indices and offsets can be computed with bit operators or simple arithmetic, because each memory address belongs in exactly one block Memory Address § However, this isn’t really flexible. If a program uses addresses 10, 110, . . . then each access will result in a cache miss and a load into cache block 10 § This cache has four blocks, but direct mapping might not let us use all of them § This can result in more misses than we might like 0000 0001 0010 0011 0100 0101 0110 0111 1000 1001 1010 1011 1100 1101 1110 1111 Index 00 01 10 11 1

A fully associative cache § A fully associative cache permits data to be stored

A fully associative cache § A fully associative cache permits data to be stored in any cache block, instead of forcing each memory address into one particular block — when data is fetched from memory, it can be placed in any unused block of the cache — this way we’ll never have a conflict between two or more memory addresses which map to a single cache block § In the previous example, we might put memory address 10 in cache block 10, and address 110 in block 11. Then subsequent repeated accesses to 10 and 110 would all be hits instead of misses. § If all the blocks are already in use, it’s usually best to replace the least recently used one, assuming that if it hasn’t used it in a while, it won’t be needed again anytime soon. 2

The price of full associativity § However, a fully associative cache is expensive to

The price of full associativity § However, a fully associative cache is expensive to implement. — Because there is no index field in the address anymore, the entire address must be used as the tag, increasing the total cache size. — Data could be anywhere in the cache, so we must check the tag of every cache block. That’s a lot of comparators! Address (32 bits) 32 Index Valid Tag (32 bits) Data . . Tag = = Hit = 3

Set associativity § An intermediate possibility is a set-associative cache — The cache is

Set associativity § An intermediate possibility is a set-associative cache — The cache is divided into groups of blocks, called sets — Each memory address maps to exactly one set in the cache, but data may be placed in any block within that set § If each set has 2 x blocks, the cache is an 2 x-way associative cache § Here are several possible organizations of an eight-block cache 1 -way associativity 8 sets, 1 block each Set 0 1 2 3 4 5 6 7 2 -way associativity 4 sets, 2 blocks each Set 0 1 2 3 Set 4 -way associativity 2 sets, 4 blocks each 0 1 4

Writing to a cache § Writing to a cache raises several additional issues §

Writing to a cache § Writing to a cache raises several additional issues § First, let’s assume that the address we want to write to is already loaded in the cache. We’ll assume a simple direct-mapped cache: Index V Tag Data . . . 110 Address Data . . . 1 11010 42803 . . . 1101 0110 42803 . . . § If we write a new value to that address, we can store the new data in the cache, and avoid an expensive main memory access [but inconsistent] HUGE problem in multiprocessors Mem[1101 0110] = 21763 Index V Tag Data . . . 110 Address 1 11010 21763 1101 0110 42803 . . . 5

Write-through caches § A write-through cache solves the inconsistency problem by forcing all writes

Write-through caches § A write-through cache solves the inconsistency problem by forcing all writes to update both the cache and the main memory Mem[1101 0110] = 21763 Index V Tag Data . . . 110 Address Data . . . 1 11010 . . . 21763 1101 0110 21763 . . . § This is simple to implement and keeps the cache and memory consistent § Why is this not so good? 6

Write-back caches § In a write-back cache, the memory is not updated until the

Write-back caches § In a write-back cache, the memory is not updated until the cache block needs to be replaced (e. g. , when loading data into a full cache set) § For example, we might write some data to the cache at first, leaving it inconsistent with the main memory as shown before — The cache block is marked “dirty” to indicate this inconsistency Mem[1101 0110] = 21763 Index V Dirty Tag Data . . . 110. . . 1 1 11010 21763 Address Data 1000 1110 1225 1101 0110 42803 . . . § Subsequent reads to the same memory address will be serviced by the cache, which contains the correct, updated data 7

Finishing the write back § We don’t need to store the new value back

Finishing the write back § We don’t need to store the new value back to main memory unless the cache block gets replaced § E. g. on a read from Mem[1000 1110], which maps to the same cache block, the modified cache contents will first be written to main memory Index V Dirty Tag Data . . . 110 1 1 11010 21763 Address Data 1000 1110 1225 1101 0110 21763 . . . § Only then can the cache block be replaced with data from address 142 Index V Dirty Tag Data . . . 110. . . 1 0 10001 1225 Address Data 1000 1110 1225 1101 0110 21763 . . . 8

Write misses § A second scenario is if we try to write to an

Write misses § A second scenario is if we try to write to an address that is not already contained in the cache; this is called a write miss § Let’s say we want to store 21763 into Mem[1101 0110] but we find that address is not currently in the cache Index V Tag Data . . . 110. . . Address Data . . . 1 00010 123456 1101 0110 6378 . . . § When we update Mem[1101 0110], should we also load it into the cache? 9

Write around caches (a. k. a. write-no-allocate) § With a write around policy, the

Write around caches (a. k. a. write-no-allocate) § With a write around policy, the write operation goes directly to main memory without affecting the cache Mem[1101 0110] = 21763 Index V Tag Data . . . 110 Address 1 00010 123456 1101 0110 21763 . . . § This is good when data is written but not immediately used again, in which case there’s no point to load it into the cache yet for (int i = 0; i < SIZE; i++) a[i] = i; 10

Allocate on write § An allocate on write strategy would instead load the newly

Allocate on write § An allocate on write strategy would instead load the newly written data into the cache Mem[214] = 21763 Index V Tag Data . . . 110. . . Address Data . . . 1 11010 21763 1101 0110 21763 . . . § If that data is needed again soon, it will be available in the cache 11

Which is it? § Given the following trace of accesses, can you determine whether

Which is it? § Given the following trace of accesses, can you determine whether the cache is write-allocate or write-no-allocate? — Assume A and B are distinct, and can be in the cache simultaneously. Miss Load A Miss Store B Hit Store A Hit Load A Miss Load B Hit Load A Answer: Write-no-allocate On a writeallocate cache this would be a hit 12

Opteron Vital Statistics CPU L 1 cache § L 1 Caches: Instruction & Data

Opteron Vital Statistics CPU L 1 cache § L 1 Caches: Instruction & Data — 64 k. B — 64 byte blocks — 2 -way set associative — 2 cycle access time § L 2 Cache: — 1 MB — 64 byte blocks — 4 -way set associative — 16 cycle access time (total, not just miss penalty) § Memory — 200+ cycle access time L 2 cache Main Memory 13

Associativity tradeoffs and miss rates § Higher associativity means more complex hardware § But

Associativity tradeoffs and miss rates § Higher associativity means more complex hardware § But a highly-associative cache will also exhibit a lower miss rate — Each set has more blocks, so there’s less chance of a conflict between two addresses which both belong in the same set — Overall, this will reduce AMAT and memory stall cycles § Figure from the textbook shows the miss rates decreasing as the associativity increases 12% Miss rate 9% 6% 3% 0% One-way Two-way Four-way Eight-way Associativity 14

Cache size and miss rates § The cache size also has a significant impact

Cache size and miss rates § The cache size also has a significant impact on performance — The larger a cache is, the less chance there will be of a conflict — Again this means the miss rate decreases, so the AMAT and number of memory stall cycles also decrease § Miss rate as a function of both the cache size and its associativity 15% 12% Miss rate 9% 1 KB 2 KB 4 KB 8 KB 6% 3% 0% One-way Two-way Four-way Eight-way Associativity 15

Block size and miss rates § Finally, miss rates relative to the block size

Block size and miss rates § Finally, miss rates relative to the block size and overall cache size — Smaller blocks do not take maximum advantage of spatial locality — But if blocks are too large, there will be fewer blocks available, and more potential misses due to conflicts 40% 35% Miss rate 30% 1 KB 25% 8 KB 20% 16 KB 64 KB 15% 10% 5% 0% 4 16 64 256 Block size (bytes) 16