ECE 463563 Fall 18 Caches reducing miss penalty

  • Slides: 13
Download presentation
ECE 463/563 Fall `18 Caches: reducing miss penalty Prof. Eric Rotenberg Fall 2018 ECE

ECE 463/563 Fall `18 Caches: reducing miss penalty Prof. Eric Rotenberg Fall 2018 ECE 463/563, Microprocessor Architecture, Prof. Eric Rotenberg 1

Reduce Miss Penalty • • L 2 cache Victim cache Write buffer Early restart,

Reduce Miss Penalty • • L 2 cache Victim cache Write buffer Early restart, critical word first Fall 2018 ECE 463/563, Microprocessor Architecture, Prof. Eric Rotenberg 2

L 2 cache • Effects of increasing L 1 cache size + Decrease L

L 2 cache • Effects of increasing L 1 cache size + Decrease L 1 miss rate − Increase L 1 hit time AAT = HTL 1 + MRL 1* MP • Instead, reduce average miss penalty by “catching” some L 1 capacity misses as hits in a larger cache between L 1 cache and main memory Acronyms: HT: hit time MR: miss rate MP: miss penalty Fall 2018 AAT = HTL 1 + MRL 1 * MPL 1 = HTL 2 + MRL 2 * MPL 2 ECE 463/563, Microprocessor Architecture, Prof. Eric Rotenberg 3

L 2 cache example Example HT MR MP L 1 1 cycle 0. 05

L 2 cache example Example HT MR MP L 1 1 cycle 0. 05 it depends L 2 4 cycles 0. 01 100 cycles L 1 cache only L 1 and L 2 caches MPL 1 100 cycles 5 cycles (4 + 0. 01*100) AAT 6 cycles (1 + 0. 05*100) 1. 25 cycles (1 + 0. 05*5) L 1 cache 4 cycles L 2 cache 100 cycles main memory Fall 2018 100 cycles main memory ECE 463/563, Microprocessor Architecture, Prof. Eric Rotenberg 4

Victim cache • See previous pptx Fall 2018 ECE 463/563, Microprocessor Architecture, Prof. Eric

Victim cache • See previous pptx Fall 2018 ECE 463/563, Microprocessor Architecture, Prof. Eric Rotenberg 5

Write misses and the processor • Processor doesn’t need to wait for write miss

Write misses and the processor • Processor doesn’t need to wait for write miss – It’s not asking for data • Don’t stall processor – Do the write in the background – Need a write buffer – Processor delegates responsibility to the write buffer, for performing the write to the memory hierarchy Fall 2018 ECE 463/563, Microprocessor Architecture, Prof. Eric Rotenberg 6

Write buffer in WBWA Scenario 1: write hit write request from processor write hit

Write buffer in WBWA Scenario 1: write hit write request from processor write hit L 1 cache WBWA policy Next level in mem. hier. (L 2 cache or main memory) Fall 2018 ECE 463/563, Microprocessor Architecture, Prof. Eric Rotenberg 7

Write buffer in WBWA Scenario 2: write miss write request from processor write miss

Write buffer in WBWA Scenario 2: write miss write request from processor write miss L 1 cache WBWA policy read block from next level data address merge Next level in mem. hier. (L 2 cache or main memory) Fall 2018 ECE 463/563, Microprocessor Architecture, Prof. Eric Rotenberg 8

Write buffer in WTNA Scenario 1: write hit write request from processor write hit

Write buffer in WTNA Scenario 1: write hit write request from processor write hit L 1 cache WTNA policy data address write to next level Next level in mem. hier. (L 2 cache or main memory) Fall 2018 ECE 463/563, Microprocessor Architecture, Prof. Eric Rotenberg 9

Write buffer in WTNA Scenario 2: write miss write request from processor write miss

Write buffer in WTNA Scenario 2: write miss write request from processor write miss L 1 cache WTNA policy data address write to next level Next level in mem. hier. (L 2 cache or main memory) Fall 2018 ECE 463/563, Microprocessor Architecture, Prof. Eric Rotenberg 10

Complication: write followed by dependent read • A read that hits in the cache

Complication: write followed by dependent read • A read that hits in the cache is ok – Since the block is in the cache, any previous writes to the block have completed • A read that misses in the cache needs special treatment – It must check the write buffer for any pending writes that it depends on – If its address matches a pending write’s address, read gets data directly from write buffer • Otherwise it could get stale data from the next level if the younger read completes before the older write Fall 2018 ECE 463/563, Microprocessor Architecture, Prof. Eric Rotenberg 11

Read miss checks if it depends on any pending writes in the write buffer

Read miss checks if it depends on any pending writes in the write buffer read request from processor read miss read address L 1 cache (WBWA or WTNA) data address =? =? Next level in mem. hier. (L 2 cache or main memory) Fall 2018 ECE 463/563, Microprocessor Architecture, Prof. Eric Rotenberg 12

Early restart, Critical word first • These techniques shave off a few cycles from

Early restart, Critical word first • These techniques shave off a few cycles from the miss penalty by not waiting for all bytes of the block • Early Restart: – As soon as the requested word arrives, forward it to the processor – Processor resumes while the rest of the block is filled into the cache • Early Restart + Critical Word First: – Problem still with Early Restart: What if requested word is in the middle of the block? – Critical Word First: Memory system returns requested word first, followed by other words Fall 2018 ECE 463/563, Microprocessor Architecture, Prof. Eric Rotenberg 13

ECE 463563 Fall 18 Reducing miss rate cacheECE 463563 Fall 18 Reducing miss rate cache
ECE 463563 Fall 20 Victim Caches Prof EricECE 463563 Fall 20 Victim Caches Prof Eric
ECE 463563 Fall 20 Caches 3 Cs modelECE 463563 Fall 20 Caches 3 Cs model
Memory Hierarchy Reducing Miss Penalty Reducing Hit TimeMemory Hierarchy Reducing Miss Penalty Reducing Hit Time
ECE 463563 Fall 20 Pipelining dynamic branch predictionECE 463563 Fall 20 Pipelining dynamic branch prediction
ECE 463563 Fall 18 Basic Cache Operation contECE 463563 Fall 18 Basic Cache Operation cont
ECE 463563 Fall 20 Prof Rotenberg Simulation ExampleECE 463563 Fall 20 Prof Rotenberg Simulation Example
ECE 463563 Fall 20 Virtual Memory Prof EricECE 463563 Fall 20 Virtual Memory Prof Eric
ECE 463563 Fall 20 RISCV instruction formats DesignECE 463563 Fall 20 RISCV instruction formats Design
ECE 463563 Fall 18 Additional ILP Topics ProfECE 463563 Fall 18 Additional ILP Topics Prof
ECE 463563 Fall 18 Pipelining dynamic branch predictionECE 463563 Fall 18 Pipelining dynamic branch prediction
ECE 463563 Fall 18 pipelining data hazards ProfECE 463563 Fall 18 pipelining data hazards Prof