Line Distillation Increasing Cache Capacity by Filtering Unused

Line Distillation: Increasing Cache Capacity by Filtering Unused Words in Cache Lines Moinuddin K. Qureshi M. Aater Suleman Yale N. Patt HPCA 2007 1

Introduction q Caches are organized at linesize granularity Helps when spatial locality is high Unused words when spatial locality is low q Unused words occupy space without contributing to cache hits q Filtering unused words allows cache to store more cache lines 2

Words used per line (avg) Problem: Not all words are useful Cache line (64 B) divided into 8 words of 8 B each (1 MB 8 -way L 2 cache) On average less than 60% words used (4. 7/8) 3

Goal: Improving cache performance q Smaller linesize can result in fewer unused words q Smaller linesize degrades cache performance ØLinesize of 32 B increases MPKI for 14 of 16 benchmarks ØAverage MPKI increases by 25% Goal: Improving cache performance by filtering unused words Insight: Words usage stabilizes as line traverses from MRU to LRU 4

Insight Footprint = 8 -bits per line that tracks word usage Recency Max recency position Stack before footprint update MRU Pos 1 Pos 2 Pos 3 Pos 4 Pos 5 Pos 6 LRU 78% Most footprint updates occur early in recency stack 5% 6% 11% Line Distillation (LDIS): Evict unused words when line crosses certain recency 5

Outline q q q Background Line Distillation Experimental Evaluation Interaction with Compression Related Work and Summary 6

Framework for LDIS Line from memory Distill Cache LOC WOC L 2 Cache Line Organized Cache Word Organized Cache ICACHE valid bits footprint DCACHE (sectored) PROCESSOR 7

Distill Cache (Operation) Four cases: 1. Cache Miss: Access to line D Traditional cache (4 -way) 2. LOC Hit: Access to line B 3. WOC Hit: Access to line A (word A 0) 4. Hole Miss: Access to line A (word A 1)Evict A[1: 6] Words used? MRU C LRU B LOC D A (A 0, A 7 used) Install A 0, A 7 WOC Invalidate all words of A in WOC. Same as. LOC traditional cache Send A 0 and A 7 to L 1 and valid bits Install Line D in and update LRU state Fetch A from Memory and install in LOC 8

Median Threshold Filtering A line with many used words can evict several lines from WOC Line X has all 8 words used X 00 A X 01 B X 04 FX 05 G X 06 H X 07 X 02 D X 03 E C 8 Lines evicted from WOC Increase lines in WOC by not installing lines for which used words > threshold “K” K = median words used in LOC line (computed at runtime) 9

Outline q q q Background Line Distillation Experimental Evaluation Interaction with Compression Related Work and Summary 10

Methodology q Configuration: L 2 cache: 1 MB 8 -way 64 B linesize (Distill cache gives 6 ways to LOC and 2 ways to WOC) Out-of-order processor with 16 KB 2 -way L 1 s 400 cycle memory q Benchmarks: 15 SPEC 2 K benchmarks + health from olden suite (A 250 M instruction slice using Sim. Point for SPEC 2 K) 11

Results LDIS (with MT) (%) Reduction in L 2 MPKI LDIS (No MT) LDIS (MT) reduces MPKI by 25% 12

Reverter Circuit (RC) q Tournament selection: Distill cache vs. traditional cache q Dynamic set sampling with 32 sets [Qureshi+ ISCA’ 06] (storage overhead of ATD: 1 KB) Distill cache Set A Set B ATD-LRU Set C SCTR Set B Set D -+ Set E Set G Set F Set G For sets A, C, D, F, H: Set H if (SCTR > 75%) Enable LDIS if (SCTR < 25%) Disable LDIS 13

Results with RC LDIS (MT, RC) (%) Reduction in L 2 MPKI LDIS (MT, No RC) RC disables LDIS when it increases MPKI. LDIS (MT, RC) reduces MPKI by 30% 14

Overheads q Storage Tags for WOC + footprint bits: 12. 2% overhead q Latency Tag-access (LOC+WOC) increases by one cycle WOC hits incur two cycles to rearrange words q Power Additional power of WOC tag-store 15

(%) IPC Improvement IPC Results LDIS improves average IPC by 12% 16

Outline q q q Background Line Distillation Experimental Evaluation Interaction with Compression Related Work and Summary 17

Compression vs. LDIS q Several proposals to increase capacity via compression q Compression and LDIS fundamentally different Ø Compression exploits redundancy in stored data Ø LDIS leverages unused words for spare capacity q Footprint Aware Compression (FAC) combines both q FAC compresses used words before installing in WOC 18

(%) Reduction in L 2 MPKI Results for FAC 50 40 30 20 10 0 LDIS Compression FAC Compression and LDIS interact positively. FAC reduces MPKI by 50% 19

Outline q q q Background Line Distillation Experimental Evaluation Interaction with Compression Related Work and Summary 20

Related work q Spatial-Temporal Cache -Gonzales+ [ICS’ 95] q Spatial Locality Prediction q Variable Linesize Cache –Johnson+ [ISCA’ 97] –Veidenbaum+ [ICS’ 99] q Spatial Footprint Prediction q Spatial Pattern Prediction –Kumar+ [ISCA’ 98], Pujara+ [HPCA’ 06] -Chen+ [HPCA’ 05] LDIS is particularly suited for large caches and outperforms predictor-based techniques without requiring separate structure for tracking spatial footprint 21

Contributions q Line Distillation: Filter unused words without a separate footprint predictor q Distill cache: Utilize extra capacity created by LDIS q Median Threshold Filtering and Reverter Circuit: Improve performance and robustness of LDIS Result: LDIS (MT+RC) reduces MPKI by 30% q Footprint Aware Compression: LDIS + compression Result: FAC reduces MPKI by 50% 22

Questions 23

Result comparing capacity 24

Line Size vs. MPKI 25

Distribution of Hit-Miss 26

Average words usage (detailed) 27

Result for 3 types of LDIS 28

Replacement q. LRU in LOC q. WOC needs variable sized replacement q. Only power-of-two sizes allowed in WOC q. Placement constrained to alignment boundary q. Random selection in case of multiple candidates 29

Background (pictorial) 30

Result LDIS vs. FAC (detailed) 31

Comparison with SFP 32

Appendix A: Other SPEC Benchmarks 33

Appendix B: Cache Size vs. Density 34

Summary q Many words in cache lines remain unused q Unused words unlikely to be accessed in less recent part of LRU stack Line Distillation (LDIS) q Distill-cache utilizes extra capacity created by LDIS q LDIS reduces MPKI by 30% and improves IPC by 12% q “Footprint Aware Compression” combines LDIS and compression to reduce MPKI by 50% 35