A Hybrid Adaptive Feedback Based Prefetcher Santhosh Verma

A Hybrid Adaptive Feedback Based Prefetcher Santhosh Verma, David Koppelman and Lu Peng Louisiana State University 1

Motivation o Can’t always expect high prefetch accuracy & timeliness o Potential can be lost when these are low o Adaptive schemes adjust aggressiveness based on effectiveness o Adaption and selectiveness as important as address prediction 2

Our Scheme – Hybrid Adaptive Prefetcher (HAP) o Start with good address prediction – Stride / Sequential hybrid n Sequential prefetching scheme requires no warmup n Stride prefetcher is more robust o Issue prefetches selectively o Incorporate a published adaptive prefetch method n Feedback Directed Prefetching (Srinath et. al, HPCA 2007) o Improve with bandwidth adaption 3

Related Work – Feedback Directed Prefetching (HPCA 2007) o Prefetcher aggressiveness defined by prefetch distance and degree o Aggressiveness adjusted dynamically based on three feedback metrics n Percentage of useful prefetches n Percentage of late prefetches n Percentage of prefetches which cause demand misses (cache pollution) 4

Differences between FDP and our scheme o Use both L 1 and L 2 prefetching n Scheme is modified to support L 1/L 2 o Use a hybrid stride / sequential prefetching scheme o A bandwidth based feedback metric is proposed o No cache pollution metric 5

Stride/Sequential Prefetching Scheme – Training Stride Prefetcher o Use a PC-indexed stride prediction scheme 1. Compute new stride using this field and current address value Stride Prediction Table Entry 2. Store computed stride 3. Increment count for unchanged stride Reset otherwise o Entry is trained if Count is above a threshold value 6

Stride/Sequential Prefetching Scheme – Issuing Prefetches o Check stride table on demand miss / hit to prefetched line n Issue stride prefetches based on degree and distance o Sequential prefetches n If no valid / trained stride entry n If previous line present in cache n Issue sequential prefetches based on degree 7

Adjusting Aggressiveness with Feedback Metrics o Prefetch Accuracy – Percentage of prefetches used by a demand request o Prefetch Lateness – Percentage of accurate prefetches which are late o Bandwidth Contention – Percentage of clock cycles during which cache bandwidth is above a threshold o Evaluate separately for L 1 and L 2 o Evaluate periodically after fixed number of cycles. Adjust aggressiveness if justified. 8

Storage efficient Miss Status Hit Registers (MSHRs) o Used to track all inflight / inqueue memory requests at both cache levels 1. Entry allocated for each 2. outstanding L 1 and / or 3. L 2 request. Valid bit set. MSHR Entry 2. Two bit cache level field indicates L 1 only, L 2 only or combined L 1 / L 2 3. Two prefetch bits indicate prefetch requests 4. Concurrent L 1 and L 2 requests to the same line share the same MSHR entry 9

Implementing Feedback Metrics o Prefetch Accuracy n Prefetch bit set for prefetched line brought into cache n Bit set in MSHR for inflight / inqueue prefetched lines n Increment accurate count if demand request finds a set bit n Reset bit after increment n Accuracy is based on percentage of total prefetches issued 10

Implementing Feedback Metrics o Prefetch Lateness n Prefetch bit (s) set in MSHR for a prefetched inflight / inqueue line n On demand miss, late prefetch detected o If a valid MSHR entry exists for this miss o If prefetch bit for the correct cache level is set n Reset bit after incrementing late count n Lateness is based on percentage of useful prefetches 11

Implementing Feedback Metrics o Bandwidth Contention - 1 n Use MSHR to monitor total outstanding L 1 and L 2 requests in every cycle n Increment counter for every cycle that total is above threshold n The contention rate is based on percentage of total cycles o Bandwidth Contention - 2 n Prefetches not issued if outstanding requests are above threshold 12

Adjusting Aggressiveness o Evaluate metrics at fixed intervals o Determine if high or low based on a threshold o May adjust aggressiveness based on following criteria Aggressiveness Policy 13

Prefetcher Aggressiveness Levels o Aggressiveness adjusted in increments of one Very Conservative Middle Aggressiveness Very Aggressive Prefetcher Aggressiveness Levels 14

Experimental Evaluation - Setup o Evaluate 15 SPEC CPU 2006 Benchmarks using CMPSim Simulator o Evaluate for three competition configurations n Config 1 – 2048 KB L 2 Cache, unlimited bandwidth n Config 2 – 2048 KB L 2 Cache, limited bandwidth n Config 3 – 512 KB L 2 Cache, limited bandwidth o Limited bandwidth configs allow one L 1 issue per cycle and one L 2 per 10 cycles 15

Experimental Evaluation - Setup o Compare our scheme, Hybrid Adaptive Predictor (HAP) to four configurations n n No prefetching Middle Aggressive Stride Very Aggressive Stride Modified Feedback Directed Prefetcher o Uses both L 1 / L 2 prefetching o Does not use a cache pollution metric 16

Results - Expectations o Very aggressive stride will do better on some, worse on other benchmarks o Adaptive schemes will perform at least as well as non-adaptive o Unlimited bandwidth and large cache configurations benefit aggressive schemes 17

Results – Bandwidth Unlimited, 2 MB L 2 Config • HAP outperforms other prefetchers for all benchmarks except lbm • Performance benefit compared to mid-aggressive stride is 11% on average and 46% versus no prefetching. 18

Results – Bandwidth Limited, 2 MB L 2 Config • HAP is best on average. Aggressive stride performs best in three benchmarks (mcf, lbm and soplex) • Performance benefit compared to mid-aggressive stride is 9% on average and 45% versus no prefetching. 19

Results – Bandwidth Limited, 512 KB L 2 Config • Results are similar to Config 2 • Performance benefit compared to mid-aggressive stride is 8% on average and 44% versus no prefetching. 20

Results (All benchmarks) – Bandwidth Limited, 2 MB L 2 Config • Additional benchmarks are mostly unaffected by prefetching • Performance benefit compared to mid-aggressive stride is 6% on average and 29% versus no prefetching for all benchmarks. 21

Conclusions o A well designed and adaptive prefetching scheme is very effective o Very aggressive stride works best for some benchmarks o A cache pollution metric may improve results 22

THANK YOU QUESTIONS? 23