Dueling Segmented LRU Replacement Algorithm Hongliang Gao Chris

Dueling Segmented LRU Replacement Algorithm Hongliang Gao Chris Wilkerson

The Basic Ideas • Auxiliary Directory: – Evaluates “dueling” replacement algorithms. • Segmented LRU list: – Reference bit protects lines with good locality. – Aging/ Random Promotion. • Adaptive Bypass: – Protect cache contents by bypassing the cache completely.

Dueling Replacement Algos Auxiliary Directory Set 0 Set 1 Set 2 Set 3 Set 4 Set 5 Set 6 • 32 sets sampled (static) • 2 policies evaluated in each sampled set. • 16 -bit mini-tags • Counter updated when policies differ. Set 7 Tag Array Saturating Counter

Review of Segmented LRU SLRU: Reference Bit 4 LRU bits per line track LRU position Tag • Reference bit is marked when a line is referenced. • Replace any non-referenced lines first. • Replace global LRU if all lines are referenced.

SLRU Features • Random Promotion – Reference bit is marked when referenced or when randomly promoted. – Eg: 1/32 newly allocated lines may randomly be selected for promotion. • Aging – Reference bits can be cleared as well as set. – Line allocations cause the reference bit of the LRU line to be cleared.

Adaptive Bypass • Misses result in allocation or bypass. Data Structure Thrashing on 4 th way No Thrashing w/o Bypass w/ Bypass Cache • Bypass based on a random probability. – Eg: 1, 1/2, 1/4, … 1/4096. – Probability is doubled/halved according to the success of previous bypasses.

SLRU w/ Adaptive Bypassing SLRU: Reference Bit 0 1 • De-allocated line tracked by partial tag. • Allocated line tracked by 4 bit pointer. • Valid Bit • Virtual Bypass Bit 16 bit partial tag for “out-of-cache” competitor 4 bit pointer for “in-cache competitor”

Frequency of Bypass

DSB impact on MPKI vs TLRU MPKI for true LRU % reduction MPKI w/ DSB

Speedup

BACKUP

SLRU w/ Adaptive Bypassing SLRU: Reference Bit • Bypassed line tracked by partial tag. • Incumbent line tracked by 4 bit pointer. 1 • Subsequent reference 0 to bypass line reduces bypass probability. • Subsequent reference 16 bit partial tag for “out-of-cache” competitor 4 bit pointer for “in-cache competitor” to incumbent increases bypass probability.

CONFIG 1 CONFIG 2 CONFIG 3 Enable bypassing for policy 0 True Enable bypassing for policy 1 False True Random promotion probability for policy 0 0 Random promotion probability for policy 1 0 0 16 Aging for policy 0 0 Aging for policy 1 1 Virtual bypassing probability 16 8 8 Initial bypassing probability 64 64 8 Second minimum bypassing probability (minimum is 0) 1/256 1/4096 Config 2: 2 Policies

auxiliary directory collects statistics replacement policy performance and updates a policy selector counter. SLRU 1 -reference bit indicates whether each line is in the reference or non-reference list. Set 0 Set 1 Set 2 4 LRU bits per line track LRU position Set 3 Set 4 Set 5 Set 6 valid bits Set 7 Tag Array 16 bit partial tag for “out-of-cache” competitor Tracking bypass 4 bit pointer for “in-cache competitor”