ARC A SELFTUNING LOW OVERHEAD REPLACEMENT CACHE Nimrod

ARC: A SELF-TUNING, LOW OVERHEAD REPLACEMENT CACHE Nimrod Megiddo Dharmendra S. Modha IBM Almaden Research Center

Introduction (I) • Caching is widely used in – storage systems – databases – web servers – processors – file systems – disk drives – RAID controllers – operating systems

Introduction (II) • ARC is a new cache replacement policy: – Scan-resistant: • Better than LRU – Self-tuning: • Avoids problem of many recent cache replacement policies – Tested on numerous workloads

Our Model (I) Cache/ Main Memory (pages) replacement policy on demand Secondary Storage (pages)

Our Model (II) • Caches stores uniformly sized items (pages) • On demand fetches into cache • Cache expulsions decided by cache replacement policy • Performance metrics include – Hit rate (= 1 - miss rate) – Overhead of policy

Previous Work (I) • Offline Optimal (MIN): replaces the page that has the greatest forward distance – Requires knowledge of future – Provides an upper-bound • Recency (LRU): – Most widely used policy • Frequency (LFU):

Previous Work (II) • LRU-2: replaces page with the least recent penultimate reference – Better hit ratio – Needs to maintain a priority queue • Corrected in 2 Q policy – Must still decide how long a page that has only been accessed once should be kept in the cache • 2 Q policy has same problem

Example Last two references to pages A and B X A X B LRU -2 expels A because B was accessed twice after this next to last reference to A X A Time LRU expels B because A was accessed after this last reference to B

Previous Work (III) • Low Inter-Reference Recency Set (LIRS) • Frequency-Based Replacement (FBR) • Least Recently/Frequently Used(LRFU): subsumes LRU and LFU – All require a tuning parameter • Automatic LRFU (ALRFU) – Adaptive version of LRFU – Still requires a tuning parameter

ARC (I) • Maintains two LRU lists – Pages that have been referenced only once (L 1) – Pages that have been referenced at least twice (L 2) • Each list has same length c as cache • Cache contains tops of both lists: T 1 and T 2 • Bottoms B 1 and B 2 are not in cache

ARC (II) L-1 |T 1| + |T 2| = c “Ghost caches” (not in memory) T 1 B 1 L-2 T 2 B 2

ARC (III) • ARC attempts to maintain a target size target_T 1 for list T 1 • When cache is full, ARC expels – LRU page from T 1 if |T 1| target_T 1 – LRU page from T 2 otherwise

ARC (IV) • If missing page was in bottom B 1 of L 1, ARC increases target_T 1= min(target_T 1+max(|B 2|/|B 1|, 1), c) • If missing page was in bottom B 2 of L 2, ARC decreases target_T 1

ARC (V) • Overall result is – Two heuristics compete with each other – Each heuristic gets rewarded any time it can show that adding more pages to its top list would have avoided a cache miss • Note that ARC has no tunable parameter

Experimental Results • Tested over 23 traces: – Always outperforms LRU – Performs as well as more sophisticated policies even when they are specifically tuned for the workload • Sole exception is 2 Q – Still outperforms 2 Q when 2 Q has no advance knowledge of the workload