Using a Victim Buffer in an Application Specific

Using a Victim Buffer in an Application. Specific Memory Hierarchy Chuanjun Zhang*, Frank Vahid** *Dept. of Electrical Engineering Dept. of Computer Science and Engineering University of California, Riverside **Also with the Center for Embedded Computer Systems at UC Irvine This work was supported by the National Science Foundation and the Semiconductor Research Corporation Chuanjun Zhang, UC Riverside 1

Low Power/Energy Techniques are Essential n Skadron et al. , 30 th ISCA Hot enough to cook an egg. n High performance processors are going to be too hot to work n Low energy dissipation is imperative for battery-driven embedded systems Low power techniques are essential to both embedded systems and high performance processors Frank Vahid, UC Riverside 2

Caches Consume Much Power n Caches consume 50% of total processor system power n n ARM 920 T and M*CORE(Segars 01, Lee 99) Caches accessed often n n Associativity reduces misses n n Consume dynamic power Less power off-chip, but more power per access Victim buffer helps n n n Add to direct-mapped cache Keep recently evicted lines in small buffer, check on miss Like higher-associativity, but without extra power per access n >50% (Jouppi 90) 10% energy savings, 4% performance improvement (Albera 99) Frank Vahid, UC Riverside Processor Cache Victim buffer Memory 3

Victim Buffer n PROCESSOR With a victim buffer n One cycle Two cycles n HIT n MISS n HIT Miss Without a victim buffer n n L 1 cache Victim buffer 22 cycles 21 One cycle on a cache hit Two cycles on a victim buffer hit Twenty two cycles on a victim buffer miss n One cycle on a cache hit Twenty one cycles on a victim buffer miss More accesses to off-chip memory OFFCHIP MEMORY Frank Vahid, UC Riverside 4

Cache Architecture with a Configurable Victim Buffer n Is a victim buffer a useful configurable cache parameter? n n n SRAM cache control circuit Thus, want ability to shut off VB for given app. One bit register A switch Four-line victim buffer shown tag reg VB misses, so extra cycle wasteful? Hardware overhead n VB on/off Helps for some applications For others, not useful n n data to processor Vdd L 1 cache data SRAM victim line 1 to mux 0 from cache s control circuit 27 -bit tag 16 -byte cache line data from next level memory control signals CAM SRAM Fully-associative victim buffer control signals to the next level memory Frank Vahid, UC Riverside 5

Hit Rate of a Victim Buffer Data cache Instruction cache Hit rate of victim buffer when added to an 8 Kbyte, 4 Kbyte, or 2 Kbyte direct-mapped cache Benchmarks from Powerstone, Media. Bench, and Spec 2000. Frank Vahid, UC Riverside 6

Computing Total Memory-Related Energy n Consider CPU stall energy and off-chip memory energy n n Excludes CPU active energy Thus, represents all memory-related energy_mem = energy_dynamic + energy_static energy_dynamic = cache_hits * energy_hit + cache_misses * energy_miss = energy_offchip_access + energy_u. P_stall + energy_cache_block_fill energy_static = cycles * energy_static_per_cycle energy_miss = k_miss_energy * energy_hit energy_static_per_cycle = k_static * energy_total_per_cycle (we varied the k’s to account for different system implementations) Underlined – measured quantities n. Simple. Scalar (cache_hits, cache_misses, cycles) n. Our layout or data sheets (others) n Frank Vahid, UC Riverside 7

Performance and Energy Benefits of Victim Buffer with a Direct-Mapped Cache Substantial benefit Should shut-off VB An 8 -line victim buffer with an 8 Kbyte direct-mapped cache (0%=DM w/o victim buffer) Configurable victim buffer is clearly useful to avoid performance penalty for certain applications Frank Vahid, UC Riverside 8

Is a Configurable Victim Buffer Useful Even With a Configurable Cache n We showed that a configurable cache can reduce memory access power by half on average n n (Zhang/Vahid/Najjar ISCA 03, ISVLSI 03) 2 Kb way Software-configurable cache n n n Line Associativity – 1, 2 or 4 ways Size: 2, 4 or 8 Kbytes Does that configurability subsume usefulness of configurable victim buffer? Frank Vahid, UC Riverside 9

Best Configurable Cache with VB Configurations n n n Optimal cache configuration when cache associativity, cache size, and victim buffer are all configurable. I and D stands for instruction cache and data cache, respectively. V stands for the victim buffer is on. n. K stands for the cache size is n Kbyte. The associativity is represented by the last four characters n n Benchmark vpr, I 2 D 1 stands for twoway instruction cache and directmapped data cache. Note that sometimes victim buffer should be on, sometimes off Frank Vahid, UC Riverside 10

Performance and Energy Benefits of Victim Buffer Added to a Configurable Cache n n An 8 -line victim buffer with a configurable cache, whose associativity, size, and line size are configurable (0%=optimal config. without VB) Still surprisingly effective Frank Vahid, UC Riverside 11

Conclusion n Configurable victim buffer useful with direct-mapped cache n n n Configurable victim buffer also useful with configurable cache n n n As much as 60% energy and 4% performance improvements for some applications Can shut off to avoid performance penalty on other apps. As much as 43% energy and 8% performance improvement for some applications Can shut off to avoid performance overhead on other applications Configurable victim buffer should be included as a softwareconfigurable parameter to direct-mapped as well as configurable caches for embedded system architectures Frank Vahid, UC Riverside 12