Synthesis of Customized Loop Caches for CoreBased Embedded

Synthesis of Customized Loop Caches for Core-Based Embedded Systems Susan Cotterell and Frank Vahid* Department of Computer Science and Engineering University of California, Riverside *Also with the Center for Embedded Computer Systems at UC Irvine This work was supported in part by the U. S. National Science Foundation and a U. S. Department of Education GAANN Fellowship

Introduction Traditional Core Based microprocessor architecture Opportunity to tune the microprocessor architecture to the program 2

Introduction • I-cache – Size – Associativity – Replacement policy I$ Processor D$ Mem Bridge • JPEG – Compression JPEG USB CCDP P 4 • Buses – Width – Bus invert/gray code 3

Introduction • Memory access can consume 50% of an embedded microprocessor’s system power – Caches tend to be power hungry • M*CORE: unified cache consumes half of total power (Lee/Moyer/Arends 99) • ARM 920 T: caches consume half of total power (Segars 01) Clocks 4% Sys. Ctl 3% CP 15 2% PATag RAM 1% Other 4% BIU 8% D Cache 19% I Cache 25% arm 9 25% I MMU 4% D MMU 5% 4

Introduction I$ Processor D$ Mem Bridge JPEG USB CCDP P 4 Advantageous to focus on the instruction fetching subsystem 5

Introduction • Techniques to reduce instruction fetch power – Program Compression • Compress only a subset of frequently used instructions (Benini 1999) • Compress procedures in a small cache (Kirvoski 1997) • Lookup table based (Lekatsas 2000) – Bus Encoding • Increment (Benini 1997) • Bus-invert (Stan 1995) • Binary/gray code (Mehta 1996) 6

Introduction • Techniques to reduce instruction fetch power (cont. ) – Efficient Cache Design • Small buffers: victim, non-temporal, speculative, and penalty to reduce miss rate (Bahar 1998) • Memory array partitioning and variation in cache sizes (Ko 1995) – Tiny Caches • Filter cache (Kin/Gupta/Magione-Smith 1997) • Dynamically loaded tagless loop cache (Lee/Moyer/Arends 1999) • Preloaded tagless loop cache (Gordon-Ross/Cotterell/Vahid 2002) 7

Cache Architectures – Filter Cache • Small L 0 direct mapped cache • Utilizes standard tag comparison and miss logic • Has low dynamic power L 1 memory – Short internal bitlines – Close to the microprocessor Filter cache (L 0) • Performance penalty of 21% due to high miss rate (Kin 1997) Processor 8

Cache Architectures – Dynamically Loaded Loop Cache • Small tagless loop cache • Alternative location to fetch instructions • Dynamically fills the loop cache L 1 memory Dynamic loop cache Mux Processor – Triggered by any short backwards branch (sbb) instruction • Flexible variation – Allows loops larger than the loop cache to be partially stored . . . add r 1, 2. . . sbb -5 Iteration 1 : detect sbb instruction Iteration 2 : fill loop cache Iteration 3 : fetch from loop cache 9

Cache Architectures – Dynamically Loaded Loop Cache (cont. ) L 1 memory • Limitations – Does not support loops with control of flow changes (cofs) – cofs terminate loop cache filling and fetching – cofs include commonly found if-then-else statements. . . add r 1, 2 bne r 1, r 2, 3. . . sbb -5 Dynamic loop cache Mux Processor Iteration 1 : detect sbb instruction Iteration 2 : fill loop cache, terminate at cof Iteration 3 : fill loop cache, terminate at cof 10

Cache Architectures – Preloaded Loop Cache • Small tagless loop cache • Alternative location to fetch instructions • Loop cache filled at compile time and remains fixed – Supports loops with cof • Fetch triggered by any short backwards branch • Start address variation. . . – Fetch begins on first iteration add r 1, 2 bne r 1, r 2, 3. . . sbb -5 loop L 1 memory Preloaded loop cache Mux Processor Iteration 1 : detect sbb instruction Iteration 2 : check to see if loop preloaded, if so fetch from cache 11

Traditional Design • Traditional Pre-fabricated IC – Typically optimized for best average case – Intended to run well across a variety of programs – Benchmark suite is used to determine which configuration L 1 memory ? Mux Processor 12

Core Based Design • Core Based Design – Know application – Opportunity to tune the architecture • Is it worth tuning the architecture to the application or is the average case good enough? microprocessor architecture 13

Evaluation Framework – Candidate Cache Configurations Type Size Number of loops/ line size Configuration Original dynamically loaded loop cache 8 -1024 entries n/a 1 -8 Flexible dynamically loaded loop cache 8 -1024 entries n/a 9 -16 Preloaded loop cache (sa) 8 -1024 entries 2 - 3 loop address registers 17 -32 Preloaded loop cache (sbb) 8 -1024 entries 2 - 6 loop address registers 33 -72 14

Evaluation Framework – Motorola's Powerstone Benchmarks Benchmark # Instr Executed Voice Encoding engine 410607 Engine Controller 1938 Bit Manipulation fir 16211 FIR Filtering binary 816 Binary Insertion g 3 fax 1128023 Group Three Fax Decode blit 22845 Graphics Application insert 1942 Insertion Sort brev 2377 Bit Reversal jpeg 4594721 JPEG Compression compress 138573 Data Compression Program summin 1909787 Handwriting Recognition crc 37650 Cyclic Redundancy Check ucbqsort 219978 U. C. B Quick Sort des 122214 Data Encryption Standard v 42 2442551 Modem Encoding/Decoding Benchmark # Instr Executed adpcm 63891 bcnt Description 15

Tool Chain - Simulation program instr trace LOOAN loop stats packed loops & explr script lcsim loop cache stats lc power calc loop cache power many configs. tech info 16

% Energy Savings Results - Averages 100 80 60 Config 11 Program Optimal 40 20 0 Benchmark • Configuration 11 (flexible/32 entry/dynamically loaded loop cache) – On average does well – 25% Instruction Fetch Energy Savings • Loop cache selection on a per application basis – Saves additional 70% Instruction Fetch Energy Savings 17

Tool Chain - Simulation program instr trace LOOAN loop stats packed loops & explr script lcsim loop cache stats lc power calc loop cache power many configs. tech info program instr trace . . . lcsim 18

Tool Chain - Estimation program instr trace LOOAN func calls packed loops loop stats loop and function call statistics li f = s*b; . . . estimator li f = s*b; estimator fast. loop cache stats lc power calc loop cache power tech info What kind of statistics? How can we use this information to model the various loop caches? 19

LOOAN Loop. . . main. 1. FCall: huff_dc_dec. . main. 2. FCall: huff_ac_dec. . main. 3. FCall: dquantz_lum. . main. 4. FCall: j_rev_dct. . main. 5. 1. 1. 1 Start End Static Size 2 1379 1395 901 1412 963 1422 1362 1432 1332 1445 1446 1448 1449 1491 1410 962 1419 1114 1429 1378 1439 1361 1473 1468 1464 1459 1490 113 16 62 8 152 8 17 8 30 29 23 17 11 Dynamic Instructions per Execution avg min max 4594721 4594611 121557 187 101 325 1407106 2337 1544 4658 430800 710 710 2181600 3628 452922 22640 2824 88 88 88 Iterations per Num Total Instrs Execution Exec avg min max 1 1 4594721 1 1 476963 600 600 1 9600 1 1 1 600 0 600 600 600 1 4800 1 1 1 600 0 21 21 21 1 452922 8 8 8 20 452800 31 31 31 160 451840 8 8 8 4800 422400 How big are the loops? Loop function Once thehierarchy, loop is called, How howcalls manytimesisdoes the loop it iterate? called? 20

Estimation – Original Dynamically Loaded Loop Cache • How many times. Staticdo Dynamic we Instructions fill the per loop Iterationscache? per Num Total Instrs Loop iter. 1: detect sbb. . main iter 2: fill . . main. 5. 1. 1. 1 Start End Execution Exec avg min max avg min mov max r 5, r 4 1491 1490 4594721 iter 4594721 1 1 4594721 1: detect sbb 1 1. . . 1491 113 4594611 1 1 476963 r 1, 2 add r 1, 2 iter 2: fill, abort 1473 29 452922 21 21 21 1 452922 1468 23 22640 8 r 1, 20 4528003 r 1, r 2, 3 r 2, at cof 22640 8 8 sub 1464 17 2824 31 31 31 160 451840 bne r 1, r 2, 3 iter 3: fill, abort 1459 11 88 88 88 8 4800 422400 mov. . . 2 1379 add 1445 1446 sub 1448. . . 1449 sbb -5 Size at cof x x. . . x x sbb -5 if( loop size <= lc size && loop iteration >= 2) fillscof if( = #!=times sbb) loop called * loop size fills = # loop called * (iter per exec– 1) * offset to 1 st cof else fills = # loop called * loop size 21

Estimation - Original Dynamically Loaded Loop Cache • How many times do we fetch from the loop cache? iter 1: detect sbb iter 2: fill iter 3: fetch from loop cache mov. . . add sub. . . sbb r 5, r 4 r 1, 2 r 1, r 2, 3 -5 iter 1: detect sbb iter 2: fill, abort at cof iter 3: fill, abort at cof xx xx mov. . . add sub bne. . . sbb r 5, r 4 r 1, 2 r 1, r 2, 3 -5 if( loop size <= lc size && loop iteration >= 3) fetch # times if( cof=== sbb) loop called * (loop iter – 2) * loop size fetch = # times loop called * (loop iter – 2) * loop size 22

Estimation • Loop Cache Equations – Each loop cache type is characterized by approximately 5 unique equations – 20 different equations in all 23

Estimation Results - Accuracy Average across all benchmarks Avg % Energy Savings 100 Simulation 80 Estimation 60 40 20 0 Cache Configuration • Ranges from 0 -16% difference • Average 2% difference 24

Estimation Results - Fidelity % Energy Savings 100 80 Simulation 60 Estimation 40 20 0 Benchmark • Does the estimation method preserve the fidelity? – summin shows the worst case – 10% – On average <1% difference in savings between loop cache chosen via simulation vs. loop cache chosen via estimation 25

Time Comparison Simulation Tool Chain Estimation Tool Chain Benchmark Num Instr LOOAN Script lcsim lc total sim LOOAN Est. lc power total est speedup Exec. Gen power time calc (sec. ) adpcm 63891 0. 31 0. 01 32. 15 0. 01 32. 48 0. 31 0. 16 0. 01 0. 48 68 compress 138573 0. 85 0. 01 82. 50 engine g 3 fax jpeg 410607 1128023 4594721 2. 12 3. 54 17. 57 0. 02 214. 99 0. 02 385. 44 0. 01 1837. 28 summin v 42 1909787 2442551 11. 42 12. 07 0. 01 83. 37 0. 85 0. 14 0. 01 1. 00 83 0. 01 217. 14 0. 01 389. 01 0. 01 1854. 87 2. 12 3. 54 17. 57 0. 08 0. 09 0. 12 0. 01 2. 21 3. 64 17. 7 98 107 105 0. 01 903. 73 0. 01 915. 17 8. 25 0. 01 1252. 48 0. 01 1264. 57 12. 27 more benchmarks in paper. . . 0. 09 0. 12 0. 01 8. 35 12. 40 110 102 AVERAGE : 67 Biggest example onlysimulation 30 for minutes – small program bottleneck Required both was methods Started looking at Media. Bench – simulation takes hours 26

Conclusion and Future Work • Important to tune the architecture to the program • Simulation methods are slow – Presented a equation based methodology which is faster than the simulation based methodology previously used – Accuracy/fidelity preserved • Future Work – Expand types of tiny caches – Look at more benchmarks • Media. Bench - several hours (up to 48 hours) for our simulations – Expand hierarchy search 27

Thank you for your attention. Questions? 28