The Expandable Split Window Paradigm for Exploiting Fine

The Expandable Split Window Paradigm for Exploiting Fine. Grain Parallelism Manoj Franklin and Gurindar S. Sohi Presented by Allen Lee May 7, 2008 CS 258 Spring 2008 1

Overview n n There exists a large amount of theoretically exploitable ILP in many sequential programs Possible to extract parallelism by considering a large “window” of instructions ¨ Large windows may have large communication arcs in the data-flow graph n Minimize communication costs by using multiple smaller windows, ordered sequentially

Definitions n Basic Block ¨ “A maximal sequence of instructions with no labels (except possibly at the first instruction) and no jumps (except possibly at the last instruction)” - CS 164 Fall 2005 Lecture 21 n Basic Window ¨ “A single-entry loop-free call-free block of (dependent) instructions”

Splitting a Large Window
![Example Pseudocode: Assembly: for(i = 0; i < 100; i++) { x = array[i] Example Pseudocode: Assembly: for(i = 0; i < 100; i++) { x = array[i]](http://slidetodoc.com/presentation_image_h/043bd9c78d21354aa6a71fead327ca6d/image-5.jpg)
Example Pseudocode: Assembly: for(i = 0; i < 100; i++) { x = array[i] + 10; if(x < 1000) array[i] = x; else array[i] = 1000; } A: R 1 = R 1 + 1 R 2 = [R 1, base] R 3 = R 2 + 10 BLT R 3, 1000, B R 3 = 1000 B: [R 1, base] = R 3 BLT R 1, 100, A Basic Window 1 A 1: R 11 = R 10 + 1 R 21 = [R 11, base] R 31 = R 21 + 10 BLT R 31, 1000, B 1 R 31 = 1000 B 1: [R 11, base] = R 31 BLT R 11, 100, A 2 Basic Window 2 A 2: R 12 = R 11 + 1 R 22 = [R 12, base] R 32 = R 22 + 10 BLT R 32, 1000, B 2 R 32 = 1000 B 2: [R 12, base] = R 32 BLT R 12, 100, A 3

Block Diagram n n n independent, identical stages in a circular queue Control unit (not shown) assigns windows to stages Windows are assigned in sequential order Head window is “committed” when execution completes Distributed units for Issue and execution ¨ Instruction supply ¨ Register file (future file) ¨ n Address Resolution Buffers (ARBs) for speculative stores

Distributed Issue and Execution n Take operations from local instruction cache and pumps them into functional units Each IE has own set of functional units Possible to connect IE units to common Functional Unit Complex

Distributed Instruction Supply n Two-level instruction cache Each stage has its own L 1 cache ¨ L 1 misses are forwarded to L 2 cache ¨ L 2 misses are forwarded to main memory ¨ n If the transferred window from L 2 is a loop, L 1 caches in subsequent stages can grab it in parallel (“snarfing”)

Distributed Register File n n n Each stage has separate register file (future file) Intra-stage dependencies enforced by doing serial execution within IE unit Inter-stage dependencies expressed using masks

Register Masks n n Concise way of letting a stage know which registers are read and written in a basic window use masks ¨ Bit mask that represents registers through which externally-created values flow in a basic block n create masks ¨ Bit mask that represents registers through which internally-created values flow out of a basic block n n n Masks fetched before instructions fetched May be statically generated at compile-time by compiler or dynamically at run-time by hardware Reduce forwarding traffic between stages

Data Memory Problem: Cannot allow speculative stores to main memory because no undo mechanism, but speculative loads to same location need to get the new value n Solution: Address Resolution Buffers n

Address Resolution Buffer n n Decentralized associative cache Two bits per stage: one for load, one for store Discard windows if load/store conflict Write store value when head window “commits”

Enforcing Control Dependencies Basic windows may be fetched using dynamic branch prediction n Branch mispredictions are handled by discarding subsequent windows n ¨ The tail pointer in the circular queue is moved back to the stage after the one containing the mispredicted branch

Simulation Environment MIPS R 2000 – R 2010 instruction set n Up to 2 instructions issued/cycle per IE n Basic window has up to 32 instructions n 64 KB, direct-mapped data cache n 4 Kword L 1 instruction cache n L 2 cache with 100% hit rate n Basic window = basic block n

Results with Unmodified Code Benchmarks eqntott espresso gcc xlisp dnasa 7 doduc fpppp matrix 300 spice 2 g 6 tomcatv n n Mean Basic Block Size No. of Stages Branch Prediction Accuracy Completion Rate 4. 19 6. 47 5. 64 5. 04 4 4 90. 14% 83. 13% 85. 11% 80. 21% 2. 04 2. 06 1. 81 1. 91 26. 60 12. 22 113. 42 21. 49 6. 14 45. 98 10 10 10 99. 13% 86. 90% 88. 86% 99. 35% 86. 95% 99. 28% 2. 73 1. 92 3. 87 5. 88 3. 23 3. 64 Completion Rate = # of completed instructions per cycle Speedup > Completion Rate

Results with Modified Code Benchmarks n No. of Stages Prediction Accuracy Completion Rate eqntott espresso 4 8 4 95. 58% 96. 14% 92. 17% 4. 23 4. 97 2. 30 dnasa 7 matrix 300 tomcatv 10 10 10 98. 95% 99. 34% 99. 31% 7. 17 7. 02 4. 49 Benchmarks with hand-optimized code ¨ Rearranged instructions ¨ Expanded basic window

Conclusion ESW exploits fine-grain parallelism by overlapping multiple windows n The design is easily expandable by adding more stages n ¨ But limits to snarfing…
- Slides: 17