EECS 583 Class 9 Classic and ILP Optimization

EECS 583 – Class 9 Classic and ILP Optimization University of Michigan October 5, 2011

Announcements & Reading Material v Look on Phorum for help with HW #2 » Daya’s post on adding/deleting instructions » Splitting a BB v Today’s class » “Compiler Code Transformations for Superscalar-Based High. Performance Systems, ” Scott Mahlke, William Chen, John Gyllenhaal, Wen-mei Hwu, Pohua Chang, and Tokuzo Kiyohara, Proceedings of Supercomputing '92, Nov. 1992, pp. 808 -817 v Next class (instruction scheduling) » “Machine Description Driven Compilers for EPIC Processors”, B. Rau, V. Kathail, and S. Aditya, HP Technical Report, HPL-9840, 1998. (long paper but informative) -1 -

HW#2 – Best Times from Last Semester v Times in seconds for each application » » » v v Perf 1 Perf 2 Perf 3 583 wc no spec LICM. 008453. 023796. 012391. 002813 with spec LICM. 000095. 000133. 000099. 001644 This translates to: 89 x, 178 x and 125 x and 1. 7 x speedup Note: These generated with an older LLVM » But, gives you an idea of what to expect. -2 -

Course Project – Time to Start Thinking About This v Mission statement: Design and implement something “interesting” in a compiler » LLVM preferred, but others are fine » Groups of 2 -3 people (1 or 4 persons is possible in some cases) » Extend existing research paper or go out on your own v Topic areas » » » » Automatic parallelization/SIMDization Optimizing for GPUs Targeting VLIW processors Memory system optimization Reliability Energy For the adventurous Ÿ Dynamic optimization Ÿ Streaming applications Ÿ Binary optimization -3 -

Course Projects – Timetable v Now » Start thinking about potential topics, identify teammates v Oct 24 -28: Project proposals » » v No class that week Daya/I will meet with each group Informal proposal discussed at meeting Written proposal (a paragraph or 2 plus some references) due Oct 31 Last 2 wks of Nov: Project checkpoint » Class as usual » Daya/I will again meet with each group » Update on your progress, what left to do v Dec 13 -16: Project demos -4 -

Sample Project Ideas (in random order) v Memory system » Cache profiler for LLVM IR – miss rates, stride determination » Data cache prefetching » Data layout for improved cache behavior v Optimizing for GPUs » Dumb Open. CL/CUDA smart Open. CL/CUDA – selection of threads/blocks and managing on-chip memory » Reducing uncoalesced memory accesses – measurement of uncoalesced accesses, code restructuring to reduce these » Matlab CUDA/Open. CL » Stream. It to CUDA/Open. CL (single or multiple GPUs) v Reliability » AVF profiling, reducing vulnerability to soft errors » Coarse-grain code duplication for soft error protection -5 -

More Project Ideas v Parallelization/SIMDization » CUDA/Open. CL/Matlab/Open. MP to x 86+SSE or Arm+NEON » DOALL loop parallelization, dependence breaking transformations » Stream parallelism analysis tool - profiling tool to identify maximal program regions the exhibit stream (or DOALL) parallelism » Access-execute program decomposition - decompose program into memory access/compute threads v Dynamic optimization (Dynamo, Dalvik VM) » Run-time DOALL loop parallelization » Run-time program analysis for security (taint tracking) » Run-time classic optimization (if they don’t have it already!) v Profiling tools » Distributed control flow/energy profiling (profiling + aggregation) -6 -

And More Project Ideas v VLIW » Superblock formation in LLVM » Acyclic BB scheduler for LLVM » Modulo BB scheduler for LLVM v Binary optimizer » Arm binary to LLVM IR, de-register allocation » X 86 binary (clean) to LLVM IR v Energy » Minimizing instruction bit flips v Misc » Program distillation - create a subset program with equivalent memory/branch behavior » Code sharing analysis for mobile applications -7 -

From Last Time: Class Problem Solution Optimize this applying 1. constant propagation 2. constant folding 3. Dead code elim r 1 = 0 r 2 = 10 r 3 = 0 r 4 = 1 r 7 = r 1 * 4 r 6 = 8 if (r 3 > 0) r 7 = r 1 * 4 if (r 3 > 0) r 2 = 0 r 6 = r 6 * r 7 r 3 = r 2 / r 6 r 3 = r 4 r 3 = r 3 + r 2 r 1 = r 6 r 2 = 0 r 3 = 1 + r 2 r 1 = 8 r 2 = r 2 + 1 r 1 = r 1 + 1 if (r 1 < 100) store (r 1, r 3) -8 -

From Last Time: Class Problem Solution Optimize this applying 1. dead code elimination 2. forward copy propagation 3. CSE r 4 = r 1 r 6 = r 15 r 2 = r 3 * r 4 r 8 = r 2 + r 5 r 9 = r 3 r 2 = r 3 * r 1 r 8 = r 2 + r 5 r 7 = load(r 2) if (r 2 > r 8) r 5 = r 9 * r 4 r 11 = r 2 r 12 = load(r 11) if (r 12 != 0) r 3 = load(r 2) r 10 = r 3 / r 6 r 11 = r 8 store (r 11, r 7) store (r 12, r 3) r 3 = r 7 store (r 8, r 7) if (r 7 != 0) store (r 12, r 3) -9 -

Loop Invariant Code Motion (LICM) v Move operations whose source operands do not change within the loop to the loop preheader r 1 = 3 r 5 = &A » Execute them only 1 x per invocation of the loop » Be careful with memory operations! » Be careful with ops not executed every iteration r 4 = load(r 5) r 7 = r 4 * 3 r 8 = r 2 + 1 r 7 = r 8 * r 4 r 3 = r 2 + 1 r 1 = r 1 + r 7 store (r 1, r 3) - 10 -

LICM (2) v Rules » » » » X can be moved src(X) not modified in loop body X is the only op to modify dest(X) for all uses of dest(X), X is in the available defs set for all exit BB, if dest(X) is live on the exit edge, X is in the available defs set on the edge if X not executed on every iteration, then X must provably not cause exceptions if X is a load or store, then there are no writes to address(X) in loop r 1 = 3 r 5 = &A r 4 = load(r 5) r 7 = r 4 * 3 r 8 = r 2 + 1 r 7 = r 8 * r 4 r 3 = r 2 + 1 r 1 = r 1 + r 7 store (r 1, r 3) - 11 -

Global Variable Migration v Assign a global variable temporarily to a register for the duration of the loop » Load in preheader » Store at exit points v Rules » X is a load or store » address(X) not modified in the loop » if X not executed on every iteration, then X must provably not cause an exception » All memory ops in loop whose address can equal address(X) must always have the same address as X - 12 - r 4 = load(r 5) r 4 = r 4 + 1 r 8 = load(r 5) r 7 = r 8 * r 4 store(r 5, r 4) store(r 5, r 7)

Induction Variable Strength Reduction v v Create basic induction variables from derived induction variables Induction variable r 5 = r 4 - 3 r 4 = r 4 + 1 » BIV (i++) Ÿ 0, 1, 2, 3, 4, . . . » DIV (j = i * 4) Ÿ 0, 4, 8, 12, 16, . . . » DIV can be converted into a BIV that is incremented by 4 v r 7 = r 4 * r 9 r 6 = r 4 << 2 Issues » Initial and increment vals » Where to place increments - 13 -

Induction Variable Strength Reduction (2) v Rules » » » v X is a *, <<, + or – operation src 1(X) is a basic ind var src 2(X) is invariant No other ops modify dest(X) != src(X) for all srcs dest(X) is a register r 5 = r 4 - 3 r 4 = r 4 + 1 Transformation » Insert the following into the preheader Ÿ new_reg = RHS(X) » If opcode(X) is not add/sub, insert to the bottom of the preheader r 7 = r 4 * r 9 Ÿ new_inc = inc(src 1(X)) opcode(X) src 2(X) » else r 6 = r 4 << 2 Ÿ new_inc = inc(src 1(X)) » Insert the following at each update of src 1(X) Ÿ new_reg += new_inc » Change X dest(X) = new_reg - 14 -

Class Problem Optimize this applying induction var str reduction r 1 = 0 r 2 = 0 r 5 = r 5 + 1 r 11 = r 5 * 2 r 10 = r 11 + 2 r 12 = load (r 10+0) r 9 = r 1 << 1 r 4 = r 9 - 10 r 3 = load(r 4+4) r 3 = r 3 + 1 store(r 4+0, r 3) r 7 = r 3 << 2 r 6 = load(r 7+0) r 13 = r 2 - 1 r 1 = r 1 + 1 r 2 = r 2 + 1 r 13, r 12, r 6, r 10 liveout - 15 -

ILP Optimization v Traditional optimizations » Redundancy elimination » Reducing operation count v ILP (instruction-level parallelism) optimizations » Increase the amount of parallelism and the ability to overlap operations » Operation count is secondary, often trade parallelism for extra instructions (avoid code explosion) v ILP increased by breaking dependences » True or flow = read after write dependence » False or (anti/output) = write after read, write after write - 16 -

Register Renaming v v Similar goal to SSA construction, but simpler Remove dependences caused by variable re-use a: r 1 = r 2 + r 3 b: r 3 = r 4 + r 5 b: r 13 = r 4 + r 5 » Re-use of source variables » Re-use of temporaries » Anti, output dependences c: r 1 = r 7 * r 8 c: r 11 = r 7 * r 8 d: r 7 = r 1 + r 5 d: r 17 = r 11 + r 5 e: r 1 = r 3 + 4 e: r 21 = r 13 + 4 f: r 4 = r 7 + 4 f: r 14 = r 17 + 4 Create a new variable to hold each unique life time Very simple transformation with straight-line code » Make each def a unique register » Substitute new name into subsequent uses - 17 -

Global Register Renaming v x= y= =y =x » A single use may have multiple reaching defs v x= y= =y Straight-line code strategy does not work Web = Collection of defs/uses which have possible value flow between them » Identify webs =x Ÿ Take a def, add all uses Ÿ Take all uses, add all reaching defs Ÿ Take all defs, add all uses Ÿ repeat until stable soln y= » Each web renamed if name is the same as another web =y - 18 -

Back Substitution v Generation of expressions by compiler frontends is very sequential y = a + b + c – d + e – f; » Account for operator precedence » Apply left-to-right within same precedence v r 9 = r 1 + r 2 r 10 = r 9 + r 3 r 11 = r 10 - r 4 r 12 = r 11 + r 5 r 13 = r 12 – r 6 Back substitution » Create larger expressions Ÿ Iteratively substitute RHS expression for LHS variable » Note – may correspond to multiple source statements » Enable subsequent optis v Optimization » Re-compute expression in a more favorable manner Subs r 12: r 13 = r 11 + r 5 – r 6 Subs r 11: r 13 = r 10 – r 4 + r 5 – r 6 Subs r 10 r 13 = r 9 + r 3 – r 4 + r 5 – r 6 Subs r 9 r 13 = r 1 + r 2 + r 3 – r 4 + r 5 – r 6 - 19 -

Tree Height Reduction v » » » v original: Re-compute expression as a balanced binary tree Obey precedence rules Essentially re-parenthesize Combine literals if possible after back subs: r 13 = r 1 + r 2 + r 3 – r 4 + r 5 – r 6 Effects » Height reduced (n terms) Ÿ n-1 (assuming unit latency) Ÿ ceil(log 2(n)) » » r 1 + r 2 Number of operations remains constant Cost r 3 – r 4 + Ÿ Temporary registers “live” longer » r 9 = r 1 + r 2 r 10 = r 9 + r 3 r 11 = r 10 - r 4 r 12 = r 11 + r 5 r 13 = r 12 – r 6 Watch out for Ÿ Always ok for integer arithmetic Ÿ Floating-point – may not be!! + r 13 - 20 - r 5 – r 6 final code: t 1 = r 1 + r 2 t 2 = r 3 – r 4 t 3 = r 5 – r 6 t 4 = t 1 + t 2 r 13 = t 4 + t 3

Class Problem Assume: + = 1, * = 3 operand arrival times 0 r 1 0 r 2 0 r 3 1 r 4 2 r 5 0 r 6 r 10 = r 1 * r 2 r 11 = r 10 + r 3 r 12 = r 11 + r 4 r 13 = r 12 – r 5 r 14 = r 13 + r 6 Back susbstitute Re-express in tree-height reduced form Account for latency and arrival times - 21 -

Optimizing Unrolled Loops loop: r 1 = load(r 2) r 3 = load(r 4) r 5 = r 1 * r 3 unroll 3 times r 6 = r 6 + r 5 r 2 = r 2 + 4 r 4 = r 4 + 4 if (r 4 < 400) goto loop: r 1 = load(r 2) iter 1 iter 2 Unroll = replicate loop body n-1 times. Hope to enable overlap of operation execution from different iterations iter 3 Not possible! - 22 - r 3 = load(r 4) r 5 = r 1 * r 3 r 6 = r 6 + r 5 r 2 = r 2 + 4 r 4 = r 4 + 4 r 1 = load(r 2) r 3 = load(r 4) r 5 = r 1 * r 3 r 6 = r 6 + r 5 r 2 = r 2 + 4 r 4 = r 4 + 4 if (r 4 < 400) goto loop

Register Renaming on Unrolled Loop loop: r 1 = load(r 2) iter 1 iter 2 iter 3 loop: r 1 = load(r 2) r 3 = load(r 4) r 5 = r 1 * r 3 r 6 = r 6 + r 5 r 2 = r 2 + 4 r 4 = r 4 + 4 if (r 4 < 400) goto loop iter 1 iter 2 iter 3 - 23 - r 3 = load(r 4) r 5 = r 1 * r 3 r 6 = r 6 + r 5 r 2 = r 2 + 4 r 4 = r 4 + 4 r 11 = load(r 2) r 13 = load(r 4) r 15 = r 11 * r 13 r 6 = r 6 + r 15 r 2 = r 2 + 4 r 4 = r 4 + 4 r 21 = load(r 2) r 23 = load(r 4) r 25 = r 21 * r 23 r 6 = r 6 + r 25 r 2 = r 2 + 4 r 4 = r 4 + 4 if (r 4 < 400) goto loop

Register Renaming is Not Enough! loop: r 1 = load(r 2) iter 1 iter 2 iter 3 = load(r 4) r 5 = r 1 * r 3 r 6 = r 6 + r 5 r 2 = r 2 + 4 r 4 = r 4 + 4 r 11 = load(r 2) r 13 = load(r 4) r 15 = r 11 * r 13 r 6 = r 6 + r 15 r 2 = r 2 + 4 r 4 = r 4 + 4 r 21 = load(r 2) r 23 = load(r 4) r 25 = r 21 * r 23 r 6 = r 6 + r 25 r 2 = r 2 + 4 r 4 = r 4 + 4 if (r 4 < 400) goto loop v v Still not much overlap possible Problems » r 2, r 4, r 6 sequentialize the iterations » Need to rename these v 2 specialized renaming optis » Accumulator variable expansion (r 6) » Induction variable expansion (r 2, r 4) - 24 -

Accumulator Variable Expansion r 16 = r 26 = 0 loop: r 1 = load(r 2) r 3 = load(r 4) r 5 = r 1 * r 3 iter 1 r 6 = r 6 + r 5 r 2 = r 2 + 4 r 4 = r 4 + 4 r 11 = load(r 2) r 13 = load(r 4) r 15 = r 11 * r 13 iter 2 r 16 = r 16 + r 15 r 2 = r 2 + 4 r 4 = r 4 + 4 r 21 = load(r 2) r 23 = load(r 4) r 25 = r 21 * r 23 iter 3 r 26 = r 26 + r 25 r 2 = r 2 + 4 r 4 = r 4 + 4 if (r 4 < 400) goto loop r 6 = r 6 + r 16 + r 26 v Accumulator variable » x = x + y or x = x – y » where y is loop variant!! v v - 25 - Create n-1 temporary accumulators Each iteration targets a different accumulator Sum up the accumulator variables at the end May not be safe for floatingpoint values

Induction Variable Expansion r 12 = r 2 + 4, r 22 = r 2 + 8 r 14 = r 4 + 4, r 24 = r 4 + 8 r 16 = r 26 = 0 v » x = x + y or x = x – y » where y is loop invariant!! loop: r 1 = load(r 2) iter 1 iter 2 iter 3 = load(r 4) r 5 = r 1 * r 3 r 6 = r 6 + r 5 r 2 = r 2 + 12 r 4 = r 4 + 12 r 11 = load(r 12) r 13 = load(r 14) r 15 = r 11 * r 13 r 16 = r 16 + r 15 r 12 = r 12 + 12 r 14 = r 14 + 12 r 21 = load(r 22) r 23 = load(r 24) r 25 = r 21 * r 23 r 26 = r 26 + r 25 r 22 = r 22 + 12 r 24 = r 24 + 12 if (r 4 < 400) goto loop r 6 = r 6 + r 16 + r 26 Induction variable v v v - 26 - Create n-1 additional induction variables Each iteration uses and modifies a different induction variable Initialize induction variables to init, init+step, init+2*step, etc. Step increased to n*original step Now iterations are completely independent !!

Better Induction Variable Expansion r 16 = r 26 = 0 loop: r 1 = load(r 2) r 3 = load(r 4) r 5 = r 1 * r 3 iter 1 r 6 = r 6 + r 5 iter 2 iter 3 v With base+displacement addressing, often don’t need additional induction variables » Just change offsets in each iterations to reflect step » Change final increments to n * original step r 11 = load(r 2+4) r 13 = load(r 4+4) r 15 = r 11 * r 13 r 16 = r 16 + r 15 r 21 = load(r 2+8) r 23 = load(r 4+8) r 25 = r 21 * r 23 r 26 = r 26 + r 25 r 2 = r 2 + 12 r 4 = r 4 + 12 if (r 4 < 400) goto loop r 6 = r 6 + r 16 + r 26 - 27 -

Homework Problem loop: r 1 = load(r 2) r 5 = r 6 + 3 r 6 = r 5 + r 1 r 2 = r 2 + 4 if (r 2 < 400) goto loop Optimize the unrolled loop: r 1 = load(r 2) r 5 = r 6 + 3 r 6 = r 5 + r 1 r 2 = r 2 + 4 if (r 2 < 400) goto loop Renaming Tree height reduction Ind/Acc expansion - 28 -