Spatial Computation Computing without GeneralPurpose Processors Mihai Budiu

Spatial Computation Computing without General-Purpose Processors Mihai Budiu mihaib@cs. cmu. edu Carnegie Mellon University July 8, 2004

Spatial Computation A computation model based on: Mihai Budiu • application-specific mihaib@cs. cmu. edu hardware Mellon University • no Carnegie interpretation • minimal resource sharing 2

The Engine Behind This Talk main( ) { signal(SIGINT, welcome); while (slides( ) && time( )) { talk( ); } } 3

Research Scope Object: future architectures Tool: compilers Evaluation: simulators 4

Research Methodology Y (e. g. , cost) state-of-the-art “reasonable limits” incremental evolution Constraint Space X (e. g. , power) new solutions 5

Outline • Introduction: problems of current architectures 100 • • 2000 1998 1996 1994 1992 1990 1988 1986 1984 1 1982 10 1980 Performance 1000 Compiling Application-Specific Hardware Pipelining ASH Evaluation Conclusions 6

Resources [Intel] • We do not worry about not having hardware resources • We worry about being able to use hardware resources 7

2009 2007 2005 2003 i s e D 2001 104 1999 105 1997 p i h 1995 107 1993 106 1991 1989 1987 1985 1983 1981 Transistors Design Complexity 1010 109 108 e z i s C r p r gne ty i v i t oduc 8

Communication vs. Computation gate wire 5 ps 20 ps Power consumption on wires is also dominant 9

Power Consumption Toasted CPU: about 2 sec after removing cooler. (Tom’s Hardware Guide) 10

Energy Efficiency ALUs Pentium 4 11

Clock Speed 3 GHz 6 GHz 10 GHz Cannot rely on global signals (clock is a global signal) 12

Instruction-Set Architecture Software ISA VERY rigid to changes (e. g. x 86 vs Itanium) Hardware 13

Our Proposal • ASH addresses these problems • ASH is not a panacea • ASH “complementary” to CPU Low ILP computation + OS + VM CPU ASH High-ILP computation $ Memory 14

Outline • Problems of current architectures • CASH: Compiling ASH – program representation – compiling C programs • Pipelining • ASH Evaluation • Conclusions 15

Application-Specific Hardware C program Compiler Dataflow IR SW ISA HW HW backend Reconfigurable/custom hw 16

Application-Specific Hardware Soft C program Compiler Dataflow IR SW backend CPU [predication] 17

Key: Intermediate Representation Our IR Traditionally • SSA + predication + speculation • Uniform for scalars and memory may-dep. CFG . . . • Explicitly encodes may-depend • Executable • Precise semantics def-use • Dataflow IR • Close to asynchronous target 18

Computation = Dataflow Programs Circuits a x = a & 7; . . . y = x >> 2; 7 & x 2 >> • Operations ) functional units • Variables ) wires • No interpretation 19

Basic Computation + latch data ack valid 20

Asynchronous Computation + data 1 latch 5 + + + ack valid 2 + 3 + 6 4 + 7 + 8 21

Distributed Control Logic global FSM ack rdy + short, local wires asynchronous control 22

Outline • Problems of current architectures • CASH: Compiling ASH – program representation – compiling C programs • Pipelining • ASH Evaluation • Conclusions 23

MUX: Forward Branches b if (x > 0) y = -x; else y = b*x; x * 0 - f > ! y SSA = no arbitration Conditionals ) Speculation critical path 24

Control Flow ) Data Flow data f Merge (label) data predicate Gateway Split (branch) p ! 25

0 Loops i int sum=0, i; for (i=0; i < 100; i++) sum += i*i; return sum; * 0 +1 < 100 sum + ! ret 26

Predication and Side-Effects addr token sequencing of side-effects no speculation pred Load data to memory token 27

Memory Access LD ST pipelined arbitrated network Monolithic Memory LD local communication global structures Future work: fragment this! complexity related work 28

CASH Optimizations • SSA-based optimizations – unreachable/dead code, gcse, strength reduction, loop-invariant code motion, software pipelining, reassociation, algebraic simplifications, induction variable optimizations, loop unrolling, inlining • Memory optimizations – dependence & alias analysis, register promotion, redundant load/store elimination, memory access pipelining, loop decoupling • Boolean optimizations – Espresso CAD tool, bitwidth analysis 29

Outline • Problems of current architectures • Compiling ASH • Pipelining • Evaluation: CASH vs. clocked designs • Conclusions 30

Pipelining i * + <= pipelined multiplier (8 stages) int sum=0, i; for (i=0; i < 100; i++) sum += i*i; return sum; 100 1 sum + step 1 31

Pipelining i * 100 1 + <= sum + step 2 32

Pipelining i * 100 1 + <= sum + step 3 33

Pipelining i * 100 1 + <= sum + step 4 34

Pipelining i i=1 100 1 + <= i=0 sum + step 5 35

Pipelining i * i=1 100 1 + <= i=0 sum + step 6 36

Pipelining i * + <= i’s loop predicate 100 1 Long latency pipe sum’s loop + step 7 37

Pipelining i * i’s loop critical path 100 1 + <= Predicate ack edge is on the critical path. sum’s loop + 38

Pipeline balancing * i 100 1 + <= i’s loop decoupling FIFO sum’s loop + step 7 39

Pipeline balancing i * i’s loop 100 1 + <= critical path decoupling FIFO sum’s loop + 40

Outline • • Problems of current architectures Compiling ASH Pipelining Evaluation: CASH vs. clocked designs • Conclusions 41

Evaluating ASH C Mediabench kernels (1 hot function/benchmark) CASH core Verilog back-end Synopsys, Cadence P/R 180 nm std. cell library, 2 V Model. Sim Mem (Verilog simulation) ASIC ~1999 technology performance numbers 42

ASH Area P 4: 217 minimal RISC core normalized area 43

ASH vs 600 MHz CPU [. 18 m] 44

Bottleneck: Memory Protocol ST LSQ LD Memory • Token release to dependents: requires round-trip to memory. • Limit study: round trip zero time ) up to 6 x speed-up. • Exploring protocol for in-order data delivery & fast token release. 45

Power DSP 110 m. P 4000 Xeon [+cache] 67000 46

Energy Efficiency 1000 x Dedicated hardware ASH media kernels Asynchronous P FPGAs General-purpose DSP Microprocessors 0. 01 0. 1 1 10 1000 Energy Efficiency [Operations/n. J] 47

Outline Problems of current architectures + Compiling ASH + Pipelining + ASH Evaluation = Future/related work & conclusions 48

Related Work Asynchronous circuits Nanotechnology Embedded systems Reconfigurable computing Dataflow machines High-level synthesis Computer architecture Compilation 49

Future Work • Optimizations for area/speed/power • Memory partitioning • Concurrency • Compiler-guided layout • Explore extensible ISAs • Hybridization with superscalar mechanisms • Reconfigurable hardware support for ASH • Formal verification 50

Grand Vision: Certified Circuit Generation • Translation validation: input ´ output • Preserve input properties – e. g. , C programs cannot deadlock – e. g. , type-safe programs cannot crash • Debug, test, verify only at source-level HLL IR IRopt Verilog gates layout formally validated How far can you go? 51

Conclusions Spatial computation strengths Feature No interpretation Advantages Energy efficiency, speed Spatial layout Short wires, no contention Asynchronous Low power, scalable Distributed No global signals Automatic compilation Design productivity, no ISA 52

• Reconfigurable hardware • Critical paths • Control logic • ASH vs. . . • ASH weaknesses • Exceptions • Normalized area • Why C? • Splitting memory • More performance • Recursive calls Backup Slides 53

Reconfigurable Hardware Interconnection network Universal gates and/or storage elements Programmable switches 54

Main RH Ingredient: RAM Cell a 0 a 1 0 0 0 1 data a 0 a 1 & a 2 Universal gate = RAM data in control 0 Switch controlled by a 1 -bit RAM cell back 55

Critical Paths b if (x > 0) y = -x; else y = b*x; x * 0 - > ! y 56

Lenient Operations b if (x > 0) y = -x; else y = b*x; x * 0 - > ! y Solves the problem of unbalanced paths back to talk 57

Asynchronous Control rdyin D ackout C ackin rdyout datain back to talk Reg = dataout

HLL to HW High-level Synthesis Reconfigurable Computing Asynchronous circuits Behavioral HDL C [subsets] Concurrent Language Synchronous Hardware configuration Asynchronous Hardware (spatial computation) Prior work This research 59

CASH vs High-Level Synthesis • CASH: the only existing tool to translate complete ANSI C to hardware • CASH generates asynchronous circuits • CASH does not treat C as an HDL – no annotations required – no reactivity model – does not handle non-C, e. g. , concurrency back 60

ASH Weaknesses • • • Low efficiency for low-ILP code Does not adapt at runtime Monolithic memory Resource waste Not flexible No support for exceptions 61

ASH Weaknesses (2) • Both branch and join not free • Static dataflow (no re-issue of same instr) • Memory is “far” • Fully static – No branch prediction – No dynamic unrolling – No register renaming • Calls/returns not lenient back 62

Branch Prediction i ASH crit path for (i=0; i < N; i++) {. . . CPU crit path 1 + < exception if (exception) break; } Predicted not taken Effectively a noop for CPU! back Predicted taken. ! & result available before inputs 63

Exceptions • Strictly speaking, C has no exceptions • In practice hard to accommodate exceptions in hardware implementations • An advantage of software flexibility: PC is single point of execution control Low ILP computation + OS + VM + exceptions CPU $$$ ASH High-ILP computation Memory back 64

Why C • Huge installed base • Embedded specifications written in C • Small and simple language – Can leverage existing tools – Simpler compiler • Techniques generally applicable • Not a toy language back 65

Performance 66

Parallelism Profile 67

Normalized Area back to talk 68

Memory Partitioning • MIT RAW project: Babb FCCM ‘ 99, Barua Hi. PC ‘ 00, Lee ASPLOS ‘ 00 • Stanford Sp. C: Semeria DAC ‘ 01, TVLSI ‘ 02 • Berkeley CCured: Necula POPL ‘ 02 • Illinois Flex. RAM: Fraguella PPo. PP ‘ 03 • Hand-annotations #pragma back to talk 69

Memory Complexity RAM addr LSQ data back to talk 70

Recursion save live values recursive call restore live values back stack 71

Me? 72