VIRAM Compiler Benchmarks and Applications Adam Janin David

V-IRAM Compiler Benchmarks and Applications Adam Janin, David Judd, Christoforos Kozyrakis, David Martin, Thinh Nguyen, Randi Thomas, David Patterson, Kathy Yelick Slide 1

Overview of V-IRAM Benchmarks • Hand-Coded Benchmarks – Media Kernels – FFT – H. 263 Video Encoder Application • Compiled Benchmarks – Matrix vector multiplication and VIRAM addressing – Floating point benchmarks – Integer benchmarks • Underway – Speech Application (Janin poster) – Scientific (DOE) Applications (Li and Oliker poster) – Data-intensive (DARPA DIS) Benchmarks (Gaeke) Slide 2

Hand-Coded Applications Update • Image processing kernels (old FPU model) – Note BLAS-2 performance Slide 3

Media Kernel Comparisons • All numbers in cycles/pixel • VIRAM uses 4 lanes, 1 sub-bank per bank • MMX and VIS results assume all data in L 1 cache Slide 4

FFT: Uses In-Register Permutations Without in-register permutations Slide 5

Scalable Design 4 lanes, 8 MB 2 lanes, 4 MB 1 lane, 2 MB 3. 2 Gops (32 -bit) 1. 6 Gops . 8 Gops • Scaling number of lanes for performance, energy, area • Number of DRAM banks may scale independently – e. g. , 16 banks rather than 8 • May also scale up to 8 lanes • Single executable file used across lane scaling experiments Slide 6

Vector Architectural State Virtual Processors (vl) VP 0 Data Registers VP 1 VPvl-1 vr 0 vr 1 vr 31 • vpw Number of VPs given by the Vector Length register vl Width of each VP given by the register vpw • vpw is one of {8 b, 16 b, 32 b, 64 b} Maximum vector length is given by a read-only register mvl • – – mvl depends on implementation and vpw: {128, 64, 32} in VIRAM-1 Slide 7

VIRAM Compiler Frontends C C++ Fortran 95 Optimizer Cray’s PDGCS Code Generators T 3 D/T 3 E C 90/T 90/SV 1 SV 2/VIRAM • Based on the Cray’s production compiler • Challenges: – narrow data types and scalar/vector memory consistency • Advantages relative to media-extensions: – powerful addressing modes and ISA independent of datapath width Slide 8

Compiler Challenges • Can compiled code effectively use VIRAM design? – Is on-chip DRAM bandwidth sufficient? – How well do multimedia applications vectorize? – Does VIRAM’s model of variable width data (VPW) fit into a compilation framework? Slide 9

Matrix-Vector Multiplication Destination vector Source vector Matrix Assume row layout • Matrix vector multiply – dot: 2 vloads, (both unit stride + a reduction) – saxpy: 2 vloads, 1 vstore (2 strided + 1 unit) • Vector matrix multiply (= mvm with column layout) – saxpy: 2 vloads, 1 vstore (all unit stride) • Sparse matrix-vector multiply – dot: 3 vloads (1 indexed, 2 unit + reduction) – saxpy: 3 vloads, 1 vstore (2 indexed, 2 unit) (column layout) Slide 10

Matrix Vector Multiplication • Performance of various source optimizations Column layout mvm = row layout vmm Column performance ~= peak Slide 11

Comparison of MVM Performance • Double precision floating point – compiled for VIRAM (note: chip only does single) – hand- or Atlas-optimized for other machines MFLOPS 100 x 100 matrix As matrix size increases, performance: – drops on cachebased designs – increases on vector designs – but 64 x 64 about 20% better on VIRAM Slide 12

Sparse MVM Performance • Performance is matrix-dependent: lp matrix – compiled for VIRAM using “independent” pragma » sparse column layout – Sparsity-optimized for other machines MFLOPS » sparse row (or blocked row) layout Slide 13

Generating Code for Variable VPW • Strategy: vectorizer determines minimum correct vpw for each loop nest – Vectorizer assumes vpw=64 initially – At end of vectorization, discard vectorized copy of loop if greatest width encountered is less than 64 and start vectorization over with new vpw. – Code gen checks vpw for each loop nest. • Limitation: a single loop nest will run at the speed of the widest type. – Reason: simplicity & performance of the common case – No attempt to split/combine loops based on vpw Slide 14

Media Benchmarks • Mostly from U Toronto’s benchmark suite • 8 -bit data, 16 -bit operations – Colorspace: strided loads/stores – Composition: unit stride – Convolve: strided • Mixed 16 and 32 -bit integer – Detect – Decrypt • 32 -bit Floating point – FIR filter – SAXPY 64: 64 element – SAXPY 1 K: 1024 element – matmul: matrix multiplication Slide 15

Integer Benchmarks • Strided access important, e. g. , RGB – narrow types limited by address generation • Outer loop vectorization and unrolling used – helps avoid short vectors – spilling can be a problem • Tiling could probably help Slide 16

Floating Point DSP Benchmarks • Performance is competitive with hand-coding • Vector length is important (e. g. , saxpy) – but multiple vectors is fine (e. g. , matmul) Slide 17

Conclusions • VIRAM ISA shows high performance on compiled code – competitive with modern processors – limitations are address generation for strided and indexed memory operations • Compiler effectively uses variable width data – allows media applications to vectorize – performance scales with inverse data width • Future compiler work – Fixed point support – Better register allocation Slide 18

Backup slides Slide 19

Performance Summary • Performance of compiled code is generally good – matmul and saxpy meet or beat hand-coded – 3 addressing modes very useful • Limitations to performance – Dependencies or inadequate compiler analysis – Inadequate memory bandwidth – Lack of address generators – Short vectors • Future compiler work – Tiling – Fixed point support – Better register allocation Slide 20

Scaling Media Benchmarks Slide 21

Compiled matrix-vector multiplication: 2 Flops/element • Easy compilation problem; stresses memory bandwidth • Compare to 304 Mflops (64 -bit) for Power 3 (handcoded) – Performance scales with number of lanes up to 4 – Need more memory banks than default DRAM macro for 8 lanes Slide 22

Outline • Why vectors for IRAM? – Including media types • The virtual lane model • Virtual processor width • Limitations to performance – Dependencies or inadequate compiler analysis – Inadequate memory bandwidth – Lack of address generators – Short vectors • Comparisons to other architectures • Conclusions Slide 23

Matrix-Vector Multiply • Scaling Matrix-Vector Multiplication Slide 24

Performance on Media Benchmarks • Using compiled code: 1, 2, 4, and 8 lanes Slide 25

Compiled matrix-vector multiplication: 2 Flops/element MFLOPS • Easy compilation problem; stresses memory bandwidth • Compare to 304 Mflops (64 -bit) for Power 3 (handcoded) – Performance scales with number of lanes up to 4 – Need more memory banks than default DRAM macro for 8 lanes Slide 26

Compiling Media Kernels on IRAM • The compiler generates code for narrow data widths, e. g. , 16 bit integer • Compilation model is simple, more scalable (across generations) than MMX, VIS, etc. – Strided and indexed loads/stores simpler than pack/unpack – Maximum vector length is longer than datapath width (256 bits); all lane scalings done with single executable Slide 27

Vector Vs. SIMD: Example • Simple image processing example: – conversion from RGB to YUV Y = [( 9798*R + 19235*G + 3736*B) / 32768] U = [(-4784*R - 9437*G + 4221*B) / 32768] + 128 V = [(20218*R – 16941*G – 3277*B) / 32768] + 128 Slide 28

VIRAM Code (22 instructions) RGBto. YUV: vlds. u. b xlmul. u. sv xlmadd. u. sv vsra. vs vadd. sv vsts. b subu bnez r_v, r_addr, stride 3, g_v, g_addr, stride 3, b_v, b_addr, stride 3, o 1_v, t 0_s, r_v o 1_v, t 1_s, g_v o 1_v, t 2_s, b_v o 1_v, s_s o 2_v, t 3_s, r_v o 2_v, t 4_s, g_v o 2_v, t 5_s, b_v o 2_v, s_s o 2_v, a_s, o 2_v o 3_v, t 6_s, r_v o 3_v, t 7_s, g_v o 3_v, t 8_s, b_v o 3_v, s_s o 3_v, a_s, o 3_v o 1_v, y_addr, stride 3, o 2_v, u_addr, stride 3, o 3_v, v_addr, stride 3, pix_s, len_s pix_s, RGBto. YUV addr_inc # # load R load G load B calculate Y # calculate U # calculate V addr_inc # store Y # store U # store V Slide 29

MMX Code (part 1) RGBto. YUV: movq mm 1, pxor mm 6, movq mm 0, psrlq mm 1, punpcklbw movq mm 7, punpcklbw movq mm 2, pmaddwd mm 0, movq mm 3, pmaddwd mm 1, movq mm 4, pmaddwd mm 2, movq mm 5, pmaddwd mm 3, punpckhbw pmaddwd mm 4, paddd mm 0, pmaddwd mm 5, movq mm 1, paddd mm 2, movq mm 6, [eax] mm 6 mm 1 16 mm 0, mm 1, mm 0 YR 0 GR mm 1 YBG 0 B mm 2 UR 0 GR mm 3 UBG 0 B mm 7, VR 0 GR mm 1 VBG 0 B 8[eax] mm 3 mm 1 ZEROS mm 6; paddd mm 4, movq mm 5, psllq mm 1, paddd mm 1, punpckhbw movq mm 3, pmaddwd mm 1, movq mm 7, pmaddwd mm 5, psrad mm 0, movq TEMP 0, movq mm 6, pmaddwd mm 6, psrad mm 2, paddd mm 1, movq mm 5, pmaddwd mm 7, psrad mm 1, pmaddwd mm 3, packssdw pmaddwd mm 5, psrad mm 4, movq mm 1, mm 5 mm 1 32 mm 7 mm 6, ZEROS mm 1 YR 0 GR mm 5 YBG 0 B 15 mm 6 mm 3 UR 0 GR 15 mm 7 UBG 0 B 15 VR 0 GR mm 0, mm 1 VBG 0 B 15 16[eax] Slide 30

MMX Code (part 2) paddd mm 6, movq mm 7, psrad mm 6, paddd mm 3, psllq mm 7, movq mm 5, psrad mm 3, movq TEMPY, packssdw movq mm 0, punpcklbw movq mm 6, movq TEMPU, psrlq mm 0, paddw mm 7, movq mm 2, pmaddwd mm 2, movq mm 0, pmaddwd mm 7, packssdw add eax, add edx, movq TEMPV, mm 7 mm 1 15 mm 5 16 mm 7 15 mm 0 mm 2, TEMP 0 mm 7, mm 0 mm 2 32 mm 0 mm 6 YR 0 GR mm 7 YBG 0 B mm 4, 24 8 mm 4 mm 6 ZEROS mm 3 movq mm 4, pmaddwd mm 6, movq mm 3, pmaddwd mm 0, paddd mm 2, pmaddwd pxor mm 7, pmaddwd mm 3, punpckhbw paddd mm 0, movq mm 6, pmaddwd mm 6, punpckhbw movq mm 7, paddd mm 3, pmaddwd mm 5, movq mm 4, pmaddwd mm 4, psrad mm 0, paddd mm 0, psrad mm 2, paddd mm 6, movq mm 5, mm 6 UR 0 GR mm 0 UBG 0 B mm 7 mm 4, mm 7 VBG 0 B mm 1, mm 6 mm 1 YBG 0 B mm 5, mm 5 mm 4 YR 0 GR mm 1 UBG 0 B 15 OFFSETW 15 mm 7 Slide 31

MMX Code (pt. 3: 121 instructions) pmaddwd mm 7, psrad mm 3, pmaddwd mm 1, psrad mm 6, paddd mm 4, packssdw pmaddwd mm 5, paddd mm 7, psrad mm 7, movq mm 6, packssdw movq mm 4, packuswb movq mm 7, paddd mm 1, paddw mm 4, psrad mm 1, movq [ebx], packuswb movq mm 5, packssdw paddw mm 5, paddw mm 3, UR 0 GR 15 VBG 0 B 15 OFFSETD mm 2, VR 0 GR mm 4 15 TEMPY mm 0, TEMPU mm 6, OFFSETB mm 5 mm 7 15 mm 6 mm 4, TEMPV mm 3, mm 7 mm 6 movq [ecx], mm 4 packuswb mm 5, add ebx, 8 add ecx, 8 movq [edx], mm 5 dec edi jnz RGBto. YUV mm 3 mm 7 mm 2 mm 4 Slide 32

IRAM Status • Chip – ISA has not changed significantly in over a year – Verilog complete, except SRAM for scalar cache – Testing framework in place • Compiler – Backend code generation complete – Continued performance improvements, especially for narrow data widths • Application & Benchmarks – Handcoded kernels better than MMX, VIS, gp DSPs » DCT, FFT, MVM, convolution, image composition, … – Compiled kernels demonstrate ISA advantages » MVM, sparse MVM, decrypt, image composition, … – Full applications: H 263 encoding (done), speech (underway) Slide 33

Backup from Dave Judd’s Talk Slide 34

VIRAM Tools • • • vas: assembler vdis: disassembler vsim-isa: simulator vsim-db: debugger vsim-p: performance simulator vsim-sync: memory consistency simulator Slide 35

Compiler Testing • C regression test suite (commercial test suite) – Scalar emphasis, C conformance – All tests pass except: » Small numerical differences due to lack on 128 f. p. support • C++ test suite – 1167 of 1183 tests execute correctly. – 12 failures in compilation: “undefined variables” – 4 failures in execution: bad answers Slide 36

Compiler Testing • Vector regression test suites (CRAY) – Specifically tests for vectorization – Compares vector and scalar results – Easy to isolate problems – “vector” status: » 59 of 62 tests pass » Some minor numerical differences » 1 bad answer, 2 integer overflow – “vector 4” status » 163 of 165 tests execute correctly » 1 bad anwer, 1 illegal use of vector inst. Slide 37

Kernel Performance: mvm matrix-vector multiplication 64 x 64, 32 bit floating pt. Hand optimized assembly code 579 mflops vcc w/ restrict keywords added 352 mflops + 1 element padding to 401 mflops avoid bank conflicts + shortloop directive Loops interchanged & outer loop vectorized by vcc. 592 mflops Slide 38

Mods to mvm code /* Original code mvm. c */ void mvm (float * A, float * X, float * Y, int n, int acol ) { int i, j; float x_elem if ( n <= 64 ) { for (i = 0; i < n; i++) { for (j = 0; j < n; j++) { Y[j] += A[j*acol+i] * x_elem; } } /* Modified code */ void mvm (float * restrict A, float * restrict X, float * restrict Y, int n, int acol ) { int i, j; < if ( n <= 64 ) { for (i = 0; i < n; i++) { #pragma shortloop for (j = 0; j < n; j++) { Y[j] += A[j*acol+i] * X[i]; } } Slide 39

Kernel performance: mm_mul matrix –matrix multiplication 64 x 64, 32 bit float, 1. 6 gigaflop theoretical peak Hand coded assembly 1. 58 gigaflops mm-mul-small. s vcc w/ restrict and shortloop keywords 0. 852 gigaflops + inner two loops in separate function, allows outer loop vectorization 1. 51 gigaflops Slide 40

Kernel performance: saxpy • 32 bit floating point ops N=64 256 1024 4096 Hand coded assembly 379 593 691 720 vcc w/restrict keywords 385 596 692 721 Slide 41

Kernel performance: motion_estimate 32 bit integer ops, finding the minimum sum of absolute differences for a reference block and a region in an image. Hand optimized assembly 1. 181 gigaops vcc w/restrict keywords 170 mops + shortloop directives 253 mops + outer loop unroll directive 257 mops* *No improvement because of spilling. Slide 42

Dongarra loops • • • 100 loops to test compiler vectorization capability Rewritten in C by Cray (? ) vcc vectorizes 74 loops vcc partially vectorizes 3 loops vcc conditionally vectorizes 3 loops 1 loop not vectorized because vector sin/cos not currently available on viram. • 19 other loops not vectorized • Data provided by Sam Williams Slide 43

Features Remaining: • Support version 3 isa and version 4 isa: – Isa changes required by Mips Inc. scalar core – Performance simulator only supports “old”isa • Finish sync support – take advantage of Cray implementation • VIRAM machine “target” – Allow easier maintainence of frontend and optimizer mods for viram • User documentation – Summary of differences w/Cray compiler – Useful options, hints for vector code Slide 44

Performance Features Remaining • • Additional tuning: instruction scheduler Support new SV 2 inliner for C/C++ Shortloop enhancements Reduce spilling – Scheduler concern with registers – Ordering of blocks for register assignment within “priority groups” – Special vector registers carried across calls • Loop unrolling for vector loops • Tune for key benchmarks Slide 45

Other Future Compiler Features ? • Support for speculative execution • Compiler extensions for fixed point hardware • Support for vector functions; vector mlib Slide 46

Summary • vcc is a reasonably robust compiler for VIRAM • Performance on kernels is good w/appropriate directives, some effort for optimum vectorization • Need to prioritize remaining work Slide 47

Codegen/optimizer issues for VIRAM • Variable virtual processor width (VPW) • Variable maximum vector register length (MVL) • Vector flag registers treated as 1 bit wide vector register • Multiple base, incr, stride regs. + autoincrement • Fixed point arithmetic (saturating add, etc. ) • Memory consistency • New vector instructions not available on SV 2 Slide 48

Generating Code for Variable MVL • Maximum vector length is not specified in IRAM ISA. • However, compiler assumes mvl at compile time – mvl based on vpw – mvl assumption dependent on VIRAM-1 hardware implementation – Recompiling required for future hardware versions if mvl changes • MVL knowledge useful for code gen and vectorizer: – register spilling – short loop vectorization – length-dependent vectorization ( and may eliminate safe vector length computation at run time) for (i = 0; i < n; i=++) a[i] = a[i+32] Slide 49

Why Vectors? • Utilizes on-chip bandwidth of IRAM – parallelism within instructions • Efficient architecture for vectorizable code – avoids area, power, and design of reorder logic – low instruction decode overhead • Multimedia algorithms are vectorizable – e. g. , vectorize across pixels in an image • Scales easily across chip generations – e. g. , 32 -way parallelism in instruction can be implemented by 1, 2, 4, 8 -way • Leverages well-known compiler technology Slide 50

Architecture Details • MIPS 64™ 5 Kc core (200 MHz) – Single-issue scalar core with 8 Kbyte I&D caches • Vector unit (200 MHz) – 8 KByte register file (32 64 b elements per register) – 256 b datapaths, can be subdivided into 16 b, 32 b, 64 b: » 2 arithmetic (1 FP, single), 2 flag processing – Memory unit » 4 address generators for strided/indexed accesses • Main memory system – 8 2 -MByte DRAM macros » 25 ns random access time, 7. 5 ns page access time – Crossbar interconnect » 12. 8 GBytes/s peak bandwidth per direction (load/store) • Off-chip interface – 2 channel DMA engine and 64 n Sys. AD bus Slide 51

Floorplan 20. 0 mm 14. 5 mm • Technology: IBM SA-27 E – 0. 18 mm CMOS, 6 metal layers • 290 mm 2 die area – 225 mm 2 for memory/logic • Transistor count: ~150 M • Power supply – 1. 2 V for logic, 1. 8 V for DRAM • Typical power consumption: 2. 0 W – 0. 5 W (scalar) + 1. 0 W (vector) + 0. 2 W (DRAM) + 0. 3 W (misc) • Peak vector performance – 1. 6/3. 2/6. 4 Gops wo. multiplyadd (64 b/32 b/16 b operations) – 3. 2/6. 4 /12. 8 Gops w. madd – 1. 6 Gflops (single-precision) • Tape-out planned for Spring ‘ 01 Slide 52

Micro-kernel results: simulated systems • Note : simulations performed with 2 load-store units and without decoupled stores or optimizations for strides 2, 3, and 4 Slide 53

Micro-kernels • Vectorization and scheduling performed manually Slide 54

Scaled system results • Near linear speedup for all application apart from i. DCT • i. DCT bottlenecks • large number of bank conflicts • 4 addresses/cycle for strided accesses Slide 55

FFT Floating Point Performance 32 bit Floating Point • Note: Pathfinder is a board with multiple FPGAs implementing a kind of block floating point that gives floating point-accurate results. Slide 56

FFT Fixed Point Performance 16 bit Fixed Point • VIRAM is competitive with high-end DSPs • Specialized “FFT chips” can do better: – CRI Scorpio 24 bit complex fixed point FFT DSP: » 1024 pt = 7 microseconds Slide 57

DCT Performance for Video Encoder • 8 x 8 DCT performance in processor cycles – Used in H. 263 encoder VIRAM (4 sub-banks, LLM) 85 Tri. Media TM-1000, Philips 160 (1. 88 x) TI TMS 320 C 62 230 (2. 71 x) Power. PC with Altivec 102 (1. 20 x) HP PA-8000 with MAX 2 147 (1. 73 x) Intel Pentium II + MMX 500 (5. 88 x) NEC V 830/A 201 (2. 36 x) • Variations on VIRAM: – 166 cycles without sub-banks – 59 cycles for VIRAM using 16 -bit data Slide 58