Krste CS 252 Feb 27 2006 Lecture 12
Krste CS 252 Feb. 27, 2006 Lecture 12, Slide 1 14: Vector Computers: an oldfashioned approach
Modern Vector Computers Krste CS 252 Feb. 27, 2006 Lecture 12, Slide 2 • Modern vector machines have relatively short vectors – ARM NEON: 16 bytes, Intel SSE: 16 bytes, Intel AVX: 32 bytes – But the trend is for them to grow longer – AVX-512 has 64 byte vectors, i. e. 16 floats • Older vector machines used much longer vectors – Cray 1 vector supercomputer used vectors of 64 floats, i. e. 256 bytes • Modern vector machines always use vector registers for everything – Modern short vector registers of 16 or 32 bytes don’t require much space on chip • Some (but not all) older vector machines took data directly from memory
Modern Vector Computers Krste CS 252 Feb. 27, 2006 Lecture 12, Slide 3 • Modern vector machines use a separate arithmetic/floating point unit per lane – Four parallel floating point adders/multipliers in SSE implementations • Older vector machines used as little as one arithmetic/floating point unit to implement vector instruction – But very deeply pipelined – Goal was to push as much work through the pipelined FP unit as possible • Vector architectures are increasingly important – Especially for low-energy computation – It’s worthwhile looking back to the time when vector computers were last really popular and successful
Supercomputers Krste CS 252 Feb. 27, 2006 Lecture 12, Slide 4 Definition of a supercomputer: • Fastest machine in world at given task • A device to turn a compute-bound problem into an I/O bound problem • Any machine costing $30 M+ • Any machine designed by Seymour Cray CDC 6600 (Cray, 1964) regarded as first supercomputer
Supercomputer Applications Krste CS 252 Feb. 27, 2006 Lecture 12, Slide 5 Typical application areas • Military research (nuclear weapons, cryptography) • Scientific research • Weather forecasting • Oil exploration • Industrial design (car crash simulation) All involve huge computations on large data sets In 70 s-80 s, Supercomputer Vector Machine
Vector Supercomputers Epitomized by Cray-1, 1976: Scalar Unit + Vector Extensions • • Load/Store Architecture Vector Registers Vector Instructions Hardwired Control Highly Pipelined Functional Units Interleaved Memory System No Data Caches No Virtual Memory Krste CS 252 Feb. 27, 2006 Lecture 12, Slide 6
Cray-1 (1976) Krste CS 252 Feb. 27, 2006 Lecture 12, Slide 7
Krste CS 252 Feb. 27, 2006 Lecture 12, Slide 8 Cray-1 (1976) 64 Element Vector Registers Single Port Memory 16 banks of 64 bit words + 8 -bit SECDED ( (Ah) + j k m ) (A 0) 80 MW/sec data load/store Tjk ( (Ah) + j k m ) (A 0) 320 MW/sec instruction buffer refill 64 T Regs Si 64 B Regs Ai Bjk S 0 S 1 S 2 S 3 S 4 S 5 S 6 S 7 A 0 A 1 A 2 A 3 A 4 A 5 A 6 A 7 NIP 64 -bitx 16 4 Instruction Buffers memory bank cycle 50 ns V 0 V 1 V 2 V 3 V 4 V 5 V 6 V 7 Vi V. Mask Vj V. Length Vk FP Add Sj FP Mul Sk FP Recip Si Int Add Int Logic Int Shift Pop Cnt Aj Ak Ai Addr Mul CIP LIP processor cycle 12. 5 ns (80 MHz)
Vector Programming Model Scalar Registers Vector Registers r 15 v 15 r 0 v 0 [0] [1] [2] [VLRMAX-1] Vector Length Register Vector Arithmetic Instructions ADDV v 3, v 1, v 2 VLR v 1 v 2 + + [0] [1] + + v 3 Vector Load and Store Instructions LV v 1, r 2 Base, r 1 Krste CS 252 Feb. 27, 2006 Lecture 12, Slide 9 Stride, r 2 v 1 [VLR-1] Vector Register Memory
Krste CS 252 Feb. 27, 2006 Lecture 12, Slide 10 Vector Instruction Set Advantages • Compact – one short instruction encodes N operations – really good for energy efficiency • Expressive, tells hardware that these N operations: – – – are independent use the same functional unit access disjoint registers access registers in the same pattern as previous instructions access a contiguous block of memory (unit-stride load/store) access memory in a known pattern (strided load/store) • Scalable – can run same object code on more or fewer parallel pipelines or lanes
Vector Arithmetic Execution • Original vector architectures were deeply pipelined • Deep pipeline (=> fast clock) to execute element operations • Simplifies control of deep pipeline because elements in vector are independent (=> guarantee of no data dependence hazards!) V 1 V 2 Krste CS 252 Feb. 27, 2006 Lecture 12, Slide 11 V 3 Six stage multiply pipeline V 3 <- v 1 * v 2
Krste CS 252 Feb. 27, 2006 Lecture 12, Slide 12 Vector Memory System Cray-1, 16 banks, 4 cycle bank busy time, 12 cycle latency • Bank busy time: Cycles between accesses to same bank Base Stride Vector Registers Address Generator 0 1 2 3 4 5 6 7 8 9 A B C D E F Memory Banks +
Vector Instruction Execution ADDV C, A, B Krste CS 252 Feb. 27, 2006 Lecture 12, Slide 13 ; add vector Cheap machine using one pipelined functional unit Expensive machine with four pipelined functional units A[6] B[6] A[24] B[24] A[25] B[25] A[26] B[26] A[27] B[27] A[5] A[4] A[3] B[5] B[4] B[3] A[20] B[20] A[21] B[21] A[22] B[22] A[23] B[23] A[16] B[16] A[17] B[17] A[18] B[18] A[19] B[19] A[12] B[12] A[13] B[13] A[14] B[14] A[15] B[15] C[2] C[8] C[9] C[10] C[11] C[4] C[5] C[6] C[7] C[0] C[1] C[2] C[3]
Krste CS 252 Feb. 27, 2006 Lecture 12, Slide 14 Vector Memory-Memory versus Vector Register Machines • Vector memory-memory instructions hold all vector operands in main memory (not vector registers) • The first vector machines, CDC Star-100 (‘ 73) and TI ASC (‘ 71), were memory-memory machines • Cray-1 (’ 76) was first vector register machine Vector Memory-Memory Code Example Source Code for (i=0; i<N; i++) { C[i] = A[i] + B[i]; D[i] = A[i] - B[i]; } ADDV C, A, B SUBV D, A, B Vector Register Code LV V 1, A LV V 2, B ADDV V 3, V 1, V 2 SV V 3, C SUBV V 4, V 1, V 2 SV V 4, D
Vector Memory-Memory vs. Vector Register Machines Krste CS 252 Feb. 27, 2006 Lecture 12, Slide 15 • Vector memory-memory architectures (VMMA) require greater main memory bandwidth, why? – All operands must be read in and out of memory • VMMAs make if difficult to overlap execution of multiple vector operations, why? – Must check dependencies on memory addresses • VMMAs incur greater startup latency – Scalar code was faster on CDC Star-100 for vectors < 100 elements – For Cray-1, vector/scalar breakeven point was around 2 elements ÞAll major vector machines since Cray-1 have had vector register architectures (we ignore vector memory-memory from now on)
Automatic Code Vectorization Krste CS 252 Feb. 27, 2006 Lecture 12, Slide 16 for (i=0; i < N; i++) C[i] = A[i] + B[i]; Vectorized Code Scalar Sequential Code load Iter. 1 add store load Iter. 2 add store load Time load Iter. 1 load add store Iter. 2 Vector Instruction Vectorization is a massive compile-time reordering of operation sequencing requires extensive loop dependence analysis
Automatic Code Vectorization Krste CS 252 Feb. 27, 2006 Lecture 12, Slide 17 • Automatic vectorization is really difficult because: • Pointer alias analysis • Really hard for compilers • Array index analysis • Compiler techniques for this are good • But hard to implement, so often poorly supported in real compilers • Vectorization often benefits from changes in the layout of data structures • Programmers are good at this • Compilers are not good at automatically changing data layouts
Vector Strip-mining Krste CS 252 Feb. 27, 2006 Lecture 12, Slide 18 Problem: Vector registers have finite length Solution: Break loops into pieces that fit into vector registers. This is known as “Stripmining”. ANDI R 1, N, 63 # N mod 64 MTC 1 VLR, R 1 # Do remainder for (i=0; i<N; i++) C[i] = A[i]+B[i]; loop: LV V 1, RA A B C DSLL R 2, R 1, 3 # Multiply by 8 Remainder DADDU RA, R 2 # Bump pointer + LV V 2, RB DADDU RB, R 2 64 elements ADDV. D V 3, V 1, V 2 + SV V 3, RC DADDU RC, R 2 DSUBU N, N, R 1 # Subtract elements + LI R 1, 64 MTC 1 VLR, R 1 # Reset full length BGTZ N, loop # Any more to do?
Vector Chaining Krste CS 252 Feb. 27, 2006 Lecture 12, Slide 19 • Vector version of register bypassing – introduced with Cray-1 LV v 1 V 2 V 1 V 3 V 4 MULV v 3, v 1, v 2 ADDV v 5, v 3, v 4 Chain Load Unit Memory Chain Mult. Add V 5
Vector Chaining Advantage Krste CS 252 Feb. 27, 2006 Lecture 12, Slide 20 • Without chaining, must wait for last element of result to be written before starting dependent instruction Load Mul Time Add • With chaining, can start dependent instruction as soon as first result appears Load Mul Add
Vector Instruction Parallelism Krste CS 252 Feb. 27, 2006 Lecture 12, Slide 21 Can overlap execution of multiple vector instructions – example machine has 32 elements per vector register and 8 lanes Load Unit load Multiply Unit Add Unit mul add time load mul add Instruction issue Complete 24 operations/cycle while issuing 1 short instruction/cycle
Krste CS 252 Feb. 27, 2006 Lecture 12, Slide 22 Vector Startup Two components of vector startup penalty – functional unit latency (time through pipeline) – dead time or recovery time (time before another vector instruction can start down pipeline) Functional Unit Latency R X X X W R X X X W R X X X W R X X X First Vector Instruction Dead Time Second Vector Instruction W
Dead Time and Short Vectors Krste CS 252 Feb. 27, 2006 Lecture 12, Slide 23 No dead time 4 cycles dead time 64 cycles active Cray C 90, Two lanes 4 cycle dead time Maximum efficiency 94% with 128 element vectors T 0, Eight lanes No dead time 100% efficiency with 8 element vectors
Vector Scatter/Gather Krste CS 252 Feb. 27, 2006 Lecture 12, Slide 24 Want to vectorize loops with indirect accesses: for (i=0; i<N; i++) A[i] = B[i] + C[D[i]] Indexed load instruction (Gather) LV v. D, r. D # Load indices in D vector LVI v. C, r. C, v. D # Load indirect from r. C base LV v. B, r. B # Load B vector ADDV. D v. A, v. B, v. C # Do add SV v. A, r. A # Store result
Vector Scatter/Gather Scatter example: for (i=0; i<N; i++) A[B[i]]++; Krste CS 252 Feb. 27, 2006 Lecture 12, Slide 25
Vector Conditional Execution Krste CS 252 Feb. 27, 2006 Lecture 12, Slide 26 Problem: Want to vectorize loops with conditional code: for (i=0; i<N; i++) if (A[i]>0) then A[i] = B[i]; Solution: Add vector mask (or flag) registers – vector version of predicate registers, 1 bit per element …and maskable vector instructions – vector operation becomes NOP at elements where mask bit is clear Code example: CVM LV v. A, r. A SGTVS. D v. A, F 0 LV v. A, r. B SV v. A, r. A # # # Turn on all elements Load entire A vector Set bits in mask register where A>0 Load B vector into A under mask Store A back to memory under mask
Masked Vector Instructions Simple Implementation – execute all N operations, turn off result writeback according to mask Density-Time Implementation – scan mask vector and only execute elements with non-zero masks M[7]=1 A[7] B[7] M[7]=1 M[6]=0 M[5]=1 M[4]=1 M[3]=0 B[6] B[5] B[4] B[3] M[6]=0 M[5]=1 A[6] A[5] A[4] A[3] M[2]=0 C[2] M[1]=1 C[1] Krste CS 252 Feb. 27, 2006 Lecture 12, Slide 27 M[4]=1 M[3]=0 M[2]=0 M[1]=1 M[0]=0 A[7] B[7] C[5] C[4] C[1] Write data port M[0]=0 Write Enable C[0] Write data port
Vector Reductions Krste CS 252 Feb. 27, 2006 Lecture 12, Slide 28 Problem: Loop-carried dependence on reduction variables sum = 0; for (i=0; i<N; i++) sum += A[i]; # Loop-carried dependence on sum Solution: Re-associate operations if possible, use binary tree to perform reduction # Rearrange as: sum[0: VL-1] = 0 # for(i=0; i<N; i+=VL) # sum[0: VL-1] += A[i: i+VL-1]; # # Now have VL partial sums in one do { VL = VL/2; sum[0: VL-1] += sum[VL: 2*VL-1] } while (VL>1) Vector of VL partial sums Stripmine VL-sized chunks Vector sum vector register # Halve vector length # Halve no. of partials
- Slides: 28