Review Professor Alvin R Lebeck Compsci 220 ECE

Review Professor Alvin R. Lebeck Compsci 220 / ECE 252 Fall 2008

Amdahl’s Law Ex. Timenew = Ex. Timeold x (1 - Fractionenhanced) + Fractionenhanced Speedupoverall = Ex. Timeold Ex. Timenew 1 = (1 - Fractionenhanced) + Fractionenhanced Comp. Sci 220 / ECE 252 Speedupenhanced 2

Review: Performance CPU time = Seconds = Instructions x Cycles Program Instruction Program x Seconds Cycle “Average Cycles Per Instruction” “Instruction Frequency” Invest Resources where time is Spent! Comp. Sci 220 / ECE 252 3

Little’s Law • Key Relationship between latency and bandwidth: • Average number in system = arrival rate * mean holding time • Example: – – – How big a wine cellar should we build? We drink (and buy) an average of 4 bottles per week On average, I want to age my wine 5 years bottles in cellar = 4 bottles/week * 52 weeks/year * 5 years = 1040 bottles Comp. Sci 220 / ECE 252 4

Review: The Five Stages of a Load Cycle 1 Cycle 2 Load Ifetch Reg/Dec Cycle 3 Cycle 4 Cycle 5 Exec Mem Wr. B • Ifetch: Instruction Fetch – Fetch the instruction from the Instruction Memory • • Reg/Dec: Registers Fetch and Instruction Decode Exec: Calculate the memory address Mem: Read the data from the Data Memory Wr. B: Write the data back to the register file Comp. Sci 220 / ECE 252 5

Its Not That Easy for Computers • What could go wrong? • Limits to pipelining: Hazards prevent next instruction from executing during its designated clock cycle – Structural hazards: HW cannot support this combination of instructions – Data hazards: Instruction depends on result of prior instruction still in the pipeline » RAW » WAR – Control hazards: Pipelining of branches & other instructions Comp. Sci 220 / ECE 252 6

Control Hazard Cycle 4 Cycle 5 Cycle 6 Cycle 7 Cycle 8 Cycle 9 Cycle 10 Cycle 11 Clk 12: Beq Ifetch Reg/Dec Exec (target is 1000) 16: R-type Ifetch Reg/Dec 20: R-type Ifetch 24: R-type Mem Wr Exec Mem Wr Reg/Dec Exec Mem Wr Ifetch Reg/Dec Exec Mem 1000: Target of Br Wr • Although Beq is fetched during Cycle 4: – Target address is NOT written into the PC until the end of Cycle 7 – Branch’s target is NOT fetched until Cycle 8 – 3 -instruction delay before the branch take effect • This is called a Control Hazard: Comp. Sci 220 / ECE 252 7

Dynamic Branch Prediction • Solution: 2 -bit counter where prediction changes only if mispredict twice: • Increment for taken, decrement for not-taken – 00, 01, 10, 11 • Helps when target is known before condition T Predict Taken T T Predict Not Taken NT NT T Predict Taken NT Predict Not Taken NT Comp. Sci 220 / ECE 252 8

Need Address @ Same Time as Prediction • Branch Target Buffer (BTB): Address of branch index to get prediction AND branch address (if taken) – Note: must check for branch match now, since can’t use wrong branch address (Figure 4. 22, p. 273) PC of Inst to fetch Predicted PC Branch Prediction: Taken or not Taken 0 … Branch folding? n-1 = Yes, use predicted PC No, not branch Procedure Return Addresses Predicted with a Stack Comp. Sci 220 / ECE 252 9

Hybrid/Competitive/Selective Branch Predictor • Different predictors work better for different branches • Pick the predictor that works best for a given branch Comp. Sci 220 / ECE 252 10

Unrolled Loop That Minimizes Stalls 1 Loop: 2 3 4 5 6 7 8 9 10 11 12 13 14 LD LD ADDD SD SD SD SUBI BNEZ SD F 0, 0(R 1) F 6, -8(R 1) F 10, -16(R 1) F 14, -24(R 1) F 4, F 0, F 2 F 8, F 6, F 2 F 12, F 10, F 2 F 16, F 14, F 2 0(R 1), F 4 -8(R 1), F 8 -16(R 1), F 12 R 1, #32 R 1, LOOP 8(R 1), F 16 • What assumptions made when moved code? – OK to move store past SUBI even though changes register – OK to move loads before stores: get right data? – When is it safe for compiler to do such changes? ; 8 -32 = -24 14 clock cycles, or 3. 5 per iteration Comp. Sci 220 / ECE 252 11

SW Pipelining Example Before: Unrolled 3 times 1 2 3 4 5 6 7 8 9 10 11 LD ADDD SD SUBI BNEZ After: Software Pipelined F 0, 0(R 1) F 4, F 0, F 2 0(R 1), F 4 F 6, -8(R 1) F 8, F 6, F 2 -8(R 1), F 8 F 10, -16(R 1) F 12, F 10, F 2 -16(R 1), F 12 R 1, #24 R 1, LOOP SD IF ADDD LD 1 2 3 4 5 Read F 4 ID IF EX ID IF LD ADDD LD SUBI BNEZ SD ADDD SD Read F 0 Mem EX ID F 0, 0(R 1) F 4, F 0, F 2 F 0, -8(R 1) 0(R 1), F 4; Stores M[i] F 4, F 0, F 2; Adds to M[i-1] F 0, -16(R 1); loads M[i-2] R 1, #8 R 1, LOOP 0(R 1), F 4, F 0, F 2 -8(R 1), F 4 WB Write F 4 Mem WB EX Mem WB Write F 0 Comp. Sci 220 / ECE 252 12

Tomasulo Organization From Memory Load Buffers From Instruction Unit FP Registers FP op queue Operand Bus Store Buffers To Memory FP adders FP multipliers Common Data Bus (CDB) Comp. Sci 220 / ECE 252 13

Tomasulo Summary • • Prevents Register as bottleneck Avoids WAR, WAW hazards of Scoreboard Allows loop unrolling in HW Not limited to basic blocks (provided branch prediction) • Lasting Contributions – Dynamic scheduling – Register renaming – Load/store disambiguation Comp. Sci 220 / ECE 252 14

Speculation (getting more ILP) • Speculation: allow an instruction to issue that is dependent on branch predicted to be taken without any consequences (including exceptions) if branch is not actually taken (“HW undo” squash) • Often combine with dynamic scheduling • Separate speculative bypassing of results from real bypassing of results – When instruction no longer speculative, write results (instruction commit) – execute out-of-order but commit in order • Memory operations (memory disambiguation) • Interrupts -> maintaining precise exceptions Comp. Sci 220 / ECE 252 15

HW support for More ILP • Need HW buffer for results of uncommitted instructions: reorder buffer – Reorder buffer can be operand source FP – Once operand commits, result is found in register Op – 3 fields: instr. type, destination, value Queue – Use reorder buffer number instead of reservation station – Instructions commit in order Res Stations – As a result, its easy to undo speculated instructions on FP Adder mispredicted branches or on exceptions Comp. Sci 220 / ECE 252 Reorder Buffer FP Regs Res Stations FP Adder 16

Recovering from Incorrect Speculation • Reorder Buffer – Register Update Unit: Reorder buffer+reservation stations combined) – P 6 Style: Reorder buffer separate from reservation stations • R 10 K style – Separate physical register file from reorder buffer – Must maintain a map of logical to physical registers • Enables easy recovery from misprediction & exceptions • Memory Disambiguation – Load/store queue (Memory Order Buffer) Comp. Sci 220 / ECE 252 17

Superscalar & VLIW • • Wider pipelines Superscalar, mulitple PCs VLIW, multiple operations for each PC Problems w/ Superscalar – – Wide fetch Dependence check Bypassing Need large window to find independent ops Comp. Sci 220 / ECE 252 18

Trace Scheduling Reorder these instructions to improve ILP Comp. Sci 220 / ECE 252 Fix-up instructions In case we were wrong 19

Trace Cache • Store traces • Enables fetch past next branch • Enables branch folding Comp. Sci 220 / ECE 252 20

Predicated/Conditional Execution (more ILP) • Avoid branch prediction by turning branches into conditionally executed instructions: if (x) then A = B op C else NOP – If false, then neither store result nor cause exception – Expanded ISA of Alpha, MIPS, Power. PC, SPARC have conditional move; PA-RISC can annul any following instr, IA-64 predicated execution. • Drawbacks to conditional instructions – Still takes a clock even if “annulled” – Stall if condition evaluated late – Complex conditions reduce effectiveness; condition becomes known late in pipeline Comp. Sci 220 / ECE 252 21

Review: ABCs of caches • • • Associativity Block size Capacity Number of sets S = C/(BA) 1 -way (Direct-mapped) – A = 1, S = C/B • N-way set-associative • Fully associativity – S = 1, C = BA • Know how a specific piece of data is found – Index, tag, block offset 22

Cache Performance CPU time = (CPU execution clock cycles + Memory stall clock cycles) x clock cycle time Memory stall clock cycles = (Reads x Read miss rate x Read miss penalty + Writes x Write miss rate x Write miss penalty) Memory stall clock cycles = Memory accesses x Miss rate x Miss penalty CPS 220 23

Improving Cache Performance Ave Mem Acc Time = Hit time + (miss rate x miss penalty) 1. Reduce the miss rate, 2. Reduce the miss penalty, or 3. Reduce the time to hit in the cache. • NUCA • Indirect Index Cache • Continual Flow Pipelines – Checkpoint recovery – Nonblocking queues to tolerate long latency operations CPS 220 24

Virtual Memory • • • Paged VM Segmented VM TLBs (where is it) Page coloring Main Memory – Interleaving – Banked Comp. Sci 220 / ECE 252 25

Multithreading & Multiprocessing • • • Multithreaded vs. multicore Simultaneous multithreaded Shared Memory vs. Message Passing SIMD vs. MIMD Snooping Coherence Protocol Directory Coherence Protocol Locks & Synchronization Memory Consistency Papers – – Power as Design Target Choice in SMT Multiscalar Niagara Comp. Sci 220 / ECE 252 26

Interconnection Networks • • Topologies Routing Deadlock Livelock Comp. Sci 220 / ECE 252 27

Papers • Reliability – Diva – Argus • Security – Rifle • Virtual Machines • Grid Processors Comp. Sci 220 / ECE 252 28

Projects • Project Presentations in D 344 LSRC • 10 am to noon Tuesday Dec 2 (D 344 LSRC) • • • Zachary Drillings, David Eitel, Alex Edelsburg, "Detouring 2: Toward Industry" Jun Pang, Meng Zhang, "FPGA Implementation and Improvement of Core Cannabalization Architecture" Risi Thonangi, Amre Shakimov, "Speeding up B-Tree Accesses using Flash-memory" Abhinav Kapur, Alex Hunter, Paul Huang, "Architectural Support for Computational Biology" Archana Ramamoorthy, Yulin Zhang, "Dynamic Branch Predication" Xuhan Peng, Mengyuan Huang, "Improving Correlated Branch Predictors“ Thursday Dec 4 (D 344 LSRC) • • George Rossin, Ben Shelton, Philp Eithier, "Adaptive Cache Replacement Algorithms" Laura Angle, Andrew First, Preeyanka Shah, "Argus Testing and Analysis" Mustafa Lokhandwal, Dongtao Liu, "Evaluating Parallel Programming Models" Pablo Gainza, Ryan Scudellari, "Evaluation of Prefix-Sum Parallel Application using CUDA, Pthreads and message passing" Keven Brown, Aleks Klimas, "Dynamically Allocated Functional Units (DAFU)" Kai Wang, Yang Jiang, Xiaoyan Yin, "Store Queue Optimization" Amrita Halappanavar, J. P. Carafo, Abhinav Mohan, "Speculative Issue Logic" Comp. Sci 220 / ECE 252 29