Pipelining CS 365 Lecture 9 Pipeline CS 365

Pipelining CS 365 Lecture 9 Pipeline CS 365 Spring 2008 GMU

Outline Today’s topic Pipelining is an implementation technique in which multiple instructions are overlapped in execution Subset of MIPS instructions lw, sw, and, or, add, sub, slt, beq Outline Pipeline high level introduction Stages, hazards Pipelined Pipeline datapath and control design CS 465 2 D. Barbara

Pipelining is Natural! Laundry example Ann, A B C D Brian, Cathy, Dave each has one load of clothes to wash, dry, and fold Washer takes 30 minutes Dryer takes 40 minutes “Folder” Pipeline takes 20 minutes CS 465 3 D. Barbara

Sequential Laundry 6 PM 7 8 9 10 11 Midnight Time 30 40 20 T a s k A B O r d e r C D Sequential laundry takes 6 hours for 4 loads If they learned pipelining, how long would laundry take? Pipeline CS 465 4 D. Barbara

Pipelined Laundry 6 PM 7 8 9 10 11 Midnight Time 30 40 T a s k 40 40 40 20 A B O r d e r C D Start work ASAP Pipelined laundry takes 3. 5 hours for 4 loads Pipeline CS 465 5 D. Barbara

Pipelining Lessons (I) 6 PM 7 8 9 Time 30 40 T a s k O r d e r 40 40 40 20 A B Multiple tasks operating simultaneously using different resources Pipelining doesn’t help latency of single task, it helps throughput of entire workload Pipeline rate is limited by slowest pipeline stage C D Pipeline Unbalanced lengths of pipeline stages reduces speedup CS 465 6 D. Barbara

Pipelining Lessons (II) 6 PM 7 8 9 Time 30 40 T a s k O r d e r 40 40 40 20 A B Potential speedup = Number pipeline stages Time to “fill” pipeline and time to “drain” it reduces speedup startup and wind down Stall for dependencies C D Pipeline CS 465 7 D. Barbara

Five Stages of Workload Cycle 1 Cycle 2 Load Ifetch Exec Mem Wr Ifetch: Instruction Fetch Reg/Dec Cycle 3 Cycle 4 Cycle 5 Fetch the instruction from the Instruction Memory Reg/Dec: Registers Fetch and Instruction Decode Exec: Calculate the memory address Mem: Read the data from the Data Memory Wr: Write the data back to the register file Pipeline CS 465 8 D. Barbara

Single Cycle, Multi Cycle, Pipeline Cycle 1 Cycle 2 Clk Single Cycle Implementation: Load Waste Store Cycle 1 Cycle 2 Cycle 3 Cycle 4 Cycle 5 Cycle 6 Cycle 7 Cycle 8 Cycle 9 Cycle 10 Clk Multiple Cycle Implementation: Load Ifetch Reg Exec Mem Wr Store Ifetch Reg Exec Mem R-type Ifetch Pipeline Implementation: Load Ifetch Reg Store Ifetch Exec Mem Wr Reg Exec Mem R-type Ifetch Pipeline Reg Exec Wr Mem CS 465 Wr 9 D. Barbara

Why Pipeline? (Performance) Suppose we execute 100 instructions Single cycle machine 45 (ns/cycle) x 1 (CPI) x 100 (inst) = 4500 ns Multicycle machine 10 (ns/cycle) x 4. 4 (CPI) (due to inst mix) x 100 (inst) = 4400 ns Ideal pipelined machine 10 (ns/cycle) x (1 (CPI) x 100 (inst) + 4 cycle drain) = 1040 ns Pipeline CS 465 10 D. Barbara

Pipelining Throughput Ideal speedup is no. of stages in the pipeline; in practice: Pipeline stage time are limited by the slowest resource, either the ALU or memory access Fill and drain time Pipeline CS 465 11 D. Barbara

Why Pipeline? (Resource) Time (clock cycles) Inst 3 Pipeline Reg Im Reg Dm Im Reg CS 465 Reg Dm ALU Inst 4 Im Dm ALU Inst 2 Reg ALU Inst 1 Im ALU O r d e r Inst 0 ALU I n s t r. Reg Dm Reg 12 D. Barbara

Pipeline Hazards prevent next instruction from executing during its designated clock cycle Structural hazards: attempt to use the same resource two different ways at the same time Data hazards: attempt to use data before it is ready E. g. , combined washer/dryer would be a structural hazard or folder busy doing something else (watching TV) One memory port E. g. , one sock of pair in dryer and one in washer; can’t fold until you get sock from washer through dryer Instruction depends on result of prior instruction still in the pipeline Control hazards: attempt to make a decision before condition is evaluated Pipeline Branch instructions CS 465 13 D. Barbara

Structural Hazard: One Memory Time (clock cycles) Instr 4 Reg Mem Reg Mem Reg ALU Instr 3 Mem ALU Instr 2 Reg ALU Instr 1 Mem ALU O r d e r Load ALU I n s t r. Mem • Solution 1: add more HW • Hazards can always be resolved by waiting Pipeline CS 465 Reg 14 D. Barbara

Structural Hazard: One Memory Time (clock cycles) Instr 2 stall Reg Mem Reg ALU Instr 1 Mem ALU Mem Reg Bubble Bubble Instr 3 Mem Reg ALU O r d e r Load ALU I n s t r. Mem Reg • Hazards can always be resolved by waiting Pipeline CS 465 15 D. Barbara

Data Hazard Example Data hazard: an instruction depends on the result of a previous instruction still in the pipeline add r 1 , r 2, r 3 sub r 4, r 1 , r 3 and r 6, r 1 , r 7 or r 8, r 1 , r 9 xor r 10, r 11 Pipeline CS 465 16 D. Barbara

Data Hazard Example Dependences backward in time are hazards Time (clock cycles) IF WB Reg Dm Im Reg ALU and r 6, r 1, r 7 MEM ALU O r d e r sub r 4, r 1, r 3 Im EX ALU I n s t r. add r 1, r 2, r 3 ID/RF or r 8, r 1, r 9 xor r 10, r 11 Reg Reg Dm Reg Compilers can help, but it gets messy and difficult Pipeline CS 465 17 D. Barbara

Data Hazard Solution Time (clock cycles) IF WB Reg Dm Im Reg ALU and r 6, r 1, r 7 MEM ALU O r d e r sub r 4, r 1, r 3 Im EX ALU I n s t r. add r 1, r 2, r 3 ID/RF or r 8, r 1, r 9 xor r 10, r 11 Reg Reg Dm Reg Solution : “forward” result from one stage to another Pipeline CS 465 18 D. Barbara

Data Hazard Even with Forwarding Time (clock cycles) IF MEM Reg Dm Im Reg ALU sub r 4, r 1, r 3 Im EX ALU lw r 1, 0(r 2) ID/RF WB Reg Dm Reg Can’t go back in time! Must delay/stall instruction dependent on loads Pipeline CS 465 19 D. Barbara

Data Hazard Even with Forwarding Time (clock cycles) IF Reg Stall MEM WB Dm Reg Im Reg ALU sub r 4, r 1, r 3 Im EX ALU lw r 1, 0(r 2) ID/RF Dm Reg Must delay/stall instruction dependent on loads Sometimes the instruction sequence can be reordered to avoid pipeline stalls Pipeline CS 465 20 D. Barbara

Control Hazards Branch instructions may change execution flow Suppose we can do decoding/branch decision/branch target computation at stage 2 Pipeline Still introduce 1 cycle stall Implementation details later CS 465 21 D. Barbara

Control Hazard Solution: Predict: guess one direction then back up if wrong Impact: 0 lost cycles per branch instruction if right, 1 if wrong Need to “Squash” and restart following instruction if wrong Prediction scheme Random prediction: correct 50% of time History based prediction: correct 90% of time Pipeline CS 465 22 D. Barbara

Control Hazard Solution: Predict Pipeline CS 465 23 D. Barbara

Pipeline Overview Summary Pipelining is a fundamental concept Multiple steps using distinct resources Utilize capabilities of the datapath by pipelined instruction processing Start next instruction while working on the current one Detect and resolve hazards Structural hazards, data hazards, control hazards All hazards can be solved by stall Other approaches: forwarding, prediction, reordering In modern processors, what really makes it hard: Exception handling Out of order execution Next: datapath design for pipeling Pipeline CS 465 24 D. Barbara

Single Cycle Datapath Pipeline CS 465 25 D. Barbara

Multi Cycle Datapath Divide the work into stages; internal registers Pipeline CS 465 26 D. Barbara

Single Cycle Pipeline Datagram What do we need to add to split the datapath into stages? Pipeline CS 465 27 D. Barbara

Pipelined Datapath 64 128 97 How many bits stored in each pipeline register? Pipeline CS 465 64 28 D. Barbara

Observations 5 stage pipeline IF, ID, EX, MEM, WB Left to right flow of instructions Instructions and data move generally from left to right Two exceptions: WB stage and the selection of PC May lead to data hazards and control hazards Why there is no pipeline register at the end of the WB stage? Last stage must update either register file, or memory, or PC Pipeline CS 465 29 D. Barbara

Pipelining the Load Instruction Cycle 1 Cycle 2 Cycle 3 Cycle 4 Cycle 5 Cycle 6 Cycle 7 Clock 1 st lw Ifetch Reg/Dec 2 nd lw Ifetch 3 rd lw Exec Mem Wr Reg/Dec Exec Mem Wr Ifetch Reg/Dec Exec Mem Wr The five independent functional units in the pipeline datapath are: Instruction Memory for the IF stage Register File’s Read Ports (bus. A and bus. B) for the ID stage ALU for the EXE stage Data Memory for the MEM stage Register File’s Write port (bus W) for the WB stage Pipeline CS 465 30 D. Barbara

The Four Stages of R type Cycle 1 Cycle 2 R-type Ifetch Reg/Dec Cycle 3 Cycle 4 Exec Wr IF: Instruction Fetch the instruction from the Instruction Memory ID: Registers Fetch and Instruction Decode EXE: ALU operates on the two register operands WB: Write the ALU output back to the register file Pipeline CS 465 31 D. Barbara

Pipelining R type and Load Instruction Cycle 1 Cycle 2 Cycle 3 Cycle 4 Cycle 5 Cycle 6 Cycle 7 Cycle 8 Cycle 9 Clock R-type Ifetch R-type Reg/Dec Exec Ifetch Reg/Dec Load Oops! We have a problem! Wr R-type Ifetch Wr Exec Mem Wr Reg/Dec Exec Wr R-type Ifetch Reg/Dec Exec Wr We have pipeline conflict or structural hazard: Two instructions try to write to the register file at the same time! Only one write port Pipeline CS 465 32 D. Barbara

Important Observation Each functional unit can only be used once per instruction Each functional unit must be used at the same stage for all instructions Delay R type’s register write by one cycle: Now R type instructions also use Reg File’s write port at Stage 5 Mem stage is a NO OP stage: nothing is being done 1 R-type Ifetch Pipeline 2 Reg/Dec 3 Exec 4 Mem 5 Wr Store Ifetch Reg/Dec Exec Mem Wr Beq Ifetch Reg/Dec Exec Mem Wr CS 465 33 D. Barbara

Pipelined Execution Cycle 1 Cycle 2 Cycle 3 Cycle 4 Cycle 5 Cycle 6 Cycle 7 Cycle 8 Cycle 9 Clock R-type Ifetch R-type Reg/Dec Exec Mem Wr Ifetch Reg/Dec Exec Mem Wr Reg/Dec Exec Mem Load R-type Ifetch Wr All instruction types have five pipeline stages Some stages may be wasted for some instructions Pipeline CS 465 34 D. Barbara

Pipelined Execution of Load Instruction Pipeline CS 465 35 D. Barbara

Pipelined Execution of Load Instruction Pipeline CS 465 36 D. Barbara

Pipelined Execution of Load Instruction Pipeline CS 465 37 D. Barbara

Pipelined Execution of Load Instruction Pipeline CS 465 38 D. Barbara

Pipelined Execution of Load Instruction Pipeline CS 465 39 D. Barbara

Pipelined Execution of Store Instruction Pipeline CS 465 40 D. Barbara

Pipelined Execution of Store Instruction Pipeline CS 465 41 D. Barbara

Observations from Load and Store Pass information needed from an earlier stage to a latter stage Each logical component of the datapath – such as IM, Reg read ports, ALU, DM, Reg write port – can be used only within a single pipeline stage. Otherwise, we would have structural hazard A bug in the pipelined datapath for load. Can you tell? Pipeline CS 465 42 D. Barbara

Modified Datapath • For basic R Type, LW/SW, and BEQ Pipeline CS 465 43 D. Barbara

Pipelined Execution for Multiple Instr. Pipeline CS 465 44 D. Barbara

Pipelined Execution for Multiple Instr. Pipeline CS 465 45 D. Barbara

Pipelined Execution for Multiple Instr. Pipeline CS 465 46 D. Barbara

Pipelined Execution for Multiple Instr. Pipeline CS 465 47 D. Barbara

Pipelined Execution for Multiple Instr. Pipeline CS 465 48 D. Barbara

Pipelined Execution for Multiple Instr. Pipeline CS 465 49 D. Barbara

Pipelined Datapath Control Fig. 6. 22 Pipeline CS 465 50 D. Barbara

Overview on Datapath Control op 00 0000 R-type 00 1101 10 0011 10 1011 00 0100 00 0010 ori lw sw beq jump Reg. Dst ALUSrc Memto. Reg. Write 1 0 0 1 0 1 1 1 x 0 x 0 x x x 0 Mem. Write Branch Jump Ext. Op ALUop 0 0 0 x “R-type” 0 0 Or 0 0 0 1 Add 1 0 0 1 Add 0 1 0 x 0 0 1 x xx Subtract For the subset of instructions under consideration ALUOp = 00 for Add, 01 for Sub, and 10 for R type Pipeline CS 465 51 D. Barbara

Observations No write control for all pipeline registers and PC since they are updated at every clock cycle To specify the control for the pipeline, set the control values during each pipeline stage Control lines can be divided into 5 groups: – – – NONE Reg. Dst, ALUOp, ALUSrc Branch, Mem. Read, Mem. Write Memto. Reg, Reg. Write Group these nine control lines into 3 subsets: IF ID ALU MEM WB ALUControl, MEMControl, WBControl signals are generated at ID stage, how to pass them to other stages? Pipeline CS 465 52 D. Barbara

Pass Control Signals Extend the pipeline registers to include control information Pipeline CS 465 53 D. Barbara

The Complete Pipelined Datapath Fig 6. 27 Pipeline CS 465 54 D. Barbara

Example Pipeline Execution Show the five instructions going through the pipeline: lw $10, 20($1) sub $11, $2, $3 and $12, $4, $5 or $13, $6, $7 add $14, $8, $9 Note that these instructions are independent from each other! Pipeline CS 465 55 D. Barbara

Clock 1 Pipeline CS 465 56 D. Barbara

Clock 2 Pipeline CS 465 57 D. Barbara

Clock 3 Pipeline CS 465 58 D. Barbara

Clock 4 Pipeline CS 465 59 D. Barbara

Clock 5 Pipeline CS 465 60 D. Barbara

Clock 6 Pipeline CS 465 61 D. Barbara

Clock 7 Pipeline CS 465 62 D. Barbara

Clock 8 Pipeline CS 465 63 D. Barbara

Clock 9 Pipeline CS 465 64 D. Barbara

Summary Overview of pipeline Stages Hazards Pipelined datapath Pipeline registers Pipelined execution Pipelined control Different signals for different stages Propagate control signals Pipeline CS 465 65 D. Barbara

Next Lecture Topic: Pipeline hazards and solutions Exception handling Reading Patterson Pipeline & Hennessy Ch 6. 4 6. 9 CS 465 66 D. Barbara