ECE 232 Hardware Organization and Design Lecture 18

ECE 232 Hardware Organization and Design Lecture 18 Pipelining Maciej Ciesielski www. ecs. umass. edu/ece/labs/vlsicad/ece 232/spr 2002/index_232. html ECE 232 L 18. Pipeline. 1 Adapted from Patterson 97 ©UCB Copyright 1998 Morgan Kaufmann Publishers

Outline ° Review • Microprogramming ° Pipelining, basic concept ° Hazards • Structural • Data hazards • Control hazards ECE 232 L 18. Pipeline. 2 Adapted from Patterson 97 ©UCB Copyright 1998 Morgan Kaufmann Publishers

Pipelining is Natural! Laundry Example A B C D ° Ann, Brian, Cathy, Dave each have one load of clothes to wash, dry, and fold ° Washer takes 30 minutes ° Dryer takes 40 minutes ° “Folder” takes 20 minutes ECE 232 L 18. Pipeline. 3 Adapted from Patterson 97 ©UCB Copyright 1998 Morgan Kaufmann Publishers

Sequential Laundry 6 PM 7 8 9 10 11 Midnight Time 30 40 20 T a s k A B O r d e r C D ° Sequential laundry takes 6 hours for 4 loads ° If they learned pipelining, how long would laundry take? ECE 232 L 18. Pipeline. 4 Adapted from Patterson 97 ©UCB Copyright 1998 Morgan Kaufmann Publishers

Pipelined Laundry: Start work ASAP 6 PM 7 8 9 10 11 Midnight Time 30 40 T a s k O r d e r 40 40 40 20 A B C D ° Pipelined laundry takes 3. 5 hours for 4 loads ECE 232 L 18. Pipeline. 5 Adapted from Patterson 97 ©UCB Copyright 1998 Morgan Kaufmann Publishers

Pipelining Lessons 6 PM 7 8 9 Time 30 40 T a s k O r d e r 40 40 40 20 ° Pipelining doesn’t help latency of single task, it helps throughput of entire workload ° Pipeline rate limited by slowest pipeline stage A ° Multiple tasks operating simultaneously using different resources B ° Potential speedup = Number pipe stages C ° Unbalanced lengths of pipe stages reduces speedup D ° Time to “fill” pipeline and time to “drain” it reduces speedup ° Stall for Dependences ECE 232 L 18. Pipeline. 6 Adapted from Patterson 97 ©UCB Copyright 1998 Morgan Kaufmann Publishers

The Five Stages of Load Cycle 1 Cycle 2 Cycle 3 Cycle 4 Cycle 5 Load Ifetch Reg/Dec Exec Mem Wr ° IFetch: Instruction Fetch • Fetch the instruction from the Instruction Memory ° Reg/Dec: Registers Fetch and Instruction Decode ° Exec: Calculate the memory address ° Mem: Read the data from the Data Memory ° Wr: Write the data back to the register file ECE 232 L 18. Pipeline. 7 Adapted from Patterson 97 ©UCB Copyright 1998 Morgan Kaufmann Publishers

Pipelined Execution Time IFetch Dcd Exec IFetch Dcd Mem WB Exec Mem WB Exec Mem IFetch Dcd Program Flow IFetch Dcd WB ° Utilization? ° Now we just have to make it work ECE 232 L 18. Pipeline. 8 Adapted from Patterson 97 ©UCB Copyright 1998 Morgan Kaufmann Publishers

Single Cycle, Multiple Cycle, vs. Pipeline Cycle 1 Cycle 2 Clk Single Cycle Implementation: Load Store Waste Cycle 1 Cycle 2 Cycle 3 Cycle 4 Cycle 5 Cycle 6 Cycle 7 Cycle 8 Cycle 9 Cycle 10 Clk Multiple Cycle Implementation: Load Ifetch Reg Exec Mem Wr Store Ifetch Reg Exec Mem R-type Ifetch Pipeline Implementation: Load Ifetch Reg Store Ifetch Exec Mem Wr Reg Exec Mem R-type Ifetch ECE 232 L 18. Pipeline. 9 Reg Exec Adapted from Patterson 97 ©UCB Wr Mem Wr Copyright 1998 Morgan Kaufmann Publishers

Why Pipeline? ° Suppose we execute 100 instructions ° Single Cycle Machine • 45 ns/cycle x 1 CPI x 100 inst = 4500 ns ° Multicycle Machine • 10 ns/cycle x 4. 6 CPI (due to inst mix) x 100 inst = 4600 ns ° Ideal pipelined machine • 10 ns/cycle x (1 CPI x 100 inst + 4 cycle drain) = 1040 ns ECE 232 L 18. Pipeline. 10 Adapted from Patterson 97 ©UCB Copyright 1998 Morgan Kaufmann Publishers

Why Pipeline? Because the resources are there! Time (clock cycles) Inst 3 Im Reg Im Dm Dm Reg Dm Im Reg Adapted from Patterson 97 ©UCB Reg Dm ALU Inst 4 ECE 232 L 18. Pipeline. 11 Reg ALU Inst 2 Reg ALU Inst 1 Im ALU O r d e r Inst 0 ALU I n s t r. Reg Dm Reg Copyright 1998 Morgan Kaufmann Publishers

Can pipelining get us into trouble? ° Yes: Pipeline Hazards • structural hazards: attempt to use the same resource two different ways at the same time E. g. , combined washer/dryer would be a structural hazard or folder busy doing something else (watching TV) • data hazards: attempt to use item before it is ready E. g. , one sock of pair in dryer and one in washer; can’t fold until get sock from washer through dryer instruction depends on result of prior instruction still in the pipeline • control hazards: attempt to make a decision before condition is evaulated E. g. , washing football uniforms and need to get proper detergent level; need to see after dryer before next load in branch instructions ° Can always resolve hazards by waiting • pipeline control must detect the hazard • take action (or delay action) to resolve hazards ECE 232 L 18. Pipeline. 12 Adapted from Patterson 97 ©UCB Copyright 1998 Morgan Kaufmann Publishers

Single Memory is a Structural Hazard Time (clock cycles) Reg Mem Reg Mem Reg ALU Instr 3 Reg ALU Instr 2 Mem ALU Instr 1 Reg ALU O r d e r Load Mem ALU I n s t r. Mem Instr 4 Reg Detection is easy in this case! (right half highlight means read, left half write) ECE 232 L 18. Pipeline. 13 Adapted from Patterson 97 ©UCB Copyright 1998 Morgan Kaufmann Publishers

Control Hazard Solutions ° Stall: wait until decision is clear • It is possible to move up decision to 2 nd stage by adding hardware to check registers as being read Beq Load Reg Mem Reg ALU Add Mem ALU O r d e r Time (clock cycles) ALU I n s t r. Mem Reg ° Impact: 2 clock cycles per branch instruction => slow ECE 232 L 18. Pipeline. 14 Adapted from Patterson 97 ©UCB Copyright 1998 Morgan Kaufmann Publishers

Control Hazard Solutions ° Predict: guess one direction then back up if wrong • Predict not taken Beq Load Reg Mem Reg ALU Add Mem ALU O r d e r Time (clock cycles) ALU I n s t r. Mem Reg ° Impact: 1 clock cycles per branch instruction if right, 2 if wrong (right 50% of time) ° More dynamic scheme: history of 1 branch ( 90%) ECE 232 L 18. Pipeline. 15 Adapted from Patterson 97 ©UCB Copyright 1998 Morgan Kaufmann Publishers

Control Hazard Solutions ° Redefine branch behavior (takes place after next instruction) “delayed branch” Beq Misc Mem Reg Mem Reg ALU Load Reg ALU Add Mem ALU O r d e r Time (clock cycles) ALU I n s t r. Reg Mem Reg ° Impact: 0 clock cycles per branch instruction if can find instruction to put in “slot” ( 50% of time) ° As launch more instruction per clock cycle, less useful ECE 232 L 18. Pipeline. 16 Adapted from Patterson 97 ©UCB Copyright 1998 Morgan Kaufmann Publishers

Data Hazard on r 1: • Dependencies backwards in time are hazards Time (clock cycles) Reg Dm Im Reg ALU and r 6, r 1, r 7 Im ALU O r d e r sub r 4, r 1, r 3 ID/RF ALU I n s t r. add r 1, r 2, r 3 IF EX or r 8, r 1, r 9 xor r 10, r 11 ECE 232 L 18. Pipeline. 17 Adapted from Patterson 97 ©UCB MEM WB Reg Reg Dm Copyright 1998 Morgan Kaufmann Publishers Reg

Data Hazard Solution: • “Forward” result from one stage to another Time (clock cycles) Reg Dm Im Reg ALU and r 6, r 1, r 7 Im ALU O r d e r sub r 4, r 1, r 3 ID/RF ALU I n s t r. add r 1, r 2, r 3 IF EX or r 8, r 1, r 9 xor r 10, r 11 MEM WB Reg Reg Dm • “or” OK if define read/write properly ECE 232 L 18. Pipeline. 18 Adapted from Patterson 97 ©UCB Copyright 1998 Morgan Kaufmann Publishers Reg

Forwarding (or Bypassing): What about Loads • Dependencies backwards in time are hazards Time (clock cycles) Im Reg Dm Im Reg ALU sub r 4, r 1, r 3 ID/RF ALU lw r 1, 0(r 2) IF EX MEM WB Reg Dm Reg • Can’t solve with forwarding: • Must delay/stall instruction dependent on loads ECE 232 L 18. Pipeline. 19 Adapted from Patterson 97 ©UCB Copyright 1998 Morgan Kaufmann Publishers

Designing a Pipelined Processor ° Go back and examine your datapath and control diagrams ° Associated resources with states ° Ensure that flows do not conflict, or figure out how to resolve ° Assert control in appropriate stage ECE 232 L 18. Pipeline. 20 Adapted from Patterson 97 ©UCB Copyright 1998 Morgan Kaufmann Publishers

Summary ° Microprogramming is a fundamental concept • implement an instruction set by building a very simple processor and interpreting the instructions • essential for very complex instructions and when few register transfers are possible ° Pipelining is a fundamental concept • multiple steps using distinct resources ° Utilize capabilities of the Datapath by pipelined instruction processing • start next instruction while working on the current one • limited by length of longest stage (plus fill/flush) • detect and resolve hazards ECE 232 L 18. Pipeline. 21 Adapted from Patterson 97 ©UCB Copyright 1998 Morgan Kaufmann Publishers