Csci 136 Computer Architecture II An Overview of

Csci 136 Computer Architecture II – An Overview of Pipelining Xiuzhen Cheng cheng@gwu. edu

Announcement Homework assignment #9, Due time – Before class, April 05. Readings: Sections 6. 1 – 6. 3 Problems: 6. 1 -6. 4, 6. 13 -6. 14 Project #3 is due on April 10, 2005

Today’s Topics: Pipelining by Analogy Pipelining is an implementation technique in which multiple instructions are overlapped in execution Subset of MIPS instructions: lw, sw, and, or, add, sub, slt, beq

Pipelining is Natural! Laundry Example Ann, Brian, Cathy, Dave each have one load of clothes to wash, dry, and fold Washer takes 30 minutes Dryer takes 40 minutes “Folder” takes 20 minutes A B C D

Sequential Laundry 6 PM 7 8 9 10 11 Midnight Time 30 40 20 T a s k O r d e r A B C D Sequential laundry takes 6 hours for 4 loads If they learned pipelining, how long would laundry take?

Pipelined Laundry: Start work ASAP 6 PM 7 8 9 10 11 Midnight Time 30 40 T a s k O r d e r 40 40 40 20 A B C D Pipelined laundry takes 3. 5 hours for 4 loads

Pipelining Lessons 6 PM 7 8 9 Time 30 40 T a s k O r d e r 40 40 40 20 Pipelining doesn’t help latency of single task, it helps throughput of entire workload Pipeline rate is limited by slowest pipeline stage A Multiple tasks operating simultaneously using different resources B Potential speedup = Number pipeline stages C Unbalanced lengths of pipeline stages reduces speedup D Time to “fill” pipeline and time to “drain” it reduces speedup Stall for Dependencies

The Five Stages of Load Cycle 1 Cycle 2 Load Ifetch Reg/Dec Cycle 3 Cycle 4 Cycle 5 Exec Mem Wr Ifetch: Instruction Fetch the instruction from the Instruction Memory Reg/Dec: Registers Fetch and Instruction Decode Exec: Calculate the memory address Mem: Read the data from the Data Memory Wr: Write the data back to the register file

Pipelining Improve performance by increasing throughput Ideal speedup is number of stages in the pipeline. Do we achieve this? NO! The computer pipeline stage time are limited by the slowest resource, either the ALU operation, or the memory access Fill and drain time

Single Cycle, Multiple Cycle, vs. Pipeline Cycle 1 Cycle 2 Clk Single Cycle Implementation: Load Store Waste Cycle 1 Cycle 2 Cycle 3 Cycle 4 Cycle 5 Cycle 6 Cycle 7 Cycle 8 Cycle 9 Cycle 10 Clk Multiple Cycle Implementation: Load Ifetch Reg Exec Mem Wr Store Ifetch Reg Pipeline Implementation: Load Ifetch Reg Store Ifetch Exec Mem Wr Reg Exec Mem R-type Ifetch Reg Exec Wr Mem Wr Exec Mem R-type Ifetch

Why Pipeline? Suppose we execute 100 instructions Single Cycle Machine 45 ns/cycle x 1 CPI x 100 inst = 4500 ns Multicycle Machine 10 ns/cycle x 4. 6 CPI (due to inst mix) x 100 inst = 4600 ns Ideal pipelined machine 10 ns/cycle x (1 CPI x 100 inst + 4 cycle drain) = 1040 ns

Why Pipeline? Because the resources are there! Time (clock cycles) Inst 3 Reg Im Reg Dm Im Reg Reg Dm ALU Inst 4 Im Dm ALU Inst 2 Reg ALU Inst 1 Im ALU O r d e r Inst 0 ALU I n s t r. Reg Dm Reg

Can pipelining get us into trouble? Yes: Pipeline Hazards Structural hazards: attempt to use the same resource two different ways at the same time E. g. , combined washer/dryer would be a structural hazard or folder busy doing something else (watching TV) Single memory cause structural hazards Data hazards: attempt to use item before it is ready E. g. , one sock of pair in dryer and one in washer; can’t fold until you get sock from washer through dryer instruction depends on result of prior instruction still in the pipeline Control hazards: attempt to make a decision before condition is evaluated E. g. , washing football uniforms and need to get proper detergent level; need to see after dryer before next load in branch instructions Can always resolve hazards by waiting pipeline control must detect the hazard take action (or delay action) to resolve hazards

Single Memory is a Structural Hazard Time (clock cycles) Instr 4 Reg Mem Reg Mem Reg ALU Instr 3 Mem ALU Instr 2 Reg ALU Instr 1 Mem ALU O r d e r Load ALU I n s t r. Mem Reg Detection is easy in this case! (right half highlight means read, left half write)

Structural Hazards limit performance Example: if 1. 3 memory accesses per instruction and only one memory access per cycle then average CPI 1. 3 otherwise resource is more than 100% utilized

Control Hazard Solution #1: Stall Add Beq Mem Reg Reg Mem Lost potential Mem Reg ALU Load Mem ALU O r d e r Time (clock cycles) ALU I n s t r. Mem Reg Stall: wait until decision is clear Impact: 2 lost cycles (i. e. 3 clock cycles per branch instruction) =>slow Move decision to end of decode by improving hardware save 1 cycle per branch If 20% instructions are BEQ, all others have CPI 1, what is the average CPI?

Control Hazard Solution #1: Stall

Control Hazard Solution #2: Predict Beq Load Reg Mem Reg ALU Add Mem ALU O r d e r Time (clock cycles) ALU I n s t r. Mem Reg Predict: guess one direction then back up if wrong Impact: 0 lost cycles per branch instruction if right, 1 if wrong (right 50% of time) Need to “Squash” and restart following instruction if wrong Produce CPI on branch of (1 *. 5 + 2 *. 5) = 1. 5 Total CPI might then be: 1. 5 *. 2 + 1 *. 8 = 1. 1 (20% branch) More dynamic scheme: history of each branch ( 90%)

Control Hazard Solution #2: Predict

Control Hazard Solution #3: Delayed Branch Misc Load Mem Reg Mem Reg ALU Beq Reg ALU Add Mem ALU O r d e r Time (clock cycles) ALU I n s t r. Mem Reg Delayed Branch: Redefine branch behavior (takes place after next instruction) Impact: 0 extra clock cycles per branch instruction if can find instruction to put in “slot” ( 50% of time) The longer the pipeline, the harder to fill Used by MIPS architecture

Control Hazard Solution #3: Delayed Branch

Data Hazard on r 1 An instruction depends on the result of a previous instruction still in the pipeline add r 1 , r 2, r 3 sub r 4, r 1 , r 3 and r 6, r 1 , r 7 or r 8, r 1 , r 9 xor r 10, r 11

Data Hazard on r 1: • Dependencies backwards in time are hazards Time (clock cycles) IF Dm Im Reg ALU xor r 10, r 11 Reg ALU or r 8, r 1, r 9 WB ALU and r 6, r 1, r 7 MEM ALU O r d e r sub r 4, r 1, r 3 Im EX ALU I n s t r. add r 1, r 2, r 3 ID/RF Reg Reg Dm Reg

Data Hazard Solution: • “Forward” result from one stage to another Time (clock cycles) IF Dm Im Reg ALU xor r 10, r 11 Reg ALU or r 8, r 1, r 9 WB ALU and r 6, r 1, r 7 MEM ALU O r d e r sub r 4, r 1, r 3 Im EX ALU I n s t r. add r 1, r 2, r 3 ID/RF Reg Reg Dm Reg • “or” OK if define read/write properly • Forwarding can’t prevent all data hazard! – lw followed by R-type?

Forwarding (or Bypassing): What about Loads? • Dependencies backwards in time are hazards Time (clock cycles) IF MEM Reg Dm Im Reg ALU sub r 4, r 1, r 3 Im EX ALU lw r 1, 0(r 2) ID/RF WB Reg Dm Reg • Can’t solve with forwarding: • Must delay/stall instruction dependent on loads

Forwarding (or Bypassing): What about Loads • Dependencies backwards in time are hazards Time (clock cycles) IF Reg Stall MEM WB Dm Reg Im Reg ALU sub r 4, r 1, r 3 Im EX ALU lw r 1, 0(r 2) ID/RF Dm Reg • Can’t solve with forwarding: • Must delay/stall instruction dependent on loads

Summary: Pipelining What makes it easy all instructions are the same length just a few instruction formats memory operands appear only in loads and stores; Memory addresses are asigned What makes it hard? structural hazards: suppose we had only one memory control hazards: need to worry about branch instructions data hazards: an instruction depends on a previous instruction We’ll build a simple pipeline and look at these issues We’ll talk about modern processors and what really makes it hard: exception handling trying to improve performance with out-of-order execution, etc.

Summary & Questions Pipelining is a fundamental concept multiple steps using distinct resources Utilize capabilities of the Datapath by pipelined instruction processing start next instruction while working on the current one limited by length of longest stage (plus fill/flush) detect and resolve hazards Questions?