ECE 313 Computer Organization Lecture 17 Pipelined Processor

ECE 313 - Computer Organization Lecture 17 - Pipelined Processor Design 1 Fall 2004 Reading: 6. 1 -6. 2 Prof. John Nestor ECE Department Lafayette College Easton, Pennsylvania 18042 nestorj@lafayette. edu Portions of these slides are derived from: Textbook figures © 1998 Morgan Kaufmann Publishers all rights reserved Tod Amon's COD 2 e Slides © 1998 Morgan Kaufmann Publishers all rights reserved Dave Patterson’s CS 152 Slides - Fall 1997 © UCB Rob Rutenbar’s 18 -347 Slides - Fall 1999 CMU ECE 313 Fall 2004 Lecture 17 - Pipelining 1 other sources as noted 1

Basic Pipelined Processor Pipeline Registers IF/ID ECE 313 Fall 2004 ID/EX Lecture 17 - Pipelining 1 EX/MEM MEM/WB 15

Single-Cycle vs. Pipelined Execution Non-Pipelined ECE 313 Fall 2004 Lecture 17 - Pipelining 1 16

Comments about Pipelining } The good news } Multiple instructions are being processed at same time } This works because stages are isolated by registers } Best case speedup of N } The bad news } Instructions interfere with each other - hazards • Example: different instructions may need the same piece of hardware (e. g. , memory) in same clock cycle • Example: instruction may require a result produced by an earlier instruction that is not yet complete } Worst case: must suspend execution - stall ECE 313 Fall 2004 Lecture 17 - Pipelining 1 17

Pipelined Example Executing Multiple Instructions } Consider the following instruction sequence: lw $r 0, 10($r 1) sw $r 3, 20($r 4) add $r 5, $r 6, $r 7 sub $r 8, $r 9, $r 10 ECE 313 Fall 2004 Lecture 17 - Pipelining 1 18

Executing Multiple Instructions Clock Cycle 1 LW ECE 313 Fall 2004 Lecture 17 - Pipelining 1 19

Executing Multiple Instructions Clock Cycle 2 SW ECE 313 Fall 2004 LW Lecture 17 - Pipelining 1 20

Executing Multiple Instructions Clock Cycle 3 ADD ECE 313 Fall 2004 SW LW Lecture 17 - Pipelining 1 21

Executing Multiple Instructions Clock Cycle 4 SUB ECE 313 Fall 2004 ADD SW Lecture 17 - Pipelining 1 LW 22

Executing Multiple Instructions Clock Cycle 5 SUB ECE 313 Fall 2004 ADD Lecture 17 - Pipelining 1 SW LW 23

Executing Multiple Instructions Clock Cycle 6 SUB ECE 313 Fall 2004 Lecture 17 - Pipelining 1 ADD SW 24

Executing Multiple Instructions Clock Cycle 7 ADD SUB ECE 313 Fall 2004 Lecture 17 - Pipelining 1 25

Executing Multiple Instructions Clock Cycle 8 SUB ECE 313 Fall 2004 Lecture 17 - Pipelining 1 26

Alternative View - Multicycle Diagram ECE 313 Fall 2004 Lecture 17 - Pipelining 1 27

Pipeline Hazards } Where one instruction cannot immediately follow another } Types of hazards } Structural hazards - attempt to use same resource twice } Control hazards - attempt to make decision before condition is evaluated } Data hazards - attempt to use data before it is ready } Can always resolve hazards by waiting ECE 313 Fall 2004 Lecture 17 - Pipelining 1 29

Structural Hazards } Attempt to use same resource twice at same time } Example: Single Memory for instructions, data } Accessed by IF stage } Accessed at same time by MEM stage } Solutions } Delay second access by one clock cycle, OR } Provide separate memories for instructions, data • This is what the book does • This is called a “Harvard Architecture” • Real pipelined processors have separate caches ECE 313 Fall 2004 Lecture 17 - Pipelining 1 30

Example Structural Hazard Single Memory Conflict ECE 313 Fall 2004 Lecture 17 - Pipelining 1 31

Control Hazards } Attempt to make a decision before condition is evaluated } Example: beq $s 0, $s 1, offset } Assume we add hardware to second stage to: } Compare fetched registers for equality } Compute branch target } This allows branch to be taken at end of second clock cycle } But, this still means result is not ready when we want to load the next instruction! ECE 313 Fall 2004 Lecture 17 - Pipelining 1 32

Control Hazard Solutions } Stall - stop loading instructions until result is available } Predict - assume an outcome and continue fetching (undo if prediction is wrong) } Delayed branch - specify in architecture that following instruction is always executed ECE 313 Fall 2004 Lecture 17 - Pipelining 1 33

Control Hazard - Stall ECE 313 Fall 2004 beq writes PC here new PC used here Lecture 17 - Pipelining 1 34

Control Hazard - Correct Prediction Fetch assuming branch taken ECE 313 Fall 2004 Lecture 17 - Pipelining 1 35

Control Hazard - Incorrect Prediction “Squashed” instruction ECE 313 Fall 2004 Lecture 17 - Pipelining 1 36

Control Hazard - Delayed Branch always executes correct PC avail. here ECE 313 Fall 2004 Lecture 17 - Pipelining 1 37

Summary - Control Hazard Solutions } Stall - stop fetching instr. until result is available } Significant performance penalty } Hardware required to stall } Predict - assume an outcome and continue fetching (undo if prediction is wrong) } Performance penalty only when guess wrong } Hardware required to "squash" instructions } Delayed branch - specify in architecture that following instruction is always executed } Compiler re-orders instructions into delay slot } Insert "NOP" (no-op) operations when can't use (~50%) } This is how original MIPS worked ECE 313 Fall 2004 Lecture 17 - Pipelining 1 38

Data Hazards } Attempt to use data before it is ready } Solutions } Stalling - wait until result is available } Forwarding- make data available inside datapath } Reordering instructions - use compiler to avoid hazards } Examples: add $s 0, $t 1 ; $s 0 = $t 0+$t 1 sub $t 2, $s 0, $t 3 ; $t 2 = $s 0 -$t 3 lw $s 0, 0($t 0) ; $s 0 = MEM[$t 0] sub $t 2, $s 0, $t 3 ; $t 2 = $s 0 -$t 3 ECE 313 Fall 2004 Lecture 17 - Pipelining 1 39

Data Hazard - Stalling ECE 313 Fall 2004 Lecture 17 - Pipelining 1 40

Data Hazards - Forwarding } Key idea: connect new value directly to next stage } Still read s 0, but ignore in favor of new result } Problem: what about load instructions? ECE 313 Fall 2004 Lecture 17 - Pipelining 1 41

Data Hazards – Forwarding the load result } STALL still required for load - data avail. after MEM } MIPS architecture calls this delayed load, initial implementations required compiler to deal with this ECE 313 Fall 2004 Lecture 17 - Pipelining 1 42

Data Hazards - Reordering Instructions } Assuming we have data forwarding, what are the hazards in this code? lw lw sw sw $t 0, $t 2, $t 0, 0($t 1) 4($t 1) } Reorder instructions to remove hazard: lw lw sw sw $t 0, $t 2, ECE 313 Fall 2004 0($t 1) 4($t 1) 0($t 1) Lecture 17 - Pipelining 1 43

Summary - Pipelining Overview } Pipelining increase throughput (but not latency) } Hazards limit performance } Structural hazards } Control hazards } Data hazards ECE 313 Fall 2004 Lecture 17 - Pipelining 1 44