COMP 541 Pipelined MIPS Montek Singh Nov 20

COMP 541 Pipelined MIPS Montek Singh Nov {20, 27}, 2017 1

Topics ã Today’s topic: Pipelining l Can think of it as: Ø A way to parallelize, or Ø A way to make better utilization of the hardware. l Goal: Try to use all hardware every clock cycle ã Reading l Section 7. 5 of textbook 2

Parallelism ã Two types of parallelism: l Spatial parallelism Ø duplicate hardware performs multiple tasks at once l Temporal parallelism Ø overlapping execution of successive instructions Ø task is broken into multiple stages – each stage operating on different parts of distinct instructions Ø also called pipelining – example: an assembly line 3

Parallelism Definitions ã Some definitions: l Token: A group of inputs processed together to produce a group of outputs Ø a “bundle” Ø an instruction is a token l Latency: Time for one token to pass from start to end l Throughput: Number of tokens processed per unit time ã Parallelism increases throughput l Often sacrificing latency Ø Why? 4

Parallelism Example: Baking Cookies! ã Ben is baking cookies l 2 -part task: Ø It takes 5 minutes to roll the cookies… Ø … and 15 minutes to bake them l After finishing one batch he immediately starts the next batch. What is the latency and throughput if Ben does NOT use parallelism? Latency = 5 + 15 = 20 min = 1/3 hour Throughput = 1 tray/ 20 min = 3 trays/hour 5

Parallelism Example: Baking Cookies! ã What is the latency and throughput if Ben uses parallelism? l Spatial parallelism: Ben asks Allysa to help, using two ovens/kitchens instead of one. l Temporal parallelism: Ben breaks the task into two stages: roll and baking. He uses two trays. While the first batch is baking he rolls the second batch, and so on. 6

Baking Cookies: Spatial Parallelism Latency = ? Throughput = ? 7

Baking Cookies: Spatial Parallelism Latency = 5 + 15 = 20 min = 1/3 hour (same) Throughput = 2 trays/ 20 min = 6 trays/hour (doubled) 8

Baking Cookies: Temporal Parallelism Latency = ? Throughput = ? Note: This scenario assumes each step is performed as soon as a resource is available. Latency will be lower if rolling is started as late as possible. 9

Baking Cookies: Temporal Parallelism Latency = 5 + 15 = 20 min = 1/3 hour Throughput = 1 trays/ 15 min = 4 trays/hour Using both types of parallelism together, the throughput would be 8 trays/hour Cost? An extra tray 10

Another example: Laundry Example: l Ann, Brian, Cathy, Dave each have one load of clothes to wash, dry, and fold A B C D l Washer takes 30 minutes l Dryer takes 40 minutes l Folding takes 20 minutes 11

Sequential Laundry 6 PM 7 8 9 10 11 Midnight Time 30 40 20 T a s k O r d e r A B C D ã Sequential laundry takes 6 hours for 4 loads With pipelining, how long would laundry take? 12

Pipelined Laundry ? ? 6 PM Time T a s k O r d e r A B C D Pipelined laundry takes ? ? hours for 4 loads 13

Pipelined Laundry 6 PM 7 8 9 10 11 Midnight Time 30 40 T a s k O r d e r 40 40 40 20 A B C D Pipelined laundry takes 3. 5 hours for 4 loads 14

Pipelining Principles ã Pipelining doesn’t help latency 6 PM 7 8 9 Time 30 40 T a s k O r d e r A 40 40 40 20 of single task, it helps throughput of entire workload ã Pipeline rate limited by slowest pipeline stage ã Multiple tasks operating simultaneously ã Potential speedup = Number B pipe stages ã Unbalanced stage delays C D reduces speedup ã Time to “fill” pipeline and to “drain” it reduces speedup l can be ignored for very large number of laundry loads 15

Pipelined MIPS ã Temporal parallelism ã Divide single-cycle processor into 5 stages: l Fetch l Decode l Execute l Memory l Writeback ã Add pipeline registers between stages l sort of like extra cookie trays 16

Pipelined MIPS ã Divide single-cycle processor into 5 stages: l Fetch (IF) / Decode (ID) / Execute (EX) / Memory (MEM) / Writeback (WB) Fetch Decode Execute Memory Write-Back 17

Single-Cycle vs. Pipelined Performance ã Pipelining l Break instruction into 5 steps l Each is 250 ps long (length of longest step) Write Reg and Read Reg occur during the first half and second half of the clock cycle, respectively. This allows more instruction overlap! 18

Pipelining Abstraction ã 5 stages of execution l instruction memory (fetch) l register file read l ALU/execution l data memory access l register file write (writeback) 19

Single-Cycle and Pipelined Datapath ã Key difference: insertion of pipeline registers 20

Multi-Cycle and Pipelined Datapath ã Key difference: full-length registers for datapath and control 21

One Problem ã There is a problem: mismatched paths l Result to write back and reg des are out of step 22

Corrected Pipelined Datapath ã Solution: One modification needed l Keep result and reg dest together Ø both go through equal number of pipeline stages l The belong together as a single “token” 23

Pipelined Control ã What is similar/different w. r. t. single-cycle MIPS? l Same control signal values l Values must be delayed and delivered in correct cycles Control signals must go through registers as well! “Token” = data + control! 24

Pipelining Challenges ã “Hazards” l when an instruction depends on results from previous instruction that has not yet completed l 2 types of hazards: Ø Data hazard: register value not written back to register file yet – an instruction produces a result needed by next instruction – new value will be stored during write-back stage – new value needed by next instr during register read following too close behind! Ø Control hazard: don’t know which is next instruction – – – next PC not decided yet cause by conditional branches cannot fetch next instr if current branch is not yet decided 25

Data Hazard: Example ã First instruction computes a result ($s 0) needed by the next 3 instructions l first two cause problems l third actually does not! Ø Why not? Register file written in first half of clock cycle, read in second half 26

Handling Data Hazards ã Static/compiler approaches: l Insert nop’s (no operations) in code at compile time l Rearrange code at compile time ã Dynamic/runtime approaches: l Forward data at run time Ø data goes directly from producer to consumer l Stall the consuming instruction at run time 27

Compile-Time Hazard Elimination Insert enough nops between dependent instrs ã Or re-arrange: move independent instructions up 28

Compile-Time Hazard Elimination ã Insert enough nops between dependent instrs Or re-arrange: move independent instructions up xor $t 6, $t 7, $t 8 xor $t 9, $t 6, $t 10 xor 29

Dynamic Approach: Data Forwarding ã Also known as register bypassing l results are actually available even though not stored in reg. l grab a copy and send where needed! Ø extra hardware needed l Note: forwarding actually not needed for sub. Why again? 30

Data Forwarding: from MEM stage X X 31

Data Forwarding: from Write. Back if ((rs. E != 0) AND (rs. E == Write. Reg. M) AND Reg. Write. M) then Forward. AE = 10 else if ((rs. E != 0) AND (rs. E == Write. Reg. W) AND Reg. Write. W)) then Forward. AE = 01 else Forward. AE = 00 X X 32

Data Forwarding ã Forward to Execute stage from either: l Memory stage (MEM) or l Write. Back (WB) stage ã Forwarding logic for Forward. AE: l for instri in Execute stage: Ø if the destination reg of instri-1 matches source reg, then forward from Mem stage Ø if the destination reg of instri-2 matches source reg, then forward from WB stage l logic equations: if ((rs. E != 0) AND (rs. E == Write. Reg. M) AND Reg. Write. M) then Forward. AE = 10 else if ((rs. E != 0) AND (rs. E == Write. Reg. W) AND Reg. Write. W) then Forward. AE = 01 else Forward. AE = 00 ã Forwarding logic for Forward. BE same, but replace rs. E with rt. E 33

Forwarding may not always work… ã Example: Load followed immediately by R-type l loads are harder to deal with because they need mem access! Ø data cannot go backwards in time! Ø need one extra cycle, compared to R-type instructions lw has a 2 -cycle latency! 34

Stalling ã Stall for a cycle, then forward l solves the Load followed by R-type problem 35

Stalling Hardware lwstall = ((rs. D == rt. E) OR (rt. D == rt. E)) AND Memto. Reg. E Stall. F = Stall. D = Flush. E = lwstall 36

Control Hazards ã beq: l branch is not determined until the fourth stage of the pipeline l Instructions after the branch are fetched before branch occurs l These instructions must be flushed if the branch is taken 37

Effect & Solutions ã Could always stall until branch is completed l Expensive: 3 cycles lost per branch! ã Could predict banch outcome, and flush if wrong l Branch misprediction penalty Ø Instructions flushed when branch is taken Ø May be reduced by determining branch earlier 38

Control Hazards: Flushing ã Flushing l turn instruction into a NOP (put all zeros) l renders it harmless! 39

Control Hazards: Original Pipeline (for comparison) 40

Control Hazards: Early Branch Resolution Introduced another data hazard in Decode stage (fix a few slides away) 41

Control Hazards with Early Branch Resolution Penalty now only one lost cycle 42

Aside: Delayed Branch ã Pipelined MIPS always executes instruction following a branch l So branch is delayed ã This allows us to avoid killing next instruction l Compilers move instruction that has no conflict w/ branch into delay slot 43

Example ã This sequence add beq … $4, $5, $6 $1, $2, label: … reordered to this beq add … $1, $2, label $4, $5, $6 label: ã The add is always executed! l If branch is taken Ø add is executed before the PC is set to the branch target l If branch is not taken Ø add is executed as the first instruction in follow-through 44

Handling the New Hazards 45

Control Forwarding and Stalling Hardware ã Forwarding logic: Forward. AD Forward. BD = = (rs. D (rt. D !=0) AND (rs. D (rt. D == == Write. Reg. M) AND Reg. Write. M ã Stalling logic: branchstall = Branch. D AND Reg. Write. E AND (Write. Reg. E == rs. D OR Write. Reg. E == rt. D) OR Branch. D AND Memto. Reg. M AND (Write. Reg. M == rs. D OR Write. Reg. M == rt. D) Stall. F = Stall. D = Flush. E = lwstall OR branchstall 46

Branch Prediction ã Strategy is to predict outcome and start fetching from predicted address l Especially important if branch penalty > 1 cycle l Guess whether branch will be taken Ø Backward branches are usually taken (loops) Ø Perhaps consider history of whether branch was previously taken to improve the guess l Start fetching from the correct place Ø Either the next instruction if prediction is branch will not be taken Ø Or, from the branch target otherwise l Good prediction reduces fraction of branches requiring a flush 47

Pipelined Performance Example ã Ideally CPI = 1 l But less due to: stalls (caused by loads and branches) ã SPECINT 2000 benchmark: l 25% loads l 10% stores l 11% branches l 2% jumps l 52% R-type ã Suppose: l 40% of loads used by next instruction l 25% of branches mispredicted l All jumps flush next instruction ã What is the average CPI? 48

Pipelined Performance Example ã SPECINT 2000 benchmark: l 25% loads l 10% stores l 11% branches l 2% jumps l 52% R-type ã Suppose: l 40% of loads used by next instruction l 25% of branches mispredicted l All jumps flush next instruction ã What is the average CPI? l Load/Branch CPI = 1 when no stalling, 2 when stalling. Thus, Ø CPIlw = 1(0. 6) + 2(0. 4) = 1. 4 Ø CPIbeq = 1(0. 75) + 2(0. 25) = 1. 25 Ø Average CPI = (0. 25)(1. 4) + (0. 1)(1) + (0. 11)(1. 25) + (0. 02)(2) + (0. 52)(1) = 1. 15 49

Pipelined Performance ã Pipelined processor critical path: Tc = max { tpcq + tmem + tsetup, 2(t. RFread + tmux + teq + t. AND + tmux + tsetup ), tpcq + tmux + t. ALU + tsetup, tpcq + tmemwrite + tsetup, 2(tpcq + tmux + t. RFwrite) } 50

Pipelined Performance Example Tc = 2(t. RFread + tmux + teq + t. AND + tmux + tsetup ) = 2[150 + 25 + 40 + 15 + 20] ps = 550 ps 51

Pipelined Performance Example ã For a program with 100 billion instructions executing on a pipelined MIPS processor, l CPI = 1. 15 l Tc = 550 ps Execution Time = (# instructions) × CPI × Tc = (100 × 109)(1. 15)(550 × 10 -12) = 63. 25 seconds Processor Execution Time (seconds) Speedup (w. r. t. single-cycle) Single-cycle 92. 5 1 Multicycle 133. 9 0. 69 Pipelined 63. 25 1. 46 52

Summary ã Pipelining benefits l use hardware more efficiently l throughput increases ã Pipelining challenges/drawbacks l latency increases l hazards ensue l energy/power consumption increases ã All modern processors are pipelined l some have way more than 5 pipeline stages Ø some Pentium’s have had 20 -40 pipeline stages! 53