Data Hazards in the Pipelined Implementation 21502 CSE

Data Hazards in the Pipelined Implementation 2/15/02 CSE 141 - Data Hazzards

Pipelined Implementation Datapath Instruction Fetch Instruction Decode/ Register Fetch Execute/ Address Calculation Memory Access Write Back Note: The IF stage may be labeled “IM” (instruction memory) Mem labeled “DM” (data memory); WB labeled “Reg” 2 CSE 141 - Data Hazzards

Data Hazards • When a result is needed in the pipeline before it is available, a “data hazard” occurs. CC 2 CC 3 IM Reg ALU DM R 2 Needed IM Reg ALU DM IM Reg ALU sub $2, $1, $3 CC 1 and $12, $5 or $13, $6, $2 add $14, $2 CC 4 3 2 3 CC 7 CC 8 R 2 Available Reg DM Reg 4 IM Reg ALU sw $15, 100($2) 1 Reg CC 6 No hazard here Hazards! Value in register hasn’t been updated Time: CC 5 DM 5 6 7 8 CSE 141 - Data Hazzards

Dealing With Data Hazards Transparent register file eliminates one hazard. Use latches rather than flip-flops in Reg file • First half-cycle of cycle 5: register 2 loaded • Second half-cycle: new value is read into pipeline state 4 Reg DM IM Reg ALU add $14, $2 IM ALU or $13, $6, $8 CC 3 ALU and $12, $6, $5 CC 2 ALU sub $2, $1, $3 CC 1 CC 4 CC 5 Reg CC 6 CC 7 CC 8 R 2 Available Reg DM Reg CSE 141 - Data Hazzards

Three ways to handle remaining Data Hazards • In Software – insert independent instructions (or no-ops) • In Hardware – insert bubbles (i. e. stall the pipeline) – data forwarding 5 CSE 141 - Data Hazzards

Dealing with Data Hazards in Software Reg DM IM Reg ALU add $12, $5 IM ALU nop CC 3 ALU nop CC 2 ALU sub $2, $1, $3 CC 1 CC 4 CC 5 CC 6 CC 7 CC 8 Reg Reg DM Reg Insert enough no-ops (or other instructions that don’t use register 2) so that data hazard doesn’t occur, 6 CSE 141 - Data Hazzards

Where are No-ops needed? sub $2, $1, $3 and $4, $2, $5 or $8, $2, $6 add $9, $4, $2 slt $1, $6, $7 7 CSE 141 - Data Hazzards

Handling Data Hazards in Hardware Stall the pipeline CC 1 CC 2 sub $2, $1, $3 IM Reg and $12, $5 IM or $13, $6, $2 add $14, $2 8 CC 3 Bubble CC 4 CC 5 DM Reg Bubble Reg IM CC 6 CC 7 DM Reg IM CC 8 Reg DM CSE 141 - Data Hazzards

Pipeline Stalls • To insure proper pipeline execution in light of register dependences, we must: – Detect the hazard – Stall the pipeline • prevent the IF and ID stages from making progress – the ID stage because we can’t go on until the dependent instruction completes correctly – the IF stage because we do not want to lose any instructions. • insert“no-ops” into later stages 9 CSE 141 - Data Hazzards

The Pipeline • What comparisons tell us when to stall? 10 CSE 141 - Data Hazzards

Stalling the Pipeline • Prevent the IF and ID stages from proceeding – don’t write the PC (PCWrite = 0) – don’t rewrite IF/ID register (IF/IDWrite = 0) • Insert “nops” – set all control signals propagating to EX/MEM/WB to zero 11 CSE 141 - Data Hazzards

The Pipeline 12 CSE 141 - Data Hazzards

Reducing Data Hazards Through Forwarding or $5, $3, $2 Reg DM IM Reg ALU IM ALU add $2, $3, $4 Reg DM Reg We could avoid stalling if we could get the ALU output from “add” to ALU input ID/EX EX/MEM MEM/WB for the “or” 13 ALU Registers 0 Data Memory 1 CSE 141 - Data Hazzards

Reducing Data Hazards Through Forwarding EX Hazard: if (EX/MEM. Reg. Write and (EX/MEM. Register. Rd != 0) and (EX/MEM. Register. Rd = ID/EX. Register. Rs)) Forward. A = 10 14 if (EX/MEM. Reg. Write and (EX/MEM. Register. Rd != 0) and (EX/MEM. Register. Rd = ID/EX. Register. Rt)) Forward. B = 10 CSE 141 - Data Hazzards (similar for the MEM stage)

Data Forwarding • Forwarding (just shown) handles two types of data hazards – EX hazard – MEM hazard • We’ve already handled the third type (WB) hazard by using a transparent reg file – if the register file is asked to read and write the same register in the same cycle, the reg file allows the write data to be forwarded to the output. 15 CSE 141 - Data Hazzards

Eliminating Data Hazards via Forwarding CC 2 CC 3 sub $2, $1, $3 IM Reg ALU DM and $6, $2, $5 IM Reg ALU DM IM Reg ALU CC 1 or $13, $6, $2 add $14, $2 sw $15, 100($2) 16 CC 4 CC 5 CC 6 CC 7 CC 8 Reg Reg DM CSE 141 - Data Hazzards

Does Forwarding eliminate all hazards? ? CC 2 CC 3 lw $2, 10($1) IM Reg ALU DM and $12, $5 IM Reg ALU DM IM Reg ALU CC 1 or $13, $6, $2 add $14, $2 sw $15, 100($2) 17 CC 4 CC 5 CC 6 CC 7 CC 8 Reg Reg DM CSE 141 - Data Hazzards

NO! You may need to stall after loads CC 1 IM or $13, $6, $2 add $14, $2 sw $15, 100($2) 18 CC 7 CC 8 DM Reg Bubble DM IM Bubble Reg DM IM Reg ALU and $12, $5 CC 6 ALU Reg CC 5 ALU lw $2, 10($1) IM CC 4 ALU CC 3 ALU CC 2 Reg CSE 141 - Data Hazzards

Try this one. . . Show stalls and forwarding for this code add $3, $2, $1 lw $4, 100($3) and $6, $4, $3 sub $7, $6, $2 19 CSE 141 - Data Hazzards

Data Hazard Key Points • Pipelining provides high throughput, but does not handle data dependences easily. • Data dependences cause data hazards. • Data hazards can be solved by: – software (no-ops) – hardware stalling – hardware forwarding • Our processor, and indeed all modern processors, use a combination of forwarding and stalling. 20 CSE 141 - Data Hazzards