Data Hazards Requiring Stall Cycles In some code

Data Hazards Requiring Stall Cycles • In some code sequence cases, potential data hazards cannot be handled by bypassing. For example: LW SUB AND OR R 1, 0 (R 2) R 4, R 1, R 5 R 6, R 1, R 7 R 8, R 1, R 9 • The LW (load word) instruction has the data in clock cycle 4 (MEM cycle). • The SUB instruction needs the data of R 1 in the beginning of that cycle. • Hazard prevented by hardware pipeline interlock causing a stall cycle. EECC 551 - Shaaban #1 Lec # 4 Fall 2000 9 -19 -2000

EECC 551 - Shaaban #2 Lec # 4 Fall 2000 9 -19 -2000

Hardware Pipeline Interlocks • A hardware pipeline interlock detects a data hazard and stalls the pipeline until the hazard is cleared. • The CPI for the stalled instruction increases by the length of the stall. • For the Previous example, (no stall cycle): LW R 1, 0(R 1) SUB R 4, R 1, R 5 AND R 6, R 1, R 7 OR R 8, R 1, R 9 IF ID IF EX ID IF MEM EX ID IF WB MEM WB EX MEM ID EX WB MEM WB With Stall Cycle: LW R 1, 0(R 1) SUB R 4, R 1, R 5 AND R 6, R 1, R 7 OR R 8, R 1, R 9 IF ID IF EX ID IF MEM STALL WB EX ID IF MEM WB EX MEM ID EX WB MEM WB EECC 551 - Shaaban #3 Lec # 4 Fall 2000 9 -19 -2000

EECC 551 - Shaaban #4 Lec # 4 Fall 2000 9 -19 -2000

EECC 551 - Shaaban #5 Lec # 4 Fall 2000 9 -19 -2000

EECC 551 - Shaaban #6 Lec # 4 Fall 2000 9 -19 -2000

EECC 551 - Shaaban #7 Lec # 4 Fall 2000 9 -19 -2000

Compiler Instruction Scheduling for Data Hazard Stall Reduction • Many types of stalls resulting from data hazards are very frequent. For example: A = B+ C produces a stall when loading the second data value (B). • Rather than allow the pipeline to stall, the compiler could sometimes schedule the pipeline to avoid stalls. • Compiler pipeline or instruction scheduling involves rearranging the code sequence (instruction reordering) to eliminate the hazard. EECC 551 - Shaaban #8 Lec # 4 Fall 2000 9 -19 -2000

Compiler Instruction Scheduling Example • For the code sequence: a=b+c d=e-f a, b, c, d , e, and f are in memory • Assuming loads have a latency of one clock cycle, the following code or pipeline compiler schedule eliminates stalls: Original code with stalls: LW Rb, b LW Rc, c Stall ADD Ra, Rb, Rc SW a, Ra LW Re, e LW Rf, f Stall SUB Rd, Re, Rf SW d, Rd Scheduled code with no stalls: LW Rb, b LW Rc, c LW Re, e ADD Ra, Rb, Rc LW Rf, f SW a, Ra SUB Rd, Re, Rf SW d, Rd EECC 551 - Shaaban #9 Lec # 4 Fall 2000 9 -19 -2000

Control Hazards • When a conditional branch is executed it may change the PC and, without any special measures, leads to stalling the pipeline for a number of cycles until the branch condition is known. • In current DLX pipeline, the conditional branch is resolved in the MEM stage resulting in three stall cycles as shown below: Branch instruction Branch successor + 1 Branch successor + 2 Branch successor + 3 Branch successor + 4 Branch successor + 5 IF ID EX MEM WB IF stall IF ID IF EX ID IF MEM EX ID IF WB MEM WB EX MEM ID EX IF ID IF Three clock cycles are wasted for every branch for current DLX pipeline EECC 551 - Shaaban #10 Lec # 4 Fall 2000 9 -19 -2000

Reducing Branch Stall Cycles Pipeline hardware measures to reduce branch stall cycles: 1 - Find out whether a branch is taken earlier in the pipeline. 2 - Compute the taken PC earlier in the pipeline. In DLX: – In DLX branch instructions BEQZ, BNEZ, test a register for equality to zero. – This can be completed in the ID cycle by moving the zero test into that cycle. – Both PCs (taken and not taken) must be computed early. – Requires an additional adder because the current ALU is not useable until EX cycle. – This results in just a single cycle stall on branches. EECC 551 - Shaaban #11 Lec # 4 Fall 2000 9 -19 -2000

Modified DLX Pipeline: Conditional Branches Completed in ID Stage EECC 551 - Shaaban #12 Lec # 4 Fall 2000 9 -19 -2000

Compile-Time Reduction of Branch Penalties • One scheme discussed earlier is to flush or freeze the pipeline by whenever a conditional branch is decoded by holding or deleting any instructions in the pipeline until the branch destination is known (zero pipeline registers, control lines)). • Another method is to predict that the branch is not taken where the state of the machine is not changed until the branch outcome is definitely known. Execution here continues with the next instruction; stall occurs here when the branch is taken. • Another method is to predict that the branch is taken and begin fetching and executing at the target; stall occurs here if the branch is not taken EECC 551 - Shaaban #13 Lec # 4 Fall 2000 9 -19 -2000

Predict Branch Not-Taken Scheme EECC 551 - Shaaban #14 Lec # 4 Fall 2000 9 -19 -2000

Static Compiler Branch Prediction Two basic methods exist to statically predict branches at compile time: 1 By examination of program behavior and the use of information collected from earlier runs of the program. – For example, a program profile may show that most forward branches and backward branches (often forming loops) are taken. The simplest scheme in this case is to just predict the branch as taken. 2 To predict branches on the basis of branch direction, choosing backward branches as taken and forward branches as not taken. EECC 551 - Shaaban #15 Lec # 4 Fall 2000 9 -19 -2000

EECC 551 - Shaaban #16 Lec # 4 Fall 2000 9 -19 -2000

Profile-Based Compiler Branch Misprediction Rates EECC 551 - Shaaban #17 Lec # 4 Fall 2000 9 -19 -2000

EECC 551 - Shaaban #18 Lec # 4 Fall 2000 9 -19 -2000

Reduction of Branch Penalties: Delayed Branch • When delayed branch is used, the branch is delayed by n cycles, following this execution pattern: conditional branch instruction sequential successor 1 sequential successor 2 ……. . sequential successorn branch target if taken • The sequential successor instruction are said to be in the branch delay slots. These instructions are executed whether or not the branch is taken. • In Practice, all machines that utilize delayed branching have a single instruction delay slot. • The job of the compiler is to make the successor instructions valid and useful instructions. EECC 551 - Shaaban #19 Lec # 4 Fall 2000 9 -19 -2000

Delayed Branch Example EECC 551 - Shaaban #20 Lec # 4 Fall 2000 9 -19 -2000

Delayed Branch-delay Slot Scheduling Strategies The branch-delay slot instruction can be chosen from three cases: A An independent instruction from before the branch: Always improves performance when used. The branch must not depend on the rescheduled instruction. B An instruction from the target of the branch: Improves performance if the branch is taken and may require instruction duplication. This instruction must be safe to execute if the branch is not taken. C An instruction from the fall through instruction stream: Improves performance when the branch is not taken. The instruction must be safe to execute when the branch is taken. The performance and usability of cases B, C is improved by using a canceling or nullifying branch. EECC 551 - Shaaban #21 Lec # 4 Fall 2000 9 -19 -2000

(A) (B) (C) EECC 551 - Shaaban #22 Lec # 4 Fall 2000 9 -19 -2000

Branch-delay Slot: Canceling Branches • In a canceling branch, a static compiler branch direction prediction is included with the branch-delay slot instruction. • When the branch goes as predicted, the instruction in the branch delay slot is executed normally. • When the branch does not go as predicted the instruction is turned into a no-op. • Canceling branches eliminate the conditions on instruction selection in delay instruction strategies B, C • The effectiveness of this method depends on whether we predict the branch correctly. EECC 551 - Shaaban #23 Lec # 4 Fall 2000 9 -19 -2000

EECC 551 - Shaaban #24 Lec # 4 Fall 2000 9 -19 -2000

DLX Performance Using Canceling Delay Branches EECC 551 - Shaaban #25 Lec # 4 Fall 2000 9 -19 -2000

Performance of Branch Schemes • The effective pipeline speedup with branch penalties: (assuming an ideal pipeline CPI of 1) Pipeline speedup = Pipeline depth 1 + Pipeline stall cycles from branches = Branch frequency X branch penalty Pipeline speedup = Pipeline Depth 1 + Branch frequency X Branch penalty EECC 551 - Shaaban #26 Lec # 4 Fall 2000 9 -19 -2000

Pipeline Performance Example • Assume the following DLX instruction mix: Type Arith/Logic Load Store branch Frequency 40% 30% of which 25% are followed immediately by an instruction using the loaded value 10% 20% of which 45% are taken • What is the resulting CPI for the pipelined DLX with forwarding and branch address calculation in ID stage when using a branch not-taken scheme? • CPI = Ideal CPI + Pipeline stall clock cycles per instruction = = 1 + 1 + 1. 165 stalls by loads + stalls by branches. 3 x. 25 x 1 +. 2 x. 45 x 1. 075 +. 09 EECC 551 - Shaaban #27 Lec # 4 Fall 2000 9 -19 -2000

Branch Penalty Example • For a pipeline similar to the MIPS R 4000, it takes three pipeline stages before the branch target address is known and an additional cycle before the branch condition is evaluated. • Assuming no stalls on the registers in the conditional comparison. The branch penalty for the three simplest branch prediction schemes: Branch Scheme Flush pipeline Predict taken Predict untaken Penalty unconditional 2. 0 Penalty untaken 3 3 0 Penalty taken 3 2 3 EECC 551 - Shaaban #28 Lec # 4 Fall 2000 9 -19 -2000

Pipelining and Handling of Exceptions • Exceptions are events that usually occur in normal program execution where the normal execution order of the instructions is changed (often called: interrupts, faults). • Types of exceptions include: • • • I/O device request Invoking an operating system service Tracing instruction execution Breakpoint (programmer-requested interrupt). Integer overflow or underflow FP anomaly Page fault (not in main memory) Misaligned memory access Memory protection violation Undefined instruction Hardware malfunctions EECC 551 - Shaaban #29 Lec # 4 Fall 2000 9 -19 -2000

Exception event IBM 360 VAX Motorola 680 x 0 Intel 80 x 86 I/O device request Input/output interruption Device interrupt Exception (Level 0. . . 7 autovector) Vectored interrupt Invoking the operating system service from a user program Supervisor call interruption Exception (change mode supervisor trap) Exception (unimplemented instruction)--on Macintosh Interrupt (INT instruction) Tracing instruction execution Not applicable Exception (trace fault) Exception (trace) Interrupt (singlestep trap) Breakpoint Not applicable Exception (breakpoint fault) Exception (illegal instruction or breakpoint) Interrupt (breakpoint trap) Integer arithmetic overflow or underflow; FP trap Program interruption (overflow or underflow exception) Exception (integer overflow trap or floating underflow fault) Exception (floating-point coprocessor errors) Interrupt (overflow trap or math unit exception) Page fault (not in main memory) Not applicable (only in 370) Exception (translation not valid fault) Exception (memorymanagement unit errors) Interrupt (page fault) Misaligned memory accesses Program interruption (specification exception) Not applicable Exception (address error) Not applicable Memory protection violations Program interruption (protection exception) Exception (access control violation fault) Exception (bus error) Interrupt (protection exception) Using undefined instructions Program interruption (operation exception) Exception (opcode privileged/ reserved fault) Exception (illegal instruction or breakpoint/unimplemented instruction) Interrupt (invalid opcode) Hardware malfunctions Machine-check interruption Exception (machine-check abort) Exception (bus error) Not applicable Power failure Machine-check interruption Urgent interrupt Not applicable Nonmaskable interrupt The names of common exceptions vary across four different architectures. EECC 551 - Shaaban #30 Lec # 4 Fall 2000 9 -19 -2000

Characteristics of Exceptions • Synchronous vs. asynchronous: Synchronous: occurs at the same place with the same data and memory allocation Asynchronous: Caused by devices external to the processor and memory. • User requested vs. coerced: User requested: The user task requests the event. Coerced: Caused by some hardware event. • User maskable vs. user nonmaskable: User maskable: Can be disabled by the user task using a mask. • Within vs. between instructions: Whether it prevents instruction completion by happening in the middle of execution. • Resuming vs. terminating: The program execution always stops after the event. Resuming: the program continues after the event. The state of the pipeline must be saved to handle this type of exception. The pipeline is restartable in this case. EECC 551 - Shaaban #31 Lec # 4 Fall 2000 9 -19 -2000

Handling of Resuming Exceptions • A resuming exception (e. g. a virtual memory page fault) usually requires the intervention of the operating system. • The pipeline must be safely shut down and its state saved for the execution to resume after the exception is handled as follows: 1 Force a trap instruction into the pipeline on the next IF. 2 Turn of all writes for the faulting instruction and all instructions in the pipeline. Place zeroes into pipeline latches starting with the instruction that caused the fault to prevent state changes. 3 The execution handling routine of the operating system saves the PC of the faulting instruction and other state data to be used to return from the exception. EECC 551 - Shaaban #32 Lec # 4 Fall 2000 9 -19 -2000

Exception Handling Issues • When using delayed branches , as many PCs as the length of the branch delay plus one need to be saved and restored to restore the state of the machine. • After the exception has been handled special instructions are needed to return the machine to the state before the exception occurred (RFE, Return to User code in DLX). • Precise exceptions imply that a pipeline is stopped so the instructions just before the faulting instruction are completed and those after it can be restarted from scratch. • Machines with arithmetic trap handlers and demand paging must support precise exceptions. EECC 551 - Shaaban #33 Lec # 4 Fall 2000 9 -19 -2000

Exceptions in DLX • The following represent problem exceptions for the DLX pipeline stages: IF ID EX MEM WB Page fault on instruction fetch; misaligned memory access; memory-protection violation. Undefined or illegal opcode Arithmetic exception Page fault on data fetch; misaligned memory access; memory-protection violation None • Example: LW IF ID EX MEM WB ADD IF ID EX MEM WB can cause a data page fault and an arithmetic exception at the same time ( LW in MEM and ADD in EX) Handled by dealing with data page fault and then restarting execution, then the second exception will occur but not the first. EECC 551 - Shaaban #34 Lec # 4 Fall 2000 9 -19 -2000

Precise Exception Handling in DLX • The instruction pipeline is required to handle exceptions of instruction i before those of instruction i+1 • The hardware posts all exceptions caused by an instruction in a status vector associated with the instruction which is carried along with the instruction as it goes through the pipeline. • Once an exception indication is set in the vector, any control signals that cause a data value write is turned off. • When an instruction enters WB the vector is checked, if any exceptions are posted, they are handled in the order they would be handled in an unpipelined machine. • Any action taken in earlier pipeline stages is invalid but cannot change the state of the machine since writes where disabled. EECC 551 - Shaaban #35 Lec # 4 Fall 2000 9 -19 -2000