CS 3350 B Computer Architecture Winter 2015 Lecture

CS 3350 B Computer Architecture Winter 2015 Lecture 6. 1: Fundamentals of Instructional Level Parallelism Marc Moreno Maza www. csd. uwo. ca/Courses/CS 3350 b [Adapted from lectures on Computer Organization and Design, Patterson & Hennessy, 5 th edition, 2011] 0

Analogy: Gotta Do Laundry q Ann, Brian, Cathy, Dave each have one load of clothes to wash, dry, fold, and put away l Washer takes 30 minutes l Dryer takes 30 minutes l “Folder” takes 30 minutes l “Stasher” takes 30 minutes to put clothes into drawers A B C D 1

Sequential Laundry 6 PM T a s k O r d e r A 7 8 9 10 11 12 1 2 AM 30 30 30 30 Time B C D q Sequential laundry takes 8 hours for 4 loads 2

Pipelined Laundry 6 PM T a s k O r d e r 7 8 9 30 30 10 11 12 1 2 AM Time A B C D q Pipelined laundry takes 3. 5 hours for 4 loads! 3

Pipelining Lessons (1/2) 6 PM T a s k O r d e r 7 8 9 q Pipelining doesn’t help latency of single task, it helps throughput of entire workload q Multiple tasks operating simultaneously using different resources q Potential speedup = Number of pipe stages q Time to “fill” pipeline and time to “drain” it reduces speedup: 2. 3 x vs. 4 x in this example Time 30 30 A B C D 4

Pipelining Lessons (2/2) q Suppose new Washer takes 20 minutes, new Stasher takes 20 minutes. How much faster is pipeline? B C q Pipeline rate limited by slowest pipeline stage D q Unbalanced lengths of pipe stages reduces speedup 6 PM T a s k O r d e r 7 8 9 Time 30 30 A 5

Recap: MIPS Three Instruction Formats 31 25 op R-format: 31 I-format: rs 25 op 31 J-format: 20 15 rt 20 rs 10 rd shamt 0 funct 15 rt 0 address offset 25 op 5 0 target address Examples: q R-format: add, sub, jr q I-format: lw, sw, beq, bne q J-format: j, jal 6

Recap: Five Stages in Executing MIPS (1) Instruction Fetch (IFetch) • Fetch an instruction; increment PC (2) Instruction Decode (Dec) • Decode instruction; read registers; sign extend offset (3) ALU (Arithmetic-Logic Unit) (Exec) • Execute R-format operations; calculate memory address; branch comparison; branch and jump completion (4) Memory Access (Mem) • Read data from memory for load or write data to memory for store (5) Register Write (WB) • Write data back to register 7

+4 Short name: rd rs rt ALU imm 1. Instruction Fetch 2. Decode/ Register Read IFetch Dec I$ Reg 5. Register 3. Execute 4. Memory Write Exec ALU Graphical Representation: Data memory registers PC instruction memory Graphical Representation D$ Mem WB Reg 8

Single Cycle CPU Clocking q All stages of an instruction completed within one long clock cycle l Clock cycle sufficiently long to allow each instruction to complete all stages without interruption within one cycle 1. Instruction Fetch 2. Decode/ Register Read 3. Execute 4. Memory 5. Reg. Write 9

Single Cycle Performance q Assume time for actions are l l q 100 ps for register read or write 200 ps for other events Clock rate of the single cycle datapath is? Instr fetch Register read ALU op Memory access Register write Total time lw 200 ps 100 ps 800 ps sw 200 ps 100 ps 200 ps R-format 200 ps 100 ps 200 ps beq 200 ps 100 ps 200 ps 700 ps 100 ps 600 ps 500 ps • What can we do to improve clock rate? • Will this improve performance as well? Want increased clock rate to mean faster programs 10

Multiple-cycle CPU Clocking q Only one stage of instruction per clock cycle l Clock is made as long as the slowest stage 1. Instruction 2. Decode/ Fetch Register Read l 3. Execute 4. Memory 5. Register Write Advantages over single cycle execution: Unused stages in a particular instruction can be skipped OR instructions can be pipelined (overlapped) 11

How Can We Make It Faster? q Split the multiple instruction cycle into smaller and smaller steps l q q There is a point of diminishing returns where as much time is spent loading the state registers as doing the work Start fetching and executing the next instruction before the current one has completed l Pipelining – (all? ) modern processors are pipelined for performance l Remember the performance equation: CPU time = CPI * CC * IC Fetch (and execute) more than one instruction at a time l Superscalar processing – stay tuned 12

A Pipelined MIPS Processor q Start the next instruction before the current one has completed l l improves throughput - total amount of work done in a given time instruction latency (execution time, delay time, response time from the start of an instruction to its completion) is not reduced Cycle 1 Cycle 2 Cycle 3 Cycle 4 Cycle 5 Cycle 6 Cycle 7 Cycle 8 lw sw R-type IFetch Dec Exec Mem WB - clock cycle (pipeline stage time) is limited by the slowest stage - for some instructions, some stages are wasted cycles 13

Why Pipeline? For Performance! q Under ideal conditions and with a large number of instructions, the speed-up from pipelining is approximately equal to the number of pipe stages. l A five-stage pipeline is nearly five times faster. Time (clock cycles) D$ Reg I$ Reg ALU Inst 4 Reg ALU Inst 3 Inst 2 I$ Reg ALU O r d e r Inst 1 Once the pipeline is full, one instruction is completed every cycle, so CPI = 1 D$ ALU Inst 0 Reg ALU I n s t r. I$ D$ Time to fill the pipeline Reg 14

Pipeline Performance q Assume l l 100 ps for register read or write 200 ps for other stages q What l time for stages is is pipelined clock rate? Compare pipelined datapath with single-cycle datapath Instr fetch Register read ALU op Memory access Register write Total time lw 200 ps 100 ps 800 ps sw 200 ps 100 ps 200 ps R-format 200 ps 100 ps 200 ps beq 200 ps 100 ps 200 ps 700 ps 100 ps 600 ps 500 ps 15

Pipeline Performance Single-cycle (Tc= 800 ps) Pipelined (Tc= 200 ps) 16

Pipeline Speedup q If all stages are balanced (i. e. all take the same time) and there are no dependencies between the instructions, l CPI = 1 (each instruction takes 5 cycles, but 1 completes each cycle) l Ideal speedup is: Number of stages = Time between instructions nonpipelined Time between instructions pipelined q Speedup due to increased throughput; Latency (time for each instruction) does not decrease q If not balanced, speedup is less • Pipelining the three lw, speedup: 2400 ps/1400 ps = 1. 7 • Add 1000, 000 more instructions, the speedup: (10^6*200+1400)/(10^6*800+2400) ~ 800/200 = 4 17

MIPS Pipeline Datapath Modifications q What do we need to add/modify in our MIPS datapath? l Add State registers between each pipeline stage to isolate them IF: IFetch ID: Dec EX: Execute MEM: Mem. Access WB: Write. Back Add Read Addr 1 Read Addr 2 Data 1 File Write Addr Write Data 16 System Clock Sign Extend Read Data 2 ALU Exec/Mem Register Read Dec/Exec Read Address IFetch/Dec PC Instruction Memory Add Data Memory Address Write Data Read Data Mem/WB Shift left 2 4 32 18

Pipelining the MIPS ISA q MIPS Instruction Set designed for pipelining q All instructions are 32 -bits l Easier to fetch and decode in one cycle l x 86: 1 - to 17 -byte instructions (x 86 HW actually translates to internal RISC instructions!) q Few and regular instruction formats, 2 source register fields always in same place l q Memory operands only in Loads and Stores l q Can decode and read registers in one step Can calculate address at 3 rd stage, access memory at 4 th stage Alignment of memory operands l Memory access takes only one cycle 19

Other Sample Pipeline Alternatives q ARM 7 IM Reg PC update IM access Intel XScale IM 1 PC update BTB access start IM access IM 2 ALU op DM access shift/rotate commit result (write back) Reg SHFT decode reg 1 access IM access ALU q decode reg access EX DM 1 Reg DM 2 DM write reg write start DM access exception ALU op shift/rotate reg 2 access 20

Can Pipelining Get us Into Trouble? q Yes: Pipeline Hazards l structural hazards: attempt to use the same resource by two different instructions at the same time l data hazards: attempt to use data before it is ready - An instruction’s source operand(s) are produced by a prior instruction still in the pipeline l control hazards: attempt to make a decision about program control flow before the condition has been evaluated and the new PC target address calculated - branch instructions q Can always resolve hazards by waiting l l pipeline control must detect the hazard and take action to resolve hazards 21

Takeaway q All modern day processors use pipelining q Pipelining doesn’t help latency of single task, it helps throughput of entire workload q Potential speedup: a CPI of 1 and a faster CC l q q Recall CPU time = CPI * CC * IC Pipeline rate limited by slowest pipeline stage l Unbalanced pipe stages make for inefficiencies l The time to “fill” pipeline and time to “drain” it can impact speedup for deep pipelines and short code runs Must detect and resolve hazards l Stalling negatively affects CPI (makes CPI more than the ideal of 1) l Compiler can arrange code to avoid hazards and stalls: Requires knowledge of the pipeline structure 22