Review Multiprocessor Basics q Q 1 How do

Review: Multiprocessor Basics q Q 1 – How do they share data? q Q 2 – How do they coordinate? q Q 3 – How scalable is the architecture? How many processors? # of Proc Communication Message passing 8 to 2048 model Shared NUMA 8 to 256 address UMA 2 to 64 Physical connection CMP&SMT. 1 Network 8 to 256 Bus 2 to 36

CMP: Multiprocessors On One Chip q By placing multiple processors, their memories and the IN all on one chip, the latencies of chip-to-chip communication are drastically reduced l ARM multi-chip core Per-CPU aliased peripherals Configurable between 1 & 4 symmetric CPUs Private peripheral bus CMP&SMT. 2 Configurable # of hardware intr Private IRQ Interrupt Distributor CPU CPU Interface CPU L 1$s Snoop Control Unit Primary AXI R/W 64 -b bus I & D CCB 64 -b bus Optional AXI R/W 64 -b bus

Multithreading on A Chip q Find a way to “hide” true data dependency stalls, cache miss stalls, and branch stalls by finding instructions (from other process threads) that are independent of those stalling instructions q Multithreading – increase the utilization of resources on a chip by allowing multiple processes (threads) to share the functional units of a single processor l Processor must duplicate the state hardware for each thread – a separate register file, PC, instruction buffer, and store buffer for each thread l The caches, buffers can be shared (although the miss rates may increase if they are not sized accordingly) l The memory can be shared through virtual memory mechanisms l Hardware must support efficient thread context switching CMP&SMT. 3

Types of Multithreading q q Fine-grain – switch threads on every instruction issue l Round-robin thread interleaving (skipping stalled threads) l Processor must be able to switch threads on every clock cycle l Advantage – can hide throughput losses that come from both short and long stalls l Disadvantage – slows down the execution of an individual thread since a thread that is ready to execute without stalls is delayed by instructions from other threads Coarse-grain – switches threads only on costly stalls (e. g. , L 2 cache misses) l Advantages – thread switching doesn’t have to be essentially free and much less likely to slow down the execution of an individual thread l Disadvantage – limited, due to pipeline start-up costs, in its ability to overcome throughput loss - Pipeline must be flushed and refilled on thread switches CMP&SMT. 4

Multithreaded Example: Sun’s Niagara (Ultra. Sparc T 1) 1. 2 GHz 1. 0 GHz Cache (I/D/L 2) 32 K/64 K/ (8 M external) 16 K/8 K/3 M Issue rate 4 issue 1 issue Pipe stages 14 stages 6 stages BHT entries 16 K x 2 -b None TLB entries 128 I/512 D 64 I/64 D Memory BW 2. 4 GB/s ~20 GB/s Transistors 29 million 200 million Power (max) 53 W CMP&SMT. 5 <60 W 4 -way MT SPARC pipe Clock rate 4 -way MT SPARC pipe 64 -b 4 -way MT SPARC pipe Data width 4 -way MT SPARC pipe Niagara 4 -way MT SPARC pipe Ultra III 4 -way MT SPARC pipe Eight fine grain multithreaded single-issue, in-order cores (no speculation, no dynamic branch prediction) 4 -way MT SPARC pipe q Crossbar 4 -way banked L 2$ Memory controllers I/O shared funct’s

Niagara Integer Pipeline q Cores are simple (single-issue, 6 stage, no branch prediction), small, and power-efficient Fetch Thrd Sel Decode Reg. File x 4 I$ ITLB Thrd Sel Mux CMP&SMT. 6 Inst bufx 4 Thrd Sel Mux Decode Thread Select Logic PC logicx 4 Execute ALU Mul Shft Div Memory D$ DTLB Stbufx 4 Instr type Cache misses Traps & interrupts Resource conflicts WB Crossbar Interface

Simultaneous Multithreading (SMT) q A variation on multithreading that uses the resources of a multiple-issue, dynamically scheduled processor (superscalar) to exploit both program ILP and threadlevel parallelism (TLP) l Most have more machine level parallelism than most programs can effectively use (i. e. , than have ILP) l With register renaming and dynamic scheduling, multiple instructions from independent threads can be issued without regard to dependencies among them - Need separate rename tables (ROBs) for each thread - Need the capability to commit from multiple threads (i. e. , from multiple ROBs) in one cycle q Intel’s Pentium 4 SMT called hyperthreading l CMP&SMT. 7 Supports just two threads (doubles the architecture state)

Threading on a 4 -way SS Processor Example Coarse MT Issue slots → Thread A Thread B Time → Thread C Thread D CMP&SMT. 8 Fine MT SMT

Multicore Xbox 360 – “Xenon” processor q q To provide game developers with a balanced and powerful platform l Three SMT processors, 32 KB L 1 D$ & I$, 1 MB UL 2 cache l 165 M transistors total l 3. 2 Ghz Near-POWER ISA l 2 -issue, 21 stage pipeline, with 128 -bit registers l Weak branch prediction – supported by software hinting l In order instructions l Narrow cores – 2 INT units, 2 128 -bit VMX units, 1 of anything else An ATI-designed 500 MZ GPU w/ 512 MB of DDR 3 DRAM l 337 M transistors, 10 MB framebuffer l 48 pixel shader cores, each with 4 ALUs CMP&SMT. 9

Xenon Diagram Core 1 Core 2 L 1 D L 1 I 1 MB UL 2 MC 0 512 MB DRAM CMP&SMT. 10 BIU/IO Intf MC 1 GPU SMC XMA Dec Core 0 DVD HDD Port Front USBs (2) Wireless MU ports (2 USBs) Rear USB (1) Ethernet IR Audio Out Flash Systems Control 3 D Core 10 MB EDRAM Video Out Analog Chip Video Out

The PS 3 “Cell” Processor Architecture q Composed of a Non-SMP Architecture l 234 M transistors @ 4 Ghz l 1 Power Processing Element, 8 “Synergistic” (SIMD) PE’s l 512 KB L 2 $ - Massively high bandwidth (200 GB/s) bus connects it to everything else l The PPE is strangely similar to one of the Xenon cores - Almost identical, really. Slight ISA differences, and fine-grained MT instead of real SMT l The real differences lie in the SPEs (21 M transistors each) - An attempt to ‘fix’ the memory latency problem by giving each processor complete control over it’s own 256 KB “scratchpad” – 14 M transistors – Direct mapped for low latency - 4 vector units per SPE, 1 of everything else – 7 M trans. CMP&SMT. 11

How to make use of the SPEs CMP&SMT. 12

What about the Software? q q q Makes use of special IBM “Hypervisor” l Like an OS for OS’s l Runs both a real time OS (for sound) and non-real time (for things like AI) Software must be specially coded to run well l The single PPE will be quickly bogged down l Must make use of SPEs wherever possible l This isn’t easy, by any standard What about Microsoft? l Development suite identifies which 6 threads you’re expected to run l Four of them are Direct. X based, and handled by the OS l Only need to write two threads, functionally CMP&SMT. 13