CS 295 Modern Systems The End of Conventional

CS 295: Modern Systems The End of Conventional Processor Performance Scaling Sang-Woo Jun Spring 2019

Moore’s Law q Typically cast as: “Performance doubles every X months” q Actually closer to: “Number of transistors per unit cost doubles every two years” The complexity for minimum component costs has increased at a rate of roughly a factor of two per year. […] Over the longer term, the rate of increase is a bit more uncertain, although there is no reason to believe it will not remain nearly constant for at least 10 years. -- Gordon Moore, Electronics, 1965 Why is Moore’s Law conflated with processor performance?

Dennard Scaling q

Single-Core Performance Scaling Projection What happened?

More Accurate Processor Power Consumption Dynamic power Prominent culprit Static power

Deeper Look At Leakage Current Constants q Subthreshold leakage: power leaked before voltage reaches threshold o Can be reduced by increasing threshold voltage (Vth) or decreasing voltage (V) o Lower voltage at the same threshold voltage -> unstable circuit o Threshold voltage does not scale at low size (Without reducing frequency … long story) o -> Voltage cannot scale!

End of Dennard Scaling q Option 1: Continue scaling frequency at increased power budget o Chip cannot be cooled! o Thermal runaway: Hotter chip -> increased resistance -> hotter chip -> …

Option 1: Continue Scaling Frequency at Increased Power Budget 0. 014 μ

Option 2: Stop Frequency Scaling Dennard Scaling Ended (~2006) Danowitz et. al. , “CPU DB: Recording Microprocessor History, ” Communications of the ACM, 2012

Looking Back: Change of Predictions

But Moore’s Law Continues Beyond 2006

State of Things at This Point (2006) q Single-thread performance scaling ended o Frequency scaling ended (Dennard Scaling) o Instruction-level parallelism scaling stalled … also around 2005 q Moore’s law continues o Double transistors every two years o What do we do with them? Instruction Level Parallelism K. Olukotun, “Intel CPU Trends”

Crisis Averted With Manycores?

Crisis Averted With Manycores?

What Happened? Only knob left to turn Gate-oxide stopped scaling Stopped scaling due to leakage Dynamic power Stopped scaling due to leakage Static power

What Used to Be: Intel Tick-Tock Model What happened (2019): In 2016, Intel deprecated tick-tock in favor of “process–architecture–optimization” Skylake (14 nm) now has five planned optimizations (Skylake, Kaby Lake, Coffee Lake, and Whiskey Lake) with no 10 nm chip in production yet

Not All Transistors Can Be Active! q Utilization wall: “With each successive process generation, the percentage of a chip that can switch at full frequency drops exponentially due to power constraints. ” -- Venkatesh, ASPLOS ‘ 10 q The following slides adapted from Michael Taylor’s 2012 talk “Is Dark Silicon Useful? Harnessing the Four Horsemen of the Coming Dark Silicon Apocalypse” – Marked ‘**’

Tradeoffs Between Cores And Frequency** 4 cores @ 1. 8 GHz Next generation … 4 x 4 cores @. 9 GHz (16 dim) … 2 x 4 cores @ 1. 8 GHz (8 cores dark, 8 dim) 4 cores @ 2 x 1. 8 GHz (12 cores dark)

The Four Horsemen** q What do we do with this dark silicon? q “Four top contenders, each of which seemed like an unlikely candidate from the beginning, carrying unwelcome burdens in design, manufacturing and programming. None is ideal, but each has its benefit and the optimal solution probably incorporates all four of them…”

The Shrinking Horseman (#1)** q “Area is expensive. Chip designers will just build smaller chips instead of having dark silicon in their designs!” q First, dark silicon doesn’t mean useless silicon, it just means it’s underclocked or not used all of the time. q There’s lots of dark silicon in current chips: o o o On-chip GPU on AMD Fusion or Intel Sandybridge for GCC • L 3 cache is very dark for applications with small working sets • SSE units for integer apps • Many of the resources in FPGAs not used by many designs (DSP blocks, PCI-E, Gig-E etc)

The Shrinking Horseman (#1)** q Competition and Margins o If there is an advantage to be had from using dark silicon, you have to use it too, to keep up with the Jones. q Diminished Returns (e. g. , $10 silicon selling for $200 today) o Savings Exponentially Diminishing: $5, $2. 5, $1. 25, 63 c o Overheads: packaging, test, marketing, etc. o Chip structures like I/O Pad Area do not scale q Exponential increase in Power Density -> Exponential Rise in Temperature q But, some chips will shrink o Nasty low margin, high competition chips; or a monopoly (Sony Cell)

The Dim Horseman (#2)** q Spatial dimming: Have enough cores to exceed power budget, but underclock them q Gen 1 & 2 Multicores (higher core count, lower freqs) q Near Threshold Voltage (NTV) Operation o Delay Loss > Energy Gain o But, make it up with lots of dim cores o Watch for Non-Ideal Speedups / Amdahl’s Law

The Dim Horseman (#2)** q Temporal Dimming : Have enough cores to exceed power budget, but use them only in bursts o Dim cores, but overclock if cold – e. g. , Intel Turbo. Boost o E. g. , ARM A 15 Core in mobile phone • A 15 power usage way above sustainable for phone. • 10 second bursts at most (big. LITTLE)

The Specialized Horseman (#3)** q “We will use all of that dark silicon area to build specialized cores, each of them tuned for the task at hand (10 -100 x more energy efficient), and only turn on the ones we need…” q Insights: o Power is now more expensive than area o Specialized logic can improve energy efficiency by 101000 x

Fine-Grained Parallelism of Special-Purpose Circuits q A = G × m 1 C = x 1 - x 2 E = y 1 - y 2 B = A × m 2 D = C 2 F = E 2 G = D + F Ret = B / G 4 cycles with basic operations A = G × m 1 × m 2 B = (x 1 - x 2)2 C = (y 1 - y 2)2 D = B + C Ret = B / G 3 cycles with compound operations May slow down clock Ret = (G × m 1 × m 2) / ((x 1 - x 2)2 + (y 1 - y 2)2) 1 cycle with even further compound operations

The Specialized Horseman (#3)** q C-cores Approach: o Fill dark silicon with Conservation Cores, or c-cores, which are automatically-generated, specialized energysaving coprocessors that save energy on common apps q Execution jumps among c-cores (hot code) and a host CPU (cold code) o Power-gate HW that is not currently in use o Coherent Memory & Patching Support for C-cores

Typical Energy Savings**

The Specialized Horseman (#3) -- Pssst q Another active thrust in this area is reconfigurable hardware acceleration using Field-Programmable Gate Arrays (FPGA) o A single FPGA fabric can be configured at runtime to act like any C-core o Not as efficient as a prefabricated C-core, but can cover any function at runtime o More on this later!

The Deus Ex Machina Horseman (#4)** q Deus Ex Machina: “A plot device whereby a seemingly unsolvable problem is suddenly and abruptly solved with the unexpected intervention of some new event, character, ability or object. ” q “MOSFETs are the fundamental problem” q “Fin. Fets, Trigate, High-K, nanotubes, 3 D, for one -time improvements, but none are sustainable solutions across process generations. ”

The Deus Ex Machina Horseman (#4)** q Possible “Beyond CMOS” Device Directions o o o Nano-electrical Mechanical Relays? Tunnel Field Effect Transistors (TFETS)? Spin-Transfer Torque MRAM (STT-MRAM)? Graphene? Human brain? DNA Computing?

Where Do GPUs Fit into This? q GPUs have hundreds of threads running at multiple gigabytes! q Much simpler processor architecture o Dozens of threads scheduled together in a SIMD fashion o Much simpler microarchitecture (doesn’t need to boot Linux!) q Much higher power budget o CPUs try to maintain 100 W power budget (Pentium 4 till now) o GPUs regularly exceed 400 W

The Exascale Challenge Department of Energy requests an exaflop machine by 2020 MIT Research nuclear reactor 1, 000, 000 floating point operations per second 6 MW Using 2016 technology, 200 MW Lynn Freeny, Department of Energy

Coming Up q Before we go into newer technologies, let’s first make sure we make good use of what we have o SIMD (SSE, AVX), Cache-optimized code, etc q “Our implementation delivers 9. 2 X the performance (RPS) and 2. 8 X the system energy efficiency (RPS/watt) of the best-published FPGA-based claims. ” o Li et. al. , Intel, “Architecting to Achieve a Billion Requests Per Second Throughput on a Single Key-Value Store Server Platform, ” ISCA 2015 o Intel software implementation of memcached

References q Kim, Nam Sung, et al. "Leakage Current: Moore’s Law Meets Static Power" computer 12 (2003): 68 -75.