Bridging the Moores Law Performance Gap with Innovation

Bridging the Moore’s Law Performance Gap with Innovation Scaling Todd Austin University of Michigan

Moore’s Law Performance Gap Today, gap is cresting 10 x 2

3

We Investigate: Who’s to Blame? ? 4

The Programmer? • Some say programmers cannot overcome hard parallel programming problems • Not the case, as parallel programming techniques and infrastructure is advancing more quickly than ever before • Map-reduce • Hadoop • Spark • No. SQL databases • Memcached • Cassandra 5

Largest NA Bitcoin Miner • GPGPU-based system • Fills 2000 sq. ft. warehouse • Computes 1 petahash/s • Reportedly generates $8 M in Bitcoins per month • Unfortunately soon to be obsolete as Bitcoin difficulty continues to scale 6

The Educator? • Perhaps CS education is failing to educate • Not the case, CS education is booming • @ Michigan CS enrollments are up nearly 250% from 5 years ago • Average entrant GPA’s are up as well • Quality of CS education is on an upward trend • Among the highest ranked educators by students (@ Michigan) • Young area has less “dead wood” than our more venerable engineering counterparts • Active-learning approach to education is the norm for CS in the US 7

CS Education in Ethiopia • I have been working with Addis Ababa Institute of Technology to develop CS and IT coursework since 2008 • Special focus on building infrastructure and developing active learning • Nearly 600 students in the CS program • 2 nd most popular major in the university • With many job opportunities • The first? 8

Silicon Technology? • Are transistors failing us, in the past scaling decreased latency, power, cost • Yes in part, silicon scaling has stalled for gate oxide layers • Due to leakage concerns, gate oxide scaling has stopped • This leaves threshold voltages high and prevents large decreases in Vdd • Because scaling has become uneven with the transistor • Dennard scaling has all but stopped • But, silicon is still bringing more transistors every generation, and will continue for perhaps a decade or so • Ultimately creates the dark silicon dilemma • Which prevents all of the chip to be active at the same time 9

The Dark Silicon Dilemma Courtesy Michael Taylor @ UCSD 10

The Dark Silicon Dilemma Courtesy Michael Taylor @ UCSD 11

The Dark Silicon Dilemma Courtesy Michael Taylor @ UCSD 12

Silicon Technology? • Are transistors failing us, in the past scaling decreased latency, power, cost • Yes in part, silicon scaling has stalled for gate oxide layers • Due to leakage concerns, gate oxide scaling has stopped • This leaves threshold voltages high and prevents large decreases in Vdd • Because scaling has become uneven with the transistor • Dennard scaling has all but stopped • But, silicon is still bringing more transistors every generation, and will continue for perhaps a decade or so • Ultimately creates the dark silicon dilemma • Which prevents all of the chip to be active at the same time 13

The Architects? • For decades, architects relied on technology scaling to provide for much of the generational design benefits • Can architects bridge the Moore’s law performance gap with innovative designs? • Perhaps, what provides the 10 x+ needed to bridge the gap? • Initially, many-core was thought to provide a scalable solution • Many-core designs provide value, but it is not a universal panacea • Architects have other tricks up their sleeves • Application-specific architectures, in particular 14

Example: Crypto. Maniac Processor [Austin] CM Proc • Circuit-level functional unit In Q Req Scheduler design tucks pre- and post. Boolean ops into clock cycle requests • Architecture-level ISA extension CM Proc Out Q results . . . CM Proc exposes pre/post-ops Keystore • Application-level programming re-expresses algorithms to leverage optimization Logical Unit XOR {long} • 20% performance benefit (could be recast as energy benefit) Pipelined 32 -Bit MUL 1 K Byte SBOX Cache AND 32 -Bit Adder Logical Unit XOR AND {tiny} 32 -Bit Rotator {short} {tiny} 15

Example: EFFEX Feature Extraction [Clemons, Austin] Typical computer vision pipeline Feature Extractio n Feature Extraction Imag e Preproce ss image Local Features, Scan Global image for Features potential feature locations Spatial. Temporal Relationshi ps Contextual Scene Reasoning Object Geometric & Semantic Reasoning Filter feature locations Generate signature of confirmed features Post process feature descriptor s Local Features, Global Features 16

Example: EFFEX Feature Extraction Vector Reduction Units Patch Memory Heterogeneous Multicore Initial EFFEX design 90 x greater efficiency for feature extraction algorithms 17

Moving Beyond Homogeneous Parallelism • Many-core designs, while helpful, will soon realize their (perhaps full) potential, what big ideas are next that can translate innovation into scaling • Heterogeneous parallelism is the key to 10 -100 x energy-efficiency gains in light of slowing silicon • C-FAR seeks sustained scaling through highly reusable composable customization, and affordable design and manufacturing 18

The Policy Makers? • Perhaps, while solutions exist, policy makers do not have the will to support research into solving this problem • Not the case, while all of the engineering sciences have been hit with funding cuts in the last decade • Computer engineering has faired better than many other engineering sciences • Investments in CE research are still flowing from industry and goverment • Example: Center for Future Architectures Research (C-FAR) • 27 PIs from 14 US universities • US$30 M investment over 5 years 19

C-FAR: Center for Future Architectures Research • Sponsored by SRC/DARPA Viability Themes • Goal: Innovation-based Storage (Steven Swanson) System Integration (Naveen Verma) Communication (Margaret Martonosi) Resilient System Design (Valeria Bertacco) Computation (David Brooks) Applications-to-Architectures (Krste Asanovic) Design Themes scaling for 2020 -2030’s What solutions are needed? Will our ideas actually work? 20

The Blame? Cost of Design • The one idea I want to leave with you today… • Successfully bridging the Moore’s Law Performance Gap is less about “How” to do it, and more about “How Much” does it cost to attempt to do it • My claim: if we can effect a 100 x reduction in the cost to bring a design to market, performance challenges will eventually solve themselves as the market flourishes with orders of magnitude more designs, some of which will be the big design wins of tomorrow 21

Don’t Believe Me? • Well, you can’t argue with Mother Nature! • r/K selection theory is a biological mechanism that organisms use to better adapt to their environment • In unstable environments, r-selection predominates as the ability to reproduce quickly is crucial • In stable environments, K-selection predominates as the ability to compete successfully for limited resources is crucial 22

Design Costs Are Skyrocketing 70 Mask Production Costs Software Development and Test Cost to Market ($ million) 60 Design, Layout, and Verification 50 40 30 20 10 0 0. 5 u 23 0. 35 u 0. 25 u 0. 18 u 0. 13 u 90 nm Silicon Technology Node 65 nm 45 nm Source: International Business Strategies 23

High Costs Kill Innovation • Heterogeneous designs often serve smaller markets 24

Outcome: “Nanodiversity” is Dwindling 12000 ASIC Design Starts 10000 8000 6000 4000 2009 2008 2007 2006 2005 2004 Year 2003 2002 2001 2000 1999 1998 1997 1996 1995 0 Source: Gartner Group 25 25

Silicon Today: The Good, the Bad and the Ugly • The Good: Moore’s law will continue for the near future • It won’t last forever, but that another problem • The Bad: Dennard scaling has all but stopped, leaving innovation to fill the performance/power scaling gap • E. g. , app-specific design, custom accelerators • The Ugly: Hardware innovation requires design diversity, which is ultimately too expensive to afford • Skyrocketing NREs will necessitate broadly applicable (vanilla and slow) H/W designs 26

The Remedy: Scale Innovation via Lower Design Cost • Ultimate goal: make customized design sufficiently inexpensive that anyone can do it anywhere • Address all NRE factors: market size, design costs, build costs • Take inspiration from Web 2. 0, and subsequent innovation explosion • Approach #1: Reduce the cost of custom hardware • With better tools that understand leverage the benefits of customization • By embracing open-source hardware design solutions • Approach #2: Widen the applicability of custom hardware • Increasing market applicability with composable customization mitigates potentially higher NREs • Approach #3: Reduce the cost of manufacturing hardware • Utilize assembly-time customization to slash the cost of customization • Approach #4: Slash software development costs • Through more effective reuse, wider adoption of open source technologies, and novel approaches to verification 27

1) Reduce the cost of custom hardware • Better tools • • • Scalable accelerator synthesis and compilation, generates code and H/W for highly reusable accelerators for a wide range of applications Composable design space exploration, composability permits novel search techniques based on machine learning, enables efficient exploration of highly complex design spaces Embrace open-source concepts • Mx. PA/FCUDA RTL Composed design space automatic composition Example: Berkeley’s RISC-V architecture CPU, GPU, Accelerator Design Spaces 28

Berkeley’s RISC V Open-Source ISA 29

2) Widen the Applicability of Custom H/W [Asanovic, Keutzer] Computer Vision Applications Computational Patterns Dense Machine Learning Multimedia Analysis Sparse … Graph Specializers with custom implementations and autotuning Glue Code Dense Code Sparse Code Graph Code ILP Engine Dense Engine Sparse Engine Graph Engine ESP Code ESP Core • ESP: Ensembles of Specialized Processors • Ensembles are algorithmic-specific processors optimized for code “patterns” • Patterns capture common operations across many applications, each with unique communication and computation structure • Approach has the promise of custom accelerator speed and efficiency that is widely applicable to general purpose programs 30

3) Reduce the cost of manufacturing H/W [Bertacco, Carloni, Mercaldi] • Brick-and-mortar silicon explores assembly-time customization, i. e. , MCMs + 3 D + FPGA interconnect H/W brick Brick-and-mortar silicon design flow: 1) Assemble brick layer 2) Connect with mortar layer 3) Package assembly 4) Deploy software • Diversity via brick ecosystem & interconnect flexibility • Brick design costs amortized across all designs • Robust interconnect and custom bricks rival ASIC speeds 31

4) Slash Software Development Costs • Implement more software reuse • Embrace more fully the open-source movement • Audience, other ideas? 32

Conclusions • Heterogeneous design could continue Moore’s law scaling through innovation alone • But, it requires a diverse hardware ecosystem with affordable customization • Affordable customization won’t happen without our help 1. Widen the applicability of customization 2. Reduce the cost of customized design 3. Reduce the cost of custom manufacturing • Resulting “nanodiversity” is a good thing • Better perf, power, cost, capability • More jobs, companies, and students • More competition and scalable innovation 33

Questions ? ? ? ? ?