ADVANCED RESEARCH IN PROCESSOR MEMORY DESIGN CS 6501

ADVANCED RESEARCH IN PROCESSOR & MEMORY DESIGN CS 6501 Samira Khan University of Virginia August 23, 2016 The content and concept of this lecture is adapted from CMU ECE 740

ROTUNDA, PAVILLIONS AND THE LAWN 2

A DESIGN BASED ON PRINCIPLES 1856 engraving • Designed by Thomas Jefferson • Unique design at that time (1819) • University surrounded by a library • Separation of church and education • Students and professors living in the pavilions • A holistic approach to training and education 3

A KEY QUESTION • How was Jefferson able to design a school that was so different from precedents? • Can have many guesses – Visionary – Passion for architecture and education – (Ultra) hard work, perseverance, dedication (over decades) – Experience of decades – Creativity – Out-of-the-box thinking – Principled design – A good understanding of past designs – Good judgment and intuition – Strong combination of skills (architecture, art, law, mathematics, horticulture, philosophy …) –… • (You will be exposed to and hopefully develop/enhance many of these skills in this course) 4

MAJOR HIGH-LEVEL GOALS OF THIS COURSE • Understand the principles • Understand the precedents • Based on such understanding: – Enable you to evaluate tradeoffs of different designs and ideas – Enable you to develop principled designs – Enable you to develop novel, out-of-the-box designs • The focus is on: – Principles, precedents, and how to use them for new designs • In Computer Architecture 5

ROLE OF THE (COMPUTER) ARCHITECT from Yale Patt’s lecture notes 6

ROLE OF THE (COMPUTER) ARCHITECT • Look backward (to the past) – Understand tradeoffs and designs, upsides/downsides, past workloads. Analyze and evaluate the past • Look forward (to the future) – Be the dreamer and create new designs. Listen to dreamers – Push the state of the art. Evaluate new design choices • Look up (towards problems in the computing stack) – Understand important problems and their nature – Develop architectures and ideas to solve important problems • Look down (towards device/circuit technology) – Understand the capabilities of the underlying technology – Predict and adapt to the future of technology (you are designing for N years ahead). Enable the future technology 7

TAKEAWAYS • Being an architect is not easy • You need to consider many things in designing a new system + have good intuition/insight into ideas/tradeoffs • But, it is fun and can be very technically rewarding • And, enables a great future – E. g. , many scientific and everyday-life innovations would not have been possible without architectural innovation that enabled very high performance systems – E. g. , your mobile phones • This course will teach you how to become a good computer architect 8

INSTRUCTOR • Samira Khan – Assistant Professor – samirakhan@virginia. edu • Post Doctorate in CMU • Ph. D from UT San Antonio • Worked at Intel, AMD, EPFL • Research Interest – Computer Architecture and Systems – Memory System Design – New Emerging Technologies – Hardware-Software Co-Design – Performance, Power, Reliability 9

AGENDA • Course Overview • Example Research Problems • How to Jump Into Research 10

WHERE TO GET UP-TO-DATE COURSE INFO? • Website • Piazza • Your email: start the subject line with CS 6501 • Office Hours 11

WHAT IS COMPUTER ARCHITECTURE • Computer Architecture: The science and art of designing, selecting, and interconnecting hardware components and designing the hardware/software interface to create a computing system that meets functional, performance, energy consumption, cost, and other specific goals. • Traditional definition: “The term architecture is used here to describe the attributes of a system as seen by the programmer, i. e. , the conceptual structure and functional behavior as distinct from the organization of the dataflow and controls, the logic design, and the physical implementation. ” Gene Amdahl, IBM Journal of R&D, April 1964 12

LEVELS OF TRANSFORMATION Problem Operating System Interface between Architecture Software and Hardware Circuits 13

THE POWER OF ABSTRACTION • Isolation • A higher level only needs to know about the interface to the lower level, not how the lower level is implemented • For example, high-level language programmer does not really need to know about the architecture • Productivity • No need to worry about decisions made in underlying levels • For example, programming in Java vs. C vs. assembly vs. binary vs. by specifying control signals of each transistor every cycle 14

CROSSING THE ABSTRACTION LAYERS • Should we always focus on our own layer? • As long as everything goes well, not knowing what happens in the underlying level (or above) is not a problem • What if – One of the layers reach a limit, there is no way to improve – There is a new disruptive change in technology that cannot be contained in a layer – New Applications that are too slow for today’s system 15

LEVELS OF TRANSFORMATION Problem Broader Scope Operating of System Computer Architecture Circuits 16

SCOPE OF THE COURSE • This course will take a broad view of processor and memory design – Beyond the ISA+microarchitecture levels – E. g. , system-architecture interfaces and interactions – E. g. , application-architecture interfaces and interactions • Out-of-the-box thinking is greatly encouraged – E. g. , research projects and readings on architectures that challenge the current dominant paradigms • processing in memory, approximate systems, persistent memory, neuromorphic computing, … – E. g. , readings on topics that are traditionally covered less in computer architecture courses 17

WHAT WILL YOU LEARN? • Hardware/software interface, major components, and programming models of a modern computing platform – State-of-the-art as well as research proposals (lots of them) – Tradeoffs and how to make them – Emphasis on cutting-edge (research & state-of-the-art) • Hands-on research in a computer architecture topic – Semester-long research project – Focus: How to design better architectures (not an intro course) • Broaden your research vision – Will read papers not only from computer architecture – But, from circuits, programming language, operating systems, graphics, networking, … 18

COURSE GOALS • Goal 1: Rethink and redesign our computing model focusing on minimizing data movement and storage. – Understand the importance of cross layer research – Gain new insight from VLSI circuits, architecture design, systems, programming languages, and new application domains – Strong emphasis on • Critical analysis of research papers (through reading and literature review assignments) • Goal 2: To provide the necessary background and experience to advance the state-of-the-art in computer architecture by performing cutting-edge research – Strong emphasis on • Developing new mechanisms that advance the state of the art (through the course research project). 19

THIS IS A GRADUATE-LEVEL CLASS • Required background: – basic architecture – basic compilers – basic OS – programming skills – spirit, excitement, and dedication for deep exploration of a topic in computer architecture 20

WHAT DO I EXPECT FROM YOU? • Work hard • Ask questions, think critically, participate in discussion • Critically review the assigned research papers & readings – Discuss/critique them online with peers and us • Use Piazza and Review Site frequently… • Start the research project early and focus • Remember “Chance favors the prepared mind. ” (Pasteur) 21

HOW WILL YOU BE EVALUATED? • • Class Participation 10% Reviews 25% Class Presentations 30% Project 35% • Participation+discussion is very important • Grading will be back-end heavy. Most of your grade will be determined late. How you prepare and manage your time is important – But grades should not be the reason for taking this course 22

HOW TO DO THE PAPER REVIEWS • 1: Brief summary – – What is the problem the paper is trying to solve? What are the key ideas of the paper? Key insights? What is the key contribution to literature at the time it was written? What are the most important things you take out from it? • 2: Strengths (most important ones) – Does the paper solve the problem well? • 3: Weaknesses (most important ones) – This is where you should think critically. Every paper/idea has a weakness. This does not mean the paper is necessarily bad. It means there is room for improvement and future research can accomplish this. • 4: Can you do (much) better? Present your thoughts/ideas. • 5: What have you learned/enjoyed/disliked in the paper? Why? • Review should be short and concise (~half a page to a page) 23

ADVICE ON PAPER REVIEWS • When doing the reviews, be very critical • Always think about better ways of solving the problem or related problems • Do background reading – Reviewing a paper/talk is the best way of learning about a research problem/topic • Think about forming a literature survey topic or a research proposal based on the paper (for future studies) 24

PRESENTATION • Summary of the Work – Problem, key ideas or insights, detailed mechanisms, and results. • Strengths and Weaknesses – Detailed discussion on both sides • Discussion – Future research directions – Alternative ways of solving the problem – Importance of the problem – Anything interesting about the research direction – What did you like? What did you not like? 25

PRESENTATION • Each class two paper presentations • One main paper from the list (Review required) • One related paper (Review encouraged) – From the list or – From the reference list or – You can pick, but needs to be approved • Sign up for the papers and slots by Friday – Read the abstracts, think about the topic – Select papers/topics you find interesting • Can select you own topic, but needs to be approved 26

RESEARCH PROJECT • More information to come… In the meantime: • Read a lot of papers; find focused problem areas to survey papers on • We will provide a list of project ideas and papers associated with them • A good way of finding topics to survey or do projects on is: – Examining the provided project ideas and papers – Reading assigned papers in lectures – Examining papers from recent conferences (ISCA, MICRO, HPCA, ASPLOS, …) 27

RESEARCH PROJECT • Your chance to explore in depth a computer architecture topic that interests you • Perhaps even publish your innovation in a top computer architecture conference. • Start thinking about your project topic from now! • Interact with me • Read the project topics handout well • Proposal due: Sep 15, 2016 28

AGENDA • Course Overview • Example Research Problems • How to Jump Into Research 29

TWO EXAMPLES New Challenge: DRAM Scaling Problem Emerging Technology: Non-Volatile Memory 30

DRAM SCALING CHALLENGE Technology Scaling DRAM Cells DRAM scaling enabled high capacity 31

MEGABITS/CHIP DRAM SCALING TREND 10000 1000 2 X/1. 5 YEARS 2 X/3 YEARS 100 10 1 1985 1995 2005 2015 START OF MASS PRODUCTION DRAM scaling is getting difficult Source: Flash Memory Summit 2013, Memcon 2014 32

DRAM SCALING CHALLENGE Technology Scaling DRAM Cells Manufacturing reliable cells at low cost is getting difficult 33

TRADITIONAL APPROACH TO ENABLE DRAM SCALING Make DRAM Reliable Unreliable DRAM Cells Reliable DRAM Cells Manufacturing Time Reliable System in the Field DRAM has strict reliability guarantee 34

NEW APPROACH Make DRAM Reliable Unreliable DRAM Cells Reliable DRAM Cells Manufacturing Time Reliable System in the Field Shift the responsibility to systems 35

VISION: SYSTEM-LEVEL DETECTION AND MITIGATION Detect and Mitigate Unreliable DRAM Cells Reliable System Detect and mitigate errors after the system has become operational ONLINE PROFILING Reduces cost, increases yield, and enables scaling 36

TOWARDS AN ONLINE PROFILING SYSTEM Initially Protect DRAM 1 with Strong ECC Test Periodically Test Parts of DRAM 2 ECC Mitigate errors and reduce ECC 3 Run tests periodically after a short interval at smaller regions of memory 37

BREAKING THE ABSTRACTION Problem OS needs to know about testing and tested pages Need to implement the testing in the hardware Need to know the circuit-level characteristics of the failures Operating System Architecture Circuits • Samira Khan+, "The Efficacy of Error Mitigation Techniques for DRAM Retention Failures: A Comparative Experimental Study”, SIGMETRICS 2014 38

TWO EXAMPLES New Challenge: DRAM Scaling Problem Emerging Technology: Non-Volatile Memory 39

STORAGE MEMORY CPU TWO-LEVEL STORAGE MODEL Ld/St DRAM FILE I/O VOLATILE FAST BYTE ADDR NONVOLATILE SLOW BLOCK ADDR 40

STORAGE MEMORY CPU TWO-LEVEL STORAGE MODEL Ld/St DRAM FILE I/O NVM PCM, STT-RAM VOLATILE FAST BYTE ADDR NONVOLATILE SLOW BLOCK ADDR Non-volatile memories combine characteristics of memory and storage 41

VISION: UNIFY MEMORY AND STORAGE CPU NVM PERSISTENT MEMORY Ld/St Provides an opportunity to manipulate persistent data directly 42

MEMORY DRAM NVM PERSISTENT MEMORY Ld/St CPU DRAM IS STILL FASTER A hybrid unified memory-storage system 43

UNIFY MEMORY AND STORAGE • Opportunity to update data in-place in memory with Ld/St interface • Do not need to move data from disk to memory, translate file to data structure and transfer to disk again – Eliminates wasted work to locate, transfer, and translate data – Improves both energy and performance – Simplifies programming model as well 44

BREAKING THE ABSTRACTION Problem Applications manipulate persistent data directly Who provides the consistency and persistency guarantee? How does hardware manage the hybrid memory? Need to know the circuit-level characteristics of the underlying technology Operating System Architecture Circuits • Jinglei Ren+, “Dual-Scheme Checkpointing: A Software-Transparent Mechanism for Supporting Crash Consistency in Persistent Memory Systems”, MICRO 2015 • Justin Meza+, “A Case for Efficient Hardware-Software Cooperative Management of Storage and Memory”, WEED 2013 45

AGENDA • Course Overview • Example Research Problems • How to Jump Into Research 46

HOW TO DO RESEARCH AND ADVANCED DEV • We will talk a lot about this in this course • Learning by example – Reading and evaluating strong and seminal papers • Learning by doing – Semester-long research project • Learning by open, critical discussions – Recitation sessions, online discussion of papers & ideas on Piazza and the Paper Review Site 47

WHAT IS THE GOAL OF RESEARCH? • To generate new insight – that can enable what previously did not exist • Research (in engineering) is a hunt for insight that can eventually impact the world 48

SOME BASIC ADVICE FOR GOOD RESEARCH • Choose great problems to solve: Have great taste – Difficult – Important – High impact • Read heavily and critically • Think big (out of the box) – Do not restrain yourself to tweaks • Aim high • Write and present really well 49

Looking here for lost keys 50

Lost keys here Looking here 51

Current Architecture Practice 52

53

Aim Here 5 -10 year s 54

Enable this point 5 -10 year s 55

The Research Formula 56

Reward If you are wildly successful, what difference will it make? 57

Effort Learn as much as possible with as little work as possible 58

Effort Do the minimum analysis and experimentation necessary to make a point 59

Research is a hunt for insight Need to get off the beaten path to find new insights 60

RECOMMENDED TALK • Bill Dally, Moving the needle: Effective Computer Architecture Research in Academy and Industry ISCA 2010 Keynote Talk. • Acknowledgment: Past few slides are from this talk 61

MORE GOOD ADVICE “The purpose of computing is insight, not numbers” Richard Hamming 62

ADVANCED RESEARCH IN PROCESSOR & MEMORY DESIGN CS 6501 Samira Khan University of Virginia August 23, 2016 The content and concept of this lecture is adapted from CMU ECE 740