Presentation to PEMWS2 Models without Borders Thomas Sterling

Presentation to: PEMWS-2 Models without Borders Thomas Sterling Arnaud & Edwards Professor, Department of Computer Science Adjunct Faculty, Department of Electrical and Computer Engineering Louisiana State University Distinguished Visiting Scientist, Oak Ridge National Laboratory CSRI Fellow, Sandia National Laboratory April 5, 2011 DEPARTMENT OF COMPUTER SCIENCE @ LOUISIANA STATE UNIVERSITY

The Status Quo DEPARTMENT OF COMPUTER SCIENCE @ LOUISIANA STATE UNIVERSITY 2

… or Maybe Not DEPARTMENT OF COMPUTER SCIENCE @ LOUISIANA STATE UNIVERSITY 3

The Challenge • Enable Extreme scale computing in this decade – – Exascale Strong scale Revolutionize STEM and national defense applications Provide practical, programmable, dependable systems • Respond to disruptive technology drivers – Multi/many core – Heterogeneous GPU – Flat-lined individual core performance • Guide future system codesign – Programming models – Operating system and runtime software stack – System and core architectures DEPARTMENT OF COMPUTER SCIENCE @ LOUISIANA STATE UNIVERSITY 4

The Strategy • Derive a new execution model – Unified to address all challenges – Single to build community wide adoption – Open to facilitate commercial products and application • Develop proof-of-concept reference implementations – Developing a diversity of software realizations and tools – Targeting a diversity of present and future architectures – Spin-off early products for HPC adoption • Empirical studies for quantitative evaluation – Driven by STEM and security related applications – Comprehensive for confidence in capability and reliability DEPARTMENT OF COMPUTER SCIENCE @ LOUISIANA STATE UNIVERSITY 5

Strategic Requirements • Performance – Efficiency – Scalability • Energy – Bounded power – Minimized energy • Reliability – Continued operation in the presence of faults • Programmability – System transparency – Portability across system classes, scales, and generations • Generality – STEM – Knowledge DEPARTMENT OF COMPUTER SCIENCE @ LOUISIANA STATE UNIVERSITY 6

Tactical Performance Requirements • Starvation – Insufficiency of concurrency of work – Impacts scalability and latency hiding – Effects programmability • Latency – Time measured distance for remote access and services – Impacts efficiency • Overhead – Critical time additional work to manage tasks & resources – Impacts efficiency and granularity for scalability • Waiting for contention resolution – Delays due to simultaneous access requests to shared physical or logical resources DEPARTMENT OF COMPUTER SCIENCE @ LOUISIANA STATE UNIVERSITY 7

PEMWS-2 Objectives • Inform and exchange – Share experiences and perspectives towards a common goal • Build a Paralle. X community – Bring together sources of complementing ideas – Identify stakeholders in government, industry, & academia – Define common needs and applications • Establish a path forward – Towards a single unified execution model – Responsive to needs of industry, national agenda & missions DEPARTMENT OF COMPUTER SCIENCE @ LOUISIANA STATE UNIVERSITY 8

Major Topics • • • Execution models as a service to the future Codelets Memory models Parcels for message-driven computation XPI – towards a low-level Paralle. X API SWARM – many core operation Habanero – parallel programming environments Self-aware declarative control HAD-HPX results Dialog and perspectives DEPARTMENT OF COMPUTER SCIENCE @ LOUISIANA STATE UNIVERSITY 9

A Quick Paralle. X Review • • • Synchronous Domains AGAS – Active Global Address Space Paralle. X Processes – with capabilities protection Computational Complexes – threads & fine grain dataflow Local Control Objects – synchronization and global distributed control state Distributed control operation – global mutable data structures Parcels – message-driven execution and continuation migration Percolation – heterogeneous control Micro-checkpointing – compute-validate-commit Self-aware – introspection and declarative management DEPARTMENT OF COMPUTER SCIENCE @ LOUISIANA STATE UNIVERSITY 10

Progress Report • Paralle. X execution model – Report version 1. 11 b • HPX-3 – Targeted to conventional platforms – Missing: processes, scalable AGAS • Applications – – Adaptive Mesh Refinement N-body Barnes Hut PXGL Linpack • XPI DEPARTMENT OF COMPUTER SCIENCE @ LOUISIANA STATE UNIVERSITY 11

Models without Borders (1) • Institutional – Not owned or claimed by any one closed organization – Not proprietary • Open source – Accessible by all academic researchers – Basis for commercial deployment and deliver vehicle • Target platform – Portable across classes, scales, and generations – Exploits unique strengths while not limited to them • Applications – Returns parallel processing to broad range of problems – Array and graph, static and dynamic, numeric and symbolic – Data scaled and strong scaled DEPARTMENT OF COMPUTER SCIENCE @ LOUISIANA STATE UNIVERSITY 12

Models without Borders (2) • Inter-agency – Shared responsibility and credit for development • International – Recognized across continents – World-wide market for compliant products • Challenge set – Robust under varied technology variations – Resilient in performance across diverse structures • Convention – Not limited by status quo – Vested interests in restricting change DEPARTMENT OF COMPUTER SCIENCE @ LOUISIANA STATE UNIVERSITY 13

Models without Borders – Requirements • • • Federation of Interested Organizations Charter and Mission of goals and process Stable funding Metrics of success Paradigm enabled – challenge driven VMG – velocity made good – Measured progress • Reference implementation – open to all, Contributed by many – Deliverable schedule of usable components – Modular architecture for distributed development DEPARTMENT OF COMPUTER SCIENCE @ LOUISIANA STATE UNIVERSITY 14

Conclusions • HPC has a narrow window of opportunity as it gropes for new balance point in the face of technology challenges • As before a change in execution model is essential for useful progress toward capable scalable systems • Paralle. X is a strong contender for this new model • Paralle. X provides a foundation for development of a consistent and comprehensive model • Intel/UDel bring needed expertise and concepts for achieving enhancements critical to Paralle. X success • Carpe Diem DEPARTMENT OF COMPUTER SCIENCE @ LOUISIANA STATE UNIVERSITY 15