SCEC Community Modeling Environment SCECCME Philip J Maechling

SCEC Community Modeling Environment (SCEC/CME) Philip J. Maechling SCEC 4 Site Review June 21, 2010

SCEC 3 Organization SCEC Director Board of Directors External Advisory Council Information Architect Planning Committee Center Administration CEO Program CME Peta. SHA Earthquake Geology Unified Structural Representation Seismic Hazard & Risk Analysis WGCEP Tectonic Geodesy Fault & Rupture Mechanics Knowledge Transfer CSEP Seismology Crustal Deformation Modeling Public Outreach So. SAFE Lithospheric Architecture & Dynamics K-12 & Informal Education Ex. GM Earthquake Forecasting & Predictability Experiential Learning & Career Advancement Special Projects Disciplinary Committees Ground Motion Prediction Focus Groups CEO Activities

Seismic Hazard Analysis Requires Integration of Computational and Structural Models Seismicity Paleoseismology Local site effects Geologic structure Faults Seismic Hazard Model Stress transfer Rupture dynamics Crustal motion Crustal deformation Seismic velocity structure

Overview SCEC/CME Computational Research SCEC/CME HPC Collaborative Research SCEC/CME Forecast Testing and Evaluation

SCEC/CME Scientific Research Objectives (1) Improve the resolution of dynamic rupture simulations by an order of magnitude to investigate realistic friction laws, near-fault stress states, and off-fault plasticity. (2) Investigate the upper frequency limit of deterministic ground-motion prediction by simulating strong motions above 3 Hz using realistic 3 D structural models for Southern California. (3) Validate and improve the Southern California structural models using full 3 D waveform tomography. (4) Transform probabilistic seismic hazard analysis (PSHA) into a physics -based science.

Community Modeling Environment (CME) SCEC/CME is a collaboration of SCEC scientists and computer scientists that performs computationally-intensive seismic hazard research. Current SCEC/CME NSF research awards include: Peta. SHA (NSF/EAR Geoinformatics) Peta. Shake (NSF/OCI Peta. Apps) SCEC Blue Waters (NSF/OCI PRAC) SCEC/CME projects include: CSEP (W. M. Keck Foundation) CISN Earthquake Early Warning Testing (USGS NEHRP) SCEC Broadband Platform (Pacific Gas and Electric)

SCEC Computational Research Experience SCEC/CME experience is that it requires more than a single program to get a significant computational research result. SCEC/CME research computing typically requires: – – – Carefully prepared input data Multiple simulation codes Verification and validation problems and solutions Scientific and software expertise Software and computer infrastructure

SCEC Computational Platform Concept • Definition of Computational Platform – A vertically integrated collection of hardware, software, and people that can repeatedly produce a useful research result • Computational Platform Characteristics – Validated simulation software and geophysical models – Re-usable simulation capabilities – Imports parameters from other systems. Exports results to other systems – IT/geoscience collaboration involved in operation – Access to High-performance hardware and large scale data and metadata management. – May use Workflow management tools

CME Project Funding

Overview SCEC/CME Computational Research SCEC/CME HPC Collaborative Research SCEC/CME Forecast Testing and Evaluation

SCEC/CME High Performance Computing (HPC) Research SCEC/CME seeks to improve predictive ground motion forecast models by continuing to integrate more realistic physics into the numerical simulations. – Increasing seismic hazard model complexity and accuracy increases computing requirements – As simulations become more physically realistic, the computational and data management requirements increase enormously

SCEC Pursuing Leadership Class Computer Systems HPC Centers 10’s of 10 TF Systems 1, 000’s of Users 100’s of 1 TF Systems Departmental HPC 10, 000’s of Users Workstations Giga. FLOPS Millions of Users Scientific Computing SCEC Data (more BYTES) 100 TF Systems 10’s of Projects Data-oriented SCEC Science Key function of the and Engineering NSF Supercomputer Environment Centers: Provide facilities over and above what can be found in the typical campus/lab Home, Lab, environment Traditional Campus, HPC Desktop environment Compute (more FLOPS) SCEC: An NSF + USGS Research Center

SCEC, USC HPCC, Tera. Grid resources used for SCEC Research USC HPCC Resources Grid SCEC Computing Resources Tera. Grid Computing Resources SCEC: An NSF + USGS Research Center

SCEC: An NSF + USGS Research Center

HPC Resources Utilized by SCEC Resource Provider Site Est. 2010 SU (Million Hours) Est. Storage (TB) USC – HPCC USC 1 70 NSF - Tera. Grid NICS 35 150 PSC 2 50 TACC 25 200 NCSA 15 150 SDSC 4 300 Argonne 10 200 Oak Ridge 25 200 NSF-Blue Waters NCSA 0 0 Open Science Grid Argonne 1 1 118 M 1, 321 TB DOE-INCITE Totals

Computer Scientist Resources Provided to SCEC by HPC Organizations SDSC Strategic Application Collaborations (SAC) - 1. 0 FTE x 3 yrs MPI code Improvements, Simulation Results Archive - 0. 5 FTE x 2 yrs Scientific Visualization Advanced Support for Tera. Grid Applications (ASTA) - 1. 0 FTE x 3 yrs Simulation support and optimizations Summary: HPC Resource Providers have provided more than 7 person-years of computer science support for SCEC/CME research during SCEC 3

SCEC HPC Computer Allocations SCEC Allocation on NSF Tera. Grid, DOE INCITE and USC HPCC Resources 100 000 Total Service Units 90 000 Service Units on NSF Tera. Grid Resources 80 000 Service Units on DOE INCITE Resources 70 000 Service Units on USC Resources 60 000 50 000 40 000 30 000 20 000 10 000 0 2002 2003 2004 2005 2006 2007 CME Allocation Year 2008 2009 2010*

SCEC MPI Code Peak Performance CME AWP-ODC Sustained Performance Achieved on Tera. Grid and DOE Supercomputers CME Allocation Year 2010 2009 2008 2007 2006 2005 2004 2003 2002 0 50 100 150 Sustained Tera. Flop/s 200 250

Petascale Research in Earthquake System Science on Blue Waters PI: Thomas H. Jordan - University of Southern California Co-PI: Jacobo Bielak - Carnegie Mellon University Southern California Earthquake Center (SCEC)

Overview SCEC/CME Computational Research SCEC/CME HPC Collaborative Research SCEC/CME Forecast Testing and Evaluation

USGS hazard mapping results in dramatic change in building codes Seismic element of 1996 Building Codes based on 1970’s maps Seismic element of 2000 & 2003 Int’l Building Code based on the 1996 USGS national seismic hazard map

Forecast Testing Should Increase Along with Forecast Impact SCEC Computational Forecast Users Scientific and Engineering Requirements for Forecast Modeling Systems Public and Governmental Forecasts Automated prospective testing of forecast models over time within collaborative forecast testing center. Engineering and Interdisciplinary Research Automated retrospective testing of forecast models using community defined validation problems. Collaborative Research Project Computational codes, structural models, and simulation results versioned with associated tests. Individual Research Project Development of new computational, data, and physical models.

CISN EEW Algorithm Performance Testing Developments CISN EEW Performance Testing Developments: 1. Implemented the Location/Magnitude oriented performance Summaries. 2. Generating nightly performance summaries (2 mo, 4 mo, 6 mo and cumulative time periods). 3. Generating Summaries for Elarm. S (UCB), Tau. C (Caltech), and Virtual Seismologist ( ETH) CISN EEW Testing site available at: http: //www. scec. org/eew Magnitude Accuracy Actual Warning Delay with Current CISN Network Theoretical Warning Delay with Zero Telemetry and Processing Delays

SCEC/CME Web Site: http: //ww. scec. org/cme End