Southern California Earthquake Center Cyber Shake Progress Update
Southern California Earthquake Center Cyber. Shake Progress Update 3 November 2014 through 4 May 2015 UGMS May 2015 Meeting Philip Maechling SCEC IT Architect 4 May 2015
November 2014 UGMS Meeting Southern California Earthquake Center • Reviewed Cyber. Shake 14. 2 hazard model based on: • • • UCERF 2 No Background Seismicity Graves and Pitarka Rupture Generator: genslip v 3. 2 3 D velocity model: CVM-S 4. 26 Min Vs : 500 m/s Max Cyber. Shake Frequency : 0. 5 Hz • Results Reviewed • PSA 3. 0, PSA 5. 0, PSA 10. 0 hazard curves and maps for 286 Sites • Rot. D 100 3. 0 s, 5. 0 s, 10 s hazard curves for 14 sites • MCER Probabilistic, Deterministic, Combined, Overall Curves for 14 Sites In Los Angeles Region • http: //scec. usc. edu/scecpedia/Cyber. Shake_MCER
Cyber. Shake Study 14. 2 Hazard Map Southern California Earthquake Center
November 2014 UGMS MCER Sites Southern California Earthquake Center
Southern California Earthquake Center Cyber. Shake 14. 2 versus Cyber. Shake 15. 4 • Cyber. Shake Study 14. 2 • • • UCERF 2 No Background Seismicity 3 D Velocity Model: CVM-S 4. 26 Min Vs: 500 m/s Velocity Meshing: 200 m Fault Meshing: 500 m Rupture Generator: genslip v 3. 2 Maximum Frequency: 0. 5 Hz PSHA 3. 0 s, 5. 0 s, 10. 0 s curves Rot. D 100 3. 0 s, 5. 0 s, 10. 0 s curves
Southern California Earthquake Center Cyber. Shake Platform: Physics-Based PSHA
Southern California Earthquake Center Cyber. Shake 14. 2 Probabilistic Hazard Model Products CS 14. 2 PSA 3. 0 Hazard Model (286 sites) CS 14. 2 Rot. D 100 3. 0 s Probabilistic Hazard Curve (LADT) GMPE Comparison Maps Ratio (CS 14. 2/ NGA-2) Rot. D 100 3. 0 s (286 Sites) CS 14. 2 Rot. D 100 3. 0 s Probabilistic, Deterministic, Combined, Overall MCER Curves (LADT) CS 14. 2 Rot. D 100 3. 0 s Probabilistic, Deterministic Lower Limit, Combined MCER Contour Maps (286 Sites)
Southern California Earthquake Center Cyber. Shake 14. 2 Rot. D 100 3 s Probabilistic Hazard Curve (LADT)
Southern California Earthquake Center Cyber. Shake 14. 2 Probabilistic MCER Curve (LADT)
Southern California Earthquake Center Cyber. Shake 14. 2 Deterministic MCER Curve (LADT)
Southern California Earthquake Center Cyber. Shake 14. 2 Combined MCER Result (LADT)
Southern California Earthquake Center Cyber. Shake 14. 2 Overall MCER Result (LADT)
Southern California Earthquake Center Cyber. Shake 14. 2 Probabilistic MCER 3 s Rot. D 100 (286 Sites)
Southern California Earthquake Center Cyber. Shake 14. 2 Deterministic MCER 3 s Rot. D 100 (286 Sites)
Southern California Earthquake Center Cyber. Shake 14. 2 Deterministic Lower Limit MCER 3 s Rot. D 100 (286 Sites)
Southern California Earthquake Center Cyber. Shake 14. 2 Combined MCER Results 3 s Rot. D 100 (286 Sites)
Southern California Earthquake Center GMPE Comparison Maps Cyber. Shake Combined MCER 3 s Rot. D 100 (286 Sites)
Southern California Earthquake Center GMPE Comparison Map GMPE Combined MCER 3 s Rot. D 100 (286 Sites)
Southern California Earthquake Center GMPE Comparison Map Ratio (Cyber. Shake 14. 2 / NGA-W 2) 3 s Rot. D 100 (286 Sites)
Southern California Earthquake Center New Cyber. Shake Computational Results May 2015 • Rot. D 100 3 S, 5 S, 10 S amplitudes calculated for all 286 Cyber. Shake 14. 2 sites • MCER Rot. D 100 3 S, 5 S, 10 S Contour Maps using 286 Cyber. Shake 14. 2 sites • Comparison Maps Cyber. Shake MCER 14. 2 versus GMPE MCER NGA-2 • Started Cyber. Shake 15. 4 Study to calculate 1 Hz CS results, including products described above, for 336 sites in Los Angeles Region • Preliminary Cyber. Shake 15. 4 results for 14 sites
Southern California Earthquake Center Cyber. Shake 14. 2 versus Cyber. Shake 15. 4 • Cyber. Shake Study 15. 4 • • • UCERF 2 No Background Seismicity 3 D Velocity Model: CVM-S 4. 26 Min Vs: 500 m/s Velocity Meshing: 100 m Fault Meshing: 200 m Rupture Generator: genslip v 3. 3. 1 Maximum Frequency: 1. 0 Hz PSHA 2. 0 s, 3. 0 s, 5. 0 s, 10. 0 s curves Rot. D 100 2. 0 s, 3. 0 s, 5. 0 s, 10. 0 s curves
Cyber. Shake Study 15. 4 sites (336) Southern California Earthquake Center 336 sites (10 km mesh + points of interest + “gap” sites) Green sites are the 50 new “gap” sites (Run 14 UGMS sites first )
Southern California Earthquake Center New Computational Results For May 2015 • Initiate Cyber. Shake 15. 4 Study using NSF and DOE Computers • Los Angeles region Hazard Model based on 336 sites at 1 Hz • Estimated 40 M Computer Hours • Estimated 500 TB+ temporary data at DOE, NSF Computers • Estimated 11 TB persistent data at SCEC
Southern California Earthquake Center Study 15. 4 Parameters • 1. 0 Hz deterministic – 100 m spacing – dt=0. 005 sec – nt=40000 timesteps • CVM-S 4. 26 – Vs min = 500 m/s • UCERF 2 • Graves & Pitarka (2014) rupture variations – 200 m rupture grid point spacing • Source filtered at 2. 0 Hz
Southern California Earthquake Center Higher Resolution Simulations Required Change in Rupture Generator
Southern California Earthquake Center Expected Study 15. 4 Data Products • CVM-S 4. 26 Los Angeles-area hazard maps • Rot. D 100 2, 3, 4, 5, 7. 5, 10 sec • Rot. D 50 2, 3, 4, 5, 7. 5, 10 sec • Geometric mean 2, 3, 5, 10 sec • • Hazard curves for 286 sites, at 2 s, 3 s, 5 s, 10 s 336 sets of 2 -component SGTs Seismograms for all ruptures (~160 M) Peak amplitudes in DB for 2 s, 3 s, 5 s, 10 s • Rot. D 100, Rot. D 50 and geometric mean SA
Southern California Earthquake Center Estimated Duration • Limiting factors: – XK node queue time • 800 XK nodes is 19% of Blue Waters – Titan -> Blue Waters • If throughput is very high, transfer could be bottleneck – USC HPC downtime for ~1 week in April • Estimated completion is 12 weeks (11 running + 1 downtime) – Based on same node availability as Study 14. 2 • Planning to request reservation on Blue Waters • Planning to request high priority on Titan
Southern California Earthquake Center Discussion Objectives • Review of Cyber. Shake 14. 2 New Computational Data Products • Preliminary Analysis of Cyber. Shake 15. 4 results for 14 Sites
Southern California Earthquake Center End
Southern California Earthquake Center Rupture Generator changes • Previous number of realizations related to fault length • # of realizations = max(10, C * Area/10. 0) – C = 0. 5 • Each realization is unique slip + hypocenter location • Supports either random or uniform hypocenter distribution
Southern California Earthquake Center Rupture Generator v 3. 3. 1 • Use of new G&P rupture generator (v 3. 3. 1) brought 1000 m and 200 m curves into agreement 0. 5 Hz UCERF 2 3 sec SA CVM-S 4 TEST site, black = 200 m, magenta = 1000 m
Southern California Earthquake Center Random vs Uniform Hypocenters • Variation counts – G&P 2010: 423 k – Uniform: 485 k – Random: 542 k • Uniform easier to interpolate 0. 5 Hz UCERF 2 3 sec SA CVMS 4. 26 WNGC site: black=random, magenta=uniform
Southern California Earthquake Center Storage Requirements • Titan – Purged: 526 TB (for SGTs and temp data) • Blue Waters – Delayed purge: 506 TB (for Titan SGTs) – Purged: 526 TB SGTs + 9 TB data products • SCEC – – Archived: 9. 1 TB (seismograms, PSA, Rot. D) Database: 268 GB (Geom @ 4 periods, Rot. D @ 6) Temporary: 608 GB (workflow logs) Shared SCEC disks have 171 TB free
Southern California Earthquake Center Computational Requirements • Per site: ~3720 node-hrs – SGTs: depends on execution site (~50%) • Titan = 2110 node-hrs / 63, 300 SUs • Blue Waters = 1760 node-hrs / 30, 200 SUs • More expensive for Titan because of padding in pilot jobs and different node-hrs -> SU conversion – PP: 1880 node-hrs / 60, 200 SUs (~50%) • Computational time: – Titan (SGTs): 355 K node-hours / 10. 7 M SUs – Blue Waters: 928 K node-hours • SGTs: 275 K GPU node-hrs, 21 K CPU node-hrs • PP: 632 K CPU node-hrs • Titan has 104 M SUs remaining • Blue Waters has 5. 3 M node-hrs remaining
Southern California Earthquake Center End
- Slides: 35