Climate Modeling as a Service ClimateHome with Existing

  • Slides: 26
Download presentation
Climate Modeling as a Service Climate@Home with Existing Infrastructure NASA IT Summit 2010 Steve

Climate Modeling as a Service Climate@Home with Existing Infrastructure NASA IT Summit 2010 Steve Armentrout, Parabon Tsengdar Lee, NASA HQ © 2008 Parabon Inc. All rights reserved. | 10 Parabon © 2009 Computation, Inc. All rights reserved. | 2010 1

Background and Participants • SBIR: Modeling and Simulation as a Service – – Tsengdar

Background and Participants • SBIR: Modeling and Simulation as a Service – – Tsengdar Lee Mike Seablom • Frontier pilot project on Nebula cloud – Chris Kemp • Climate@Home – NASA SIVO – Northrop Grumman – Parabon Core Application Developer Domain Programmer M&Saa. S QA Tester End User © 2010 Parabon Computation, Inc.

Motivations • Climate Change is our Copernican Controversy • Climate modeling is under intense

Motivations • Climate Change is our Copernican Controversy • Climate modeling is under intense scrutiny • Model validation and transparency are needed • Sensitivity experiments could be large and costly • An “@home” style project could: – – Advance Earth system science Engage the public © 2010 Parabon Computation, Inc.

Problem Specifications and Dimensions • NASA Model E Climate Model • Each experiment: Monthly

Problem Specifications and Dimensions • NASA Model E Climate Model • Each experiment: Monthly diagnostics for an ensemble of 1, 000 sims each spanning 100 simulated years = 1. 2 M outputs • Output files 20 -200 MB => 240 TB per experiment • Storage requirements > 1 PB • 1 simulated month ~ 1 hour of processing => © 2010 Parabon Computation, Inc.

Problem Specifications and Dimensions Climate@Home or Dont-Try-This@Home © 2010 Parabon Computation, Inc.

Problem Specifications and Dimensions Climate@Home or Dont-Try-This@Home © 2010 Parabon Computation, Inc.

Requirements • Client must be unobtrusive • Client must support Windows, Mac and Linux

Requirements • Client must be unobtrusive • Client must support Windows, Mac and Linux • Linux required for Model E execution • Results must be bit-for-bit reproducible © 2010 Parabon Computation, Inc.

Requirements • Client must be unobtrusive • Client must support Windows, Mac and Linux

Requirements • Client must be unobtrusive • Client must support Windows, Mac and Linux • Linux required for Model E execution • Results must be bit-for-bit reproducible • Solution: VM Management © 2010 Parabon Computation, Inc.

Climate@Home Grid Architecture Frontier Grid Server on Nebula Cloud NEBULA E NASA Ames NASA

Climate@Home Grid Architecture Frontier Grid Server on Nebula Cloud NEBULA E NASA Ames NASA Goddard General Public Frontier Data Server on Amazon S 3 © 2010 Parabon Computation, Inc.

Client © 2010 Parabon Computation, Inc.

Client © 2010 Parabon Computation, Inc.

Client © 2010 Parabon Computation, Inc.

Client © 2010 Parabon Computation, Inc.

Client © 2010 Parabon Computation, Inc.

Client © 2010 Parabon Computation, Inc.

Client © 2010 Parabon Computation, Inc.

Client © 2010 Parabon Computation, Inc.

Client © 2010 Parabon Computation, Inc.

Client © 2010 Parabon Computation, Inc.

Client Ryan Boller, NASA Sciences and Exploration Directorate © 2010 Parabon Computation, Inc.

Client Ryan Boller, NASA Sciences and Exploration Directorate © 2010 Parabon Computation, Inc.

Problem Decomposition t … © 2010 Parabon Computation, Inc.

Problem Decomposition t … © 2010 Parabon Computation, Inc.

Problem Decomposition t m m+1 m+2 m+3 … Calculate monthly checkpoints once © 2010

Problem Decomposition t m m+1 m+2 m+3 … Calculate monthly checkpoints once © 2010 Parabon Computation, Inc.

Problem Decomposition t m m+1 m+2 m+3 … Calculate monthly checkpoints once Save them

Problem Decomposition t m m+1 m+2 m+3 … Calculate monthly checkpoints once Save them in a repository © 2010 Parabon Computation, Inc.

Problem Decomposition t m m+1 m+2 m m+1 Save them in a repository m+2

Problem Decomposition t m m+1 m+2 m m+1 Save them in a repository m+2 m+3 … Calculate monthly checkpoints once Subsequently, perform deep diagnostics runs in parallel fashion m+4 … © 2010 Parabon Computation, Inc.

Problem Decomposition t m m+1 m+2 m m+1 Save them in a repository m+2

Problem Decomposition t m m+1 m+2 m m+1 Save them in a repository m+2 m+4 … m+3 … Calculate monthly checkpoints once Subsequently, perform deep diagnostics runs in parallel fashion Solution credit: Mike Seablom © 2010 Parabon Computation, Inc.

Speedup and Cost • In round numbers, time to compute ensemble … – –

Speedup and Cost • In round numbers, time to compute ensemble … – – > 100 years on single computer 50 days on 1, 000 computers 12 hours on 100, 000 computers NASA has ~80, 000 desktops © 2010 Parabon Computation, Inc.

Speedup and Cost • In round numbers, time to compute ensemble … – –

Speedup and Cost • In round numbers, time to compute ensemble … – – > 100 years on single computer 50 days on 1, 000 computers 12 hours on 100, 000 computers NASA has ~80, 000 desktops • Cost = Marginal cost of electricity – Incremental wattage expenditure: ~30 W – “Flat out” (worst case) marginal cost: $0. 07 per © 2010 Parabon Computation, Inc.

Modeling & Simulation as a Service Core Application Developer Domain Programmer QA Tester End

Modeling & Simulation as a Service Core Application Developer Domain Programmer QA Tester End User © 2010 Parabon Computation, Inc.

Climate Modeling as a Service © 2010 Parabon Computation, Inc.

Climate Modeling as a Service © 2010 Parabon Computation, Inc.

Climate Modeling as a Service © 2010 Parabon Computation, Inc.

Climate Modeling as a Service © 2010 Parabon Computation, Inc.

What’s New Since 25 June? Frontier Grid Server on Nebula Cloud NEBULA E NASA

What’s New Since 25 June? Frontier Grid Server on Nebula Cloud NEBULA E NASA Ames NASA Goddard General Public Frontier Data Server on Amazon S 3 © 2010 Parabon Computation, Inc.

Climate Modeling as a Service Climate@Home with Existing Infrastructure NASA IT Summit 2010 Steve

Climate Modeling as a Service Climate@Home with Existing Infrastructure NASA IT Summit 2010 Steve Armentrout, Parabon Tsengdar Lee, NASA HQ © 2008 Parabon Inc. All rights reserved. | 10 Parabon © 2009 Computation, Inc. All rights reserved. | 2010 26