Federal Department of Home Affairs FDHA Federal Office

  • Slides: 24
Download presentation
Federal Department of Home Affairs FDHA Federal Office of Meteorology and Climatology Meteo. Swiss

Federal Department of Home Affairs FDHA Federal Office of Meteorology and Climatology Meteo. Swiss Operational COSMO Demonstrator OPCODE André Walser and Oliver Fuhrer Meteo. Swiss COSMO-GM, Rome, 5 -9 September 2011

Project overview • Additional proposal to the Swiss HP 2 C initiative to build

Project overview • Additional proposal to the Swiss HP 2 C initiative to build an “OPerational COSMO DEmonstrator (OPCODE)” • Project proposal accepted by end of May • Start of project 1 June 2011 until end of 2012 • Project resources: • second contract with IT company SCS to continue collaboration until end of 2012 • 2 new positions at Meteo. Swiss for about 1 year • Swiss HPC center CSCS • C 2 SM (collaboration with ETH Zurich and others) OPCODE | COSMO-GM 2011 André Walser (andre. [email protected] ch) 2

Main goals • Leverage the research results of the ongoing HP 2 C COSMO

Main goals • Leverage the research results of the ongoing HP 2 C COSMO project • Prototyp implementation of the COSMO production suite of Meteo. Swiss making aggressive use of GPU technology • Meteo. Swiss ready to buy a GPU based hardware for the 2015 production machine • Same time-to-solution on substantially cheaper hardware: Cray XT 4 (3 cabinets) OPCODE | COSMO-GM 2011 André Walser (andre. [email protected] ch) GPU based hardware (a few rack units) 3

GPU perspectives GFLOPS per Watt is expected to increase strongly in the next years

GPU perspectives GFLOPS per Watt is expected to increase strongly in the next years OPCODE | COSMO-GM 2011 André Walser (andre. [email protected] ch) 4

Workflow on demonstrator OPCODE | COSMO-GM 2011 André Walser (andre. walser@meteoswiss. ch) 5

Workflow on demonstrator OPCODE | COSMO-GM 2011 André Walser (andre. [email protected] ch) 5

Current production scheme 46 49 34 25 -72 h forecast (00 UTC) and TC

Current production scheme 46 49 34 25 -72 h forecast (00 UTC) and TC products 0 -24 h forecast (00 UTC) and TC products 11 0 -24 h forecast (00 UTC) and TC products 7 3 h assimilation (21 UTC) 01 3 h assimilation (21 UTC) COSMO-7 / COSMO-2 suite: 61 Elapsed time in min COSMO-7 assimilation COSMO-7 forecast COSMO-7 TC products COSMO-2 assimilation COSMO-2 forecast COSMO-2 TC products • Time-critical post-processing takes about 15 minutes longer than forecasts for both COSMO-2 and COSMO-7 • current bottleneck is post-processing tool fieldextra entire suite has to be optimized for demonstrator OPCODE | COSMO-GM 2011 André Walser (andre. [email protected] ch) 6

Two workpages • Workpage A: Porting remainig parts of opr COSMO code @ Meteo.

Two workpages • Workpage A: Porting remainig parts of opr COSMO code @ Meteo. Swiss to demonstrator • Workpage B: Porting suite to demonstrator, optimize it, and operate it OPCODE | COSMO-GM 2011 André Walser (andre. [email protected] ch) 7

Work package A To use full speed-up, data has to remain on GPU within

Work package A To use full speed-up, data has to remain on GPU within a time step; sent to CPU for I/O only COSMO workflow: Input What’s still missing for a full GPU implementation? Physics Dynamics Assimilation Boundary Conditions Diagnostics Output OPCODE | COSMO-GM 2011 André Walser (andre. [email protected] ch) 8

Work package A To use full speed-up, data has to remain on GPU within

Work package A To use full speed-up, data has to remain on GPU within a time step; sent to CPU for I/O only COSMO workflow: What’s still missing for a full GPU implementation? Input Physics HPC 2 Dynamics HPC 2 Assimilation Boundary Conditions Diagnostics Output OPCODE | COSMO-GM 2011 André Walser (andre. [email protected] ch) 9

Tasks Work Package A Task A 1. Dynamical Core: complete/update HP 2 C code

Tasks Work Package A Task A 1. Dynamical Core: complete/update HP 2 C code SCS OPCODE | COSMO-GM 2011 André Walser (andre. [email protected] ch) 10

Task A 2: Inter-/intra-GPU parallelization • COSMO requires a communication library with halo-update as

Task A 2: Inter-/intra-GPU parallelization • COSMO requires a communication library with halo-update as well as several other communications (e. g. global reduce, gather, scatter) • e. g. peer-to-peer: OPCODE | COSMO-GM 2011 André Walser (andre. [email protected] ch) 11

Work Package A Task A 1. Dynamical Core: complete/update HP 2 C code SCS

Work Package A Task A 1. Dynamical Core: complete/update HP 2 C code SCS A 2. Inter-GPU parallelization: library for halo-updates, global reductions, scatters, gathers CSCS OPCODE | COSMO-GM 2011 André Walser (andre. [email protected] ch) 12

Work Package A Task A 1. Dynamical Core: complete/update HP 2 C code SCS

Work Package A Task A 1. Dynamical Core: complete/update HP 2 C code SCS A 2. Inter-GPU parallelization: library for halo-updates, global reductions, scatters, gathers CSCS A 3. Interoperability C++/CUDA/Fortran: common compile system, Unified Virtual Addressing SCS OPCODE | COSMO-GM 2011 André Walser (andre. [email protected] ch) 13

A 4. Data Assimilation: Porting to GPU Assimilation part is a huge code! OPCODE

A 4. Data Assimilation: Porting to GPU Assimilation part is a huge code! OPCODE | COSMO-GM 2011 André Walser (andre. [email protected] ch) 14

Work Package A Task A 1. Dynamical Core: complete/update HP 2 C code SCS

Work Package A Task A 1. Dynamical Core: complete/update HP 2 C code SCS A 2. Inter-GPU parallelization: library for halo-updates, global reductions, scatters, gathers CSCS A 3. Interoperability C++/CUDA/Fortran: common compile system, Unified Virtual Addressing SCS A 4. Data Assimilation: Porting to GPU Meteo. Swiss OPCODE | COSMO-GM 2011 André Walser (andre. [email protected] ch) 15

Work Package A Task A 1. Dynamical Core: complete/update HP 2 C code SCS

Work Package A Task A 1. Dynamical Core: complete/update HP 2 C code SCS A 2. Inter-GPU parallelization: library for halo-updates, global reductions, scatters, gathers CSCS A 3. Interoperability C++/CUDA/Fortran: common compile system, Unified Virtual Addressing SCS A 4. Data Assimilation: Porting to GPU Meteo. Swiss A 5. I/O: Software layer controling copying of fields from CPU to GPU and vice versa for I/O C 2 SM? OPCODE | COSMO-GM 2011 André Walser (andre. [email protected] ch) 16

Work Package A Task A 1. Dynamical Core: complete/update HP 2 C code SCS

Work Package A Task A 1. Dynamical Core: complete/update HP 2 C code SCS A 2. Inter-GPU parallelization: library for halo-updates, global reductions, scatters, gathers CSCS A 3. Interoperability C++/CUDA/Fortran: common compile system, Unified Virtual Addressing SCS A 4. Data Assimilation: Porting to GPU Meteo. Swiss A 5. I/O: Software layer controling copying of fields from CPU to GPU and vice versa for I/O C 2 SM? A 6. Porting other code parts (BC, diagnostics) to GPU SCS OPCODE | COSMO-GM 2011 André Walser (andre. [email protected] ch) 17

Work Package B Task B 1. Hardware CSCS B 2. System Software CSCS OPCODE

Work Package B Task B 1. Hardware CSCS B 2. System Software CSCS OPCODE | COSMO-GM 2011 André Walser (andre. [email protected] ch) 18

Work Package B Task B 1. Hardware CSCS B 2. System Software CSCS B

Work Package B Task B 1. Hardware CSCS B 2. System Software CSCS B 3. COSMO-Package: Porting and optimization of steering scripts Meteo. Swiss OPCODE | COSMO-GM 2011 André Walser (andre. [email protected] ch) 19

Work Package B Task B 1. Hardware CSCS B 2. System Software CSCS B

Work Package B Task B 1. Hardware CSCS B 2. System Software CSCS B 3. COSMO-Package: Porting and optimization of steering scripts Meteo. Swiss B 4. Post-processing: Parallelization of post-processing tools, additional work in fieldextra (partly paid by “COSMO license money”) Meteo. Swiss OPCODE | COSMO-GM 2011 André Walser (andre. [email protected] ch) 20

Work Package B Task B 1. Hardware CSCS B 2. System Software CSCS B

Work Package B Task B 1. Hardware CSCS B 2. System Software CSCS B 3. COSMO-Package: Porting and optimization of steering scripts Meteo. Swiss B 4. Post-processing: Parallelization of post-processing tools, additional work in fieldextra (partly paid by “COSMO license money”) Meteo. Swiss B 5. Setup and Testing Meteo. Swiss OPCODE | COSMO-GM 2011 André Walser (andre. [email protected] ch) 21

Organization 1. 7 FTE SCS, C 2 SM OPCODE | COSMO-GM 2011 André Walser

Organization 1. 7 FTE SCS, C 2 SM OPCODE | COSMO-GM 2011 André Walser (andre. [email protected] ch) 0. 9 FTE new position @Meteo. Swiss 1 year still open 1. 9 FTE new collaborator @ Meteo. Swiss 15 months, CSCS 22

Schedule OPCODE | COSMO-GM 2011 André Walser (andre. walser@meteoswiss. ch) 23

Schedule OPCODE | COSMO-GM 2011 André Walser (andre. [email protected] ch) 23

Thank you ! OPCODE | COSMO-GM 2011 André Walser (andre. walser@meteoswiss. ch) 24

Thank you ! OPCODE | COSMO-GM 2011 André Walser (andre. [email protected] ch) 24