Higher Ed Research May 8 th 2012 Sections

Higher Ed & Research May 8 th, 2012

Sections Included * Molecular Dynamics Applications Overview AMBER NAMD GROMACS LAMMPS * In fullscreen mode, click on link to view a particular module. Click on NVIDIA logo in each slide to return to this page.

Molecular Dynamics (MD) Applications Features Supported GPU Perf Release Status Notes/Benchmarks AMBER PMEMD Explicit Solvent & GB Implicit Solvent 89. 44 ns/day JAC NVE on 16 X 2090 s Released Multi‐GPU, multi‐node AMBER 12 NAMD Full electrostatics with PME and most simulation features 6. 44 ns/days STMV 585 X 2050 s Released 100 M atom capable Multi‐GPU, multi‐node GROMACS Implicit (5 x), Explicit (2 x) Solvent via Open. MM 165 ns/Day DHFR 4 X C 2075 s 4. 5 Single GPU released 4. 6 Multi‐GPU Released http: //biowulf. nih. gov/apps/gromacs‐ gpu. html LAMMPS Lennard‐Jones, Gay‐ Berne, Tersoff 3. 5‐ 15 x Released. Multi‐GPU, multi‐node 1 billion atom on Lincoln: Application http: //ambermd. org/gpus/benchmarks. htm#Benchmarks NAMD 2. 8. 2. 9 version April 12 http: //biowulf. nih. gov/apps/namd/na md_ bench. html http: //lammps. sandia. gov/bench. html # machine GPU Perf compared against Multi-core x 86 CPU socket. GPU Perf benchmarked on GPU supported features and may be a kernel to kernel perf comparison

New/Additional MD Applications Ramping Application Features Supported GPU Perf Release Status Notes Abalone TBD Simulations 4‐ 29 X (on 1060 GPU) Released Single GPU. Agile Molecule, Inc. ACEMD Written for use on GPUs 160 ns/day Released Production bio‐molecular dynamics (MD) software specially optimized to run on single and multi‐GPUs DL_POLY Two‐body Forces, Link‐ cell Pairs, Ewald SPME forces, Shake VV 4 x V 4. 0 Source only Results Published Multi‐GPU, multi‐node supported Written for use on GPUs 2 X (32 CPU cores vs. 2 10 XX GPUs) Released, Version 0. 9. 2 Single and multi‐GPU. HOOMD‐ Blue GPU Perf compared against Multi‐core x 86 CPU socket. GPU Perf benchmarked on GPU supported features and may be a kernel to kernel perf comparison

GPU Value to Molecular Dynamics What Study disease & discover drugs Predict drug and protein interactions Why Speed of simulations is critical Enables study of: Longer timeframes Larger systems More simulations How GPUs increase throughput & accelerate simulations AMBER 11 Application 4. 6 x performance increase with 2 GPUs with only a 54% added cost* • • AMBER 11 Cellulose NPT on 2 x E 5670 CPUS + 2 x Tesla C 2090 s (per node) vs. 2 xc. E 5670 CPUs (per node) Cost of CPU node assumed to be $9333. Cost of adding two (2) 2090 s to single node is assumed to be $5333 GPU Ready Applications Abalone ACEMD AMBER DL_PLOY GAMESS GROMACS LAMMPS NAMD GPU Test Drive Pre-configured Applications AMBER 11 NAMD 2. 8

All Key MD Codes are GPU Ready AMBER, NAMD, GROMACS, LAMMPS Life and Material Sciences Great multi‐GPU performance Additional MD GPU codes: Abalone, ACEMD, HOOMD‐Blue Focus: scaling to large numbers of GPUs

Outstanding AMBER Results with GPUs

Run AMBER Faster Up to 5 x Speed Up With GPUs ns/day 60 AMBER 11 JAC Simulation time, ns/day DHFR (NVE) 23, 558 Atoms CPU Supercomputer Cluster Performance Scaling 50 56. 45 46. 01 45. 58 40 ns/Day 58. 28 31. 57 30 21. 29 ns/Day 14. 16 20 13. 49 8. 22 10 2. 44 GPU+CPU 4. 55 0 1 node 2 nodes CPU only 4 nodes 8 nodes CPU + 1 x C 2050 per node NICS Kraken (Athena) “…with two GPUs we can run a single simulation as fast as on 128 CPUs of a Cray XT 3 or on 1024 CPUs of an IBM Blue. Gene/L machine. We can try things that were undoable before. It still blows my mind. ” Axel Kohlmeyer Temple University

AMBER Make Research More Productive with GPUs Relative Cost and Performance Benefit Scale 5. 0 318% Higher Performance 4. 0 3. 0 AMBER 11 on 2 X E 5670 CPUs (per node) AMBER 11 on 2 X E 5670 CPUs + 2 X Tesla M 2090 s (per node) 2. 0 54% Additional Expense 1. 0 Base node configuration: Dual Xeon X 5670 s and Dual Tesla M 2090 GPUs per node With GPU No GPU 0. 0 Cost of 1 Node Performance Speed-up Adding Two 2090 GPUs to a Node Yields a > 4 x Performance Increase

Run NAMD Faster STMV Up to 7 x Speed Up With GPUs 1, 066, 628 Atoms Ns/Day Apo. A-1 92, 224 Atoms ns/Day 2. 94 NAMD 2. 8 Benchmark 3. 5 3 2. 5 GPU+CPU 2 CPU only 1. 5 1 ns/Day 0. 51 0. 5 0 GPU+CPU Test Platform: 1 Node, Dual Tesla M 2090 GPU (6 GB), Dual Intel 4 -core Xeon (2. 4 GHz), NAMD 2. 8, CUDA 4. 0, ECC On. Visit www. nvidia. com/simcluster for more information on speed up results, configuration and test models. 1 2 4 8 # of Compute Nodes 12 16 NAMD 2. 8 B 1 + unreleaesd patch, STMV Benchmark A Node is Dual-Socket, Quad-core x 5650 with 2 Tesla M 2070 GPUs Performance numbers for 2 M 2070 8 cores (GPU+CPU) vs. 8 cores (CPU)

Make Research More Productive with GPUs Relative Cost and Performance Benefit Scale 4. 0 NAMD 2. 8 on 2 X E 5670 CPUs (per node) NAMD 2. 8 on 2 X E 5670 CPUs + 2 X Tesla C 2070 s (per node) 3. 0 250% Higher 54% Additional Expense With GPU 1. 0 No GPU 0. 0 Cost of 1 Node Performance Speed-up Get up to a 250% Performance Increase (STMV – 1, 066628 atoms)

GROMACS Partnership Overview Erik Lindahl, David van der Spoel, Berk Hess are head authors and project leaders. Szilárd Páll is a key GPU developer. 2010: single GPU support (Open. MM library in GROMACS 4. 5) NVIDIA Dev Tech resources allocated to GROMACS code 2012: GROMACS 4. 6 will support multi‐GPU nodes as well as GPU clusters

GROMACS 4. 6 Release Features GROMACS Multi‐GPU Expected in April 2012 Multi‐GPU support ‐ GPU acceleration is one of main focus: majority of features will be accelerated in 4. 6 in a transparent fashion PME simulations get special attention, and most of the effort will go into making these algorithms well load‐balanced Reaction‐Field and Cut‐Off simulations also run accelerated List of non‐supported GPU accelerated features will be quite short

GROMACS 4. 6 Alpha Release Absolute Performance Absolute performance of GROMACS running CUDA - and SSE-accelerated non-bonded kernels with PME on 3 -12 CPU cores and 1 -4 GPUs. Simulations with cubic and truncated dodecahedron cells, pressure coupling, as well as virtual interaction sites enabling 5 fs are shown Benchmark systems: RNAse in water with 24040 atoms in cubic and 16816 atoms in truncated dodecahedron box Settings: electrostatics cut-off auto-tuned >0. 9 nm, LJ cut-off 0. 9 nm, 2 fs and 5 fs (with vsites) time steps Hardware: workstation with 2 x Intel Xeon X 5650 (6 C), 4 x NVIDIA Tesla C 2075

GROMACS 4. 6 Alpha Release Strong Scaling Strong scaling of GPU‐accelerated GROMACS with PME and reaction‐field on: Up to 40 cluster nodes with 80 GPUs Benchmark system: water box with 1. 5 M particles Settings: electrostatics cut‐off auto‐tuned >0. 9 nm for PME and 0. 9 nm for reaction‐field, LJ cut‐off 0. 9 nm, 2 fs time steps Hardware: Bullx cluster nodes with 2 x Intel Xeon E 5649 (6 C), 2 x NVIDIA Tesla M 2090, 2 x QDR Infiniband 40 Gb/s

GROMACS 4. 6 Alpha Release PME Weak Scaling Weak scaling of the GPU‐accelerated GROMACS with reaction‐field and PME on: 3‐ 12 CPU cores and 1‐ 4 GPUs. The gradient background indicates the range of system Sizes which fall beyond the typical single‐node production size. Benchmark systems: water boxes size ranging from 1. 5 k to 3 M particles. Settings: electrostatics cut‐off auto‐ tuned >0. 9 nm for PME and 0. 9 nm for reaction‐field, LJ cut‐off 0. 9 nm, 2 fs time steps. Hardware: workstation with 2 x Intel Xeon X 5650 (6 C), 4 x NVIDIA Tesla C 2075.

GROMACS 4. 6 Alpha Release Rxn‐Field Weak Scaling Weak scaling of the GPU‐accelerated GROMACS with reaction‐field and PME on: 3‐ 12 CPU cores and 1‐ 4 GPUs. The gradient background indicates the range of system sizes which fall beyond the typical single‐node production size Benchmark systems: water boxes size ranging from 1. 5 k to 3 M particles Settings: electrostatics cut‐off auto‐ tuned >0. 9 nm for PME and 0. 9 nm for reaction‐field, LJ cut‐off 0. 9 nm, 2 fs time steps Hardware: workstation with 2 x Intel Xeon X 5650 (6 C), 4 x NVIDIA Tesla C 2075

GROMACS 4. 6 Alpha Release Weak Scaling Weak scaling of the CUDA non‐bonded force kernel on Ge. Force and Tesla GPUs. Perfect weak scaling, challenges for strong scaling Benchmark systems: water boxes size ranging from 1. 5 k to 3 M particles Settings: electrostatics & LJ cut‐off 1. 0 nm, 2 fs time steps Hardware: workstation with 2 x Intel Xeon X 5650 (6 C) CPUs, 4 x NVIDIA Tesla C 2075

LAMMPS Released GPU Features and Future Plans LAMMPS August 2009 First GPU accelerated support LAMMPS Aug. 22, 2011 Selected accelerated Non‐bonded short‐range potentials (SP, MP, DP support) Lennard‐Jones (several variants with & without coulombic interactions) Morse Buckingham CHARMM Tabulated Course grain SDK Anisotropic Gay‐Bern RE‐squared “Hybrid” combinations (GPU accel & no GPU accel) Particle‐Particle‐Mesh (SP or DP) Neighbor list builds Longer Term* Improve performance on smaller particle counts Neighbor List is the problem Improve long‐range performance MPI/Poisson Solve is the problem Additional pair potential support (including expensive advanced force fields) – See “Tremendous Opportunity for GPUs” slide* Performance improvements focused to specific science problems * Courtesy of Michael Brown at ORNL and Paul Crozier at Sandia Labs

LAMMPS 8. 6 x Speed-up with GPUs W. M. Brown, “GPU Acceleration in LAMMPS”, 2011 LAMMPS Workshop

LAMMPS 4 x Faster on Billion Atoms Billion Atom Lennard-Jones Benchmark 103 Seconds 29 Seconds 288 GPUs + CPUs 1920 x 86 CPUs Test Platform: NCSA Lincoln Cluster with S 1070 1 U GPU servers attached CPU-only Cluster- Cray XT 5

LAMMPS 4 X-15 X Speedups Gay-Berne RE-Squared From August 2011 LAMMPS Workshop Courtesy of W. Michael Brown, ORNL

LAMMPS Conclusions Runs both with individual multi‐GPU node, as well as GPU clusters Outstanding raw performance! Performance is 3 x‐ 40 X higher than equivalent CPU code Impressive linear strong scaling Good weak scaling, scales to a billion particles Tremendous opportunity to GPU accelerate other force fields