Astro GridD Challenge of Large Simulations GRAPE and

  • Slides: 27
Download presentation
Astro. Grid-D Challenge of Large Simulations – GRAPE and GRID Report by: Rainer Spurzem

Astro. Grid-D Challenge of Large Simulations – GRAPE and GRID Report by: Rainer Spurzem (ZAH-ARI) spurzem(AT)ari. uni-heidelberg. de

Astrophysical Use. Case NBODY 6++ n n n Astrophysical Case for direct N-body: Star

Astrophysical Use. Case NBODY 6++ n n n Astrophysical Case for direct N-body: Star Clusters, Galactic Nuclei, Black Holes Special Hardware GRAPE, MPRACE (FPGA) GRAPE in the Grid, Astrogrid-D, International Thanks, Credits in the Astrogrid-D to • the ARI-ZAH group J. Wambsganß, J. Steinacker, T. Brüsemeister • Harry Enke (AIP) and the entire WG-5, Astrogrid-D May 2007 OGF 20 2

N-Body + Grav. Waves @ ARI: Peter Berczik, Ingo Berentzen, Jonathan Downing, Miguel Preto,

N-Body + Grav. Waves @ ARI: Peter Berczik, Ingo Berentzen, Jonathan Downing, Miguel Preto, Gabor Kupi, Christoph Eichhorn David Merritt (RIT, USA)… in VESF/LSC collaboration: on gravitational wave modelling from dense star clusters: Pau Amaro-Seoane (AEI, Potsdam, D) G. Schäfer, A. Gopakumar (Univ. Jena, D) M. Benacquista (UT Brownsville, USA) Jun Makino and colleagues in Tokyo… …support, cooperation, over many years… May 2007 Further collaborations: Sverre Aarseth (Io. A Cambridge UK) Seppo Mikkola (U Turku, FIN) OGF 20 3

Direct N-Body Software High Accuracy Integrators for Systems with long-range force + relaxation (gravothermal)

Direct N-Body Software High Accuracy Integrators for Systems with long-range force + relaxation (gravothermal) S. J. Aarseth, S. Mikkola (ca. 20. 000 lines): • Hierarchical Block Time Steps • Ahmad-Cohen Neighbour Scheme • Kustaanheimo-Stiefel and Chain-Regular. for bound subsystems of N<6 (Quaternions!) • 4 th order Hermite scheme (pred/corr) • Bulirsch-Stoer (for KS) • NBODY 6 (Aarseth 1999) • NBODY 6++ (Spurzem 1999) using MPI/shmem • Parallel Binary Integration in Progress • Parallel GRAPE Use (Harfst, Gualandris, Merritt, Spurzem, Berczik, Portegies Zwart, 2007) May 2007 OGF 20 4

Direct N-Body Software High Accuracy Integrators: Record with GRAPE cluster at 2 million particles!

Direct N-Body Software High Accuracy Integrators: Record with GRAPE cluster at 2 million particles! Harfst, Gualandris, Merritt, Spurzem, Berczik Baumgardt, Heggie, Hut Baumgardt, Makino ● ● by D. C. Heggie Via www. maths. ed. ac. uk Larger N needed! May 2007 OGF 20 5

Globular Cluster ω Centauri (Central Region) Ground Based View May 2007 OGF 20 6

Globular Cluster ω Centauri (Central Region) Ground Based View May 2007 OGF 20 6

Detection of Gravitational Waves? Was Einstein right? May 2007 OGF 20 7

Detection of Gravitational Waves? Was Einstein right? May 2007 OGF 20 7

Example: VIRGO Detector in Cascina near Pisa, Italy May 2007 OGF 20 8

Example: VIRGO Detector in Cascina near Pisa, Italy May 2007 OGF 20 8

Basic idea of any GRAPE N-body code: ~N May 2007 ~N^2 OGF 20 9

Basic idea of any GRAPE N-body code: ~N May 2007 ~N^2 OGF 20 9

Hardware - GRAPE ~128 Gflops for a price ~5 K USD; Memory for up

Hardware - GRAPE ~128 Gflops for a price ~5 K USD; Memory for up to 128 K particles GRAPE 6 a PCI board GRAPE 6 a, -BL - PCI Board for PC-Clusters PROGRAPE-4, FPGA based board from RIKEN (Hamada) GRAPE 7 – new FPGA based board from Tokyo Univ. (Fukushige) GRAPE-DR – new board from Makino et al. NAOJ MPRACE 1, 2 – FPGA boards from Univ. Mannheim/GRACE (Kugel et al. ) May 2007 OGF 20 10

May 2007 OGF 20 11

May 2007 OGF 20 11

ARI 32 node GRAPE 6 a clusters • 32 dual-Xeon 3. 0 GHz nodes

ARI 32 node GRAPE 6 a clusters • 32 dual-Xeon 3. 0 GHz nodes • 32 GRAPE 6 a • 14 TB RAID • Infiniband link (10 Gb/s) • Speed: ~4 Tflops • N up to 4 M • Cost: ~500 K USD • Funding: NSF/NASA/RIT • 32 dual-Xeon 3. 2 GHz nodes • 32 GRAPE 6 a • 32 FPGA • 7 TB RAID • Dual port Infiniband link (20 Gb/s) • Speed: ~4 Tflops • N up to 4 M • Cost: ~380 K EUR • Funding: Volkswagen/Baden-Württemberg May 2007 OGF 20 Infiniband Dual 20 Gb/s 12

ARI-ZAH + RIT GRAPE 6 a clusters Performance Analysis (3. 2 Tflop/s): Harfst et

ARI-ZAH + RIT GRAPE 6 a clusters Performance Analysis (3. 2 Tflop/s): Harfst et al. 2007, New Astron. May 2007 OGF 20 13

Hardware May 2007 OGF 20 15

Hardware May 2007 OGF 20 15

May 2007 OGF 20 16

May 2007 OGF 20 16

May 2007 OGF 20 17

May 2007 OGF 20 17

May 2007 OGF 20 18

May 2007 OGF 20 18

May 2007 OGF 20 19

May 2007 OGF 20 19

May 2007 OGF 20 20

May 2007 OGF 20 20

Parallel PP on GRAPE 6 a cluster ARI Cluster: ~3. 2 Tlop/s sustained Harfst,

Parallel PP on GRAPE 6 a cluster ARI Cluster: ~3. 2 Tlop/s sustained Harfst, Gualandris, Merritt, Spurzem, Portegies Zwart, Berczik, New Astron. 2007. May 2007 OGF 20 21

Visualisation With S. Dominiczak W. Frings John-von. Neumann Institute for Computing (NIC) FZ Jülich

Visualisation With S. Dominiczak W. Frings John-von. Neumann Institute for Computing (NIC) FZ Jülich google for xnbody May 2007 OGF 20 22

Present Situation - Requirements n n n DEISA / Unicore Project (without special hardware)

Present Situation - Requirements n n n DEISA / Unicore Project (without special hardware) Xnbody Visualization with FZ Jülich (Unicore) NBODY 6++ Use. Case in Astrogrid-D (Globus GT 4. 0) Simple JSDL Job ok Parallel Job + GRAPE/MPRACE request in progress Astrogrid-D Applied: Participation in AGENA Network (EGEE) Goal: share and load-balance unique GRAPE/MPRACE resources in international frame, prepare in Europe May 2007 OGF 20 23

International Interest in GRAPE-Grid Members of Astrogrid-D: ARI-ZAH Univ. Heidelberg, D Main Astron. Obs.

International Interest in GRAPE-Grid Members of Astrogrid-D: ARI-ZAH Univ. Heidelberg, D Main Astron. Obs. Kiev, UA Candidates: Univ. Amsterdam, NL Obs. Astroph. Marseille, F Fessenkov Obs. , Almaty, KZ May 2007 OGF 20 24

NBODY 6++ as a portable JSDL Job User Interface will be based on Java

NBODY 6++ as a portable JSDL Job User Interface will be based on Java or XML May 2007 OGF 20 25

WG 5: Current status, Job Management Entscheidung für die Job Submission Data Language (JSDL)

WG 5: Current status, Job Management Entscheidung für die Job Submission Data Language (JSDL) wird vom open grid forum (OGF) unterstützt GUI JSDL jsdlproc RSL/XML (GT 4. 2 wird gerade entwickelt und wird JSDL direkt unterstützen) GT 4. 0 May 2007 OGF 20 26

WG 5: Current status, Scheduler/Broker n Open Source n Leightweight Metascheduler on top of

WG 5: Current status, Scheduler/Broker n Open Source n Leightweight Metascheduler on top of GT 2. 4/GT 4 n Central Server Architecture Support of GGF DRMAA standard API for job submission and management Simple round robin/flooding scheduling algorithm, but extensible May 2007 OGF 20 27

WG 5: Current status, Scheduler/Broker Information System GT 4 Resources Matchmaking Gridway hydra. ari.

WG 5: Current status, Scheduler/Broker Information System GT 4 Resources Matchmaking Gridway hydra. ari. uni-heidelberg. de Scheduler / Broker Job Status: “gwps” May 2007 OGF 20 28

Challenge Challenge Challenge Challenge Challenge Challenge Challenge ChallengeChallenge Challenge Challenge Challenge Challenge Challenge Challenge Challenge
The Astro GridD Information Service Stellaris A centralThe Astro GridD Information Service Stellaris A central
Diseases Management in Grape Nursery Grape seedlings areDiseases Management in Grape Nursery Grape seedlings are
Astro Meteorology at the University of Valparaso AstroAstro Meteorology at the University of Valparaso Astro
Introduction to Astro Image J Photometry Software AstroIntroduction to Astro Image J Photometry Software Astro
Astro circumpolare e astro non circumpolare Z jAstro circumpolare e astro non circumpolare Z j
Astro WISE LC workshop VESUVIO beyond Astro WISEAstro WISE LC workshop VESUVIO beyond Astro WISE
Simulations CS 2123 What are simulations Simulations attemptSimulations CS 2123 What are simulations Simulations attempt
Digital Pattern Simulations Pattern Simulations Pattern Simulations WhatDigital Pattern Simulations Pattern Simulations Pattern Simulations What
Study of SevenEleven Team grape candys challenge PurposeStudy of SevenEleven Team grape candys challenge Purpose
Astro 10 Lecture 14 Large Scale Structure andAstro 10 Lecture 14 Large Scale Structure and
Large intestine Anatomy of the Large Intestine LargeLarge intestine Anatomy of the Large Intestine Large
How LARGE is a large class Teaching LargeHow LARGE is a large class Teaching Large
LARGE INTESTINE LARGE INTESTINE COLOR CODE THE LARGELARGE INTESTINE LARGE INTESTINE COLOR CODE THE LARGE
CATE Overview Astro 2020 Large Mission Concepts SeptemberCATE Overview Astro 2020 Large Mission Concepts September
Survey Form for Astro Challenge Crash Course httpsSurvey Form for Astro Challenge Crash Course https
Challenge Title Expert Training Agenda Challenge Overview ChallengeChallenge Title Expert Training Agenda Challenge Overview Challenge
Fed Challenge Fed Challenge College Fed Challenge isFed Challenge Fed Challenge College Fed Challenge is
THE CUBOID CHALLENGE CHALLENGE CHALLENGE ACCEPTED Rough FigureTHE CUBOID CHALLENGE CHALLENGE CHALLENGE ACCEPTED Rough Figure
Retrieval Practice Challenge Grid Insert challenge Insert challengeRetrieval Practice Challenge Grid Insert challenge Insert challenge
Challenge 1 Daily 8 22221 Challenge 2 ChallengeChallenge 1 Daily 8 22221 Challenge 2 Challenge
Simulations and Software Volker Friese Simulations Software ComputingSimulations and Software Volker Friese Simulations Software Computing
Prototype simulations Test beam simulations Conclusion Prototype andPrototype simulations Test beam simulations Conclusion Prototype and
Simulations and Learning By Andrew Carpino Roleplay SimulationsSimulations and Learning By Andrew Carpino Roleplay Simulations