Theory Grid and VO 9162021 Matthias Steinmetz AIP

Theory, Grid and VO 9/16/2021 Matthias Steinmetz (AIP)

Characteristics of a Grid: network of IT-Ressourcen Data analysis PC Cluster M I D D L E W A R E Data Archive Data analysis Super Computer Data Archive application PC Cluster Super computer Telescopes User Resources are "virtualized", i. e. can not be identified individually 9/16/2021

VO and Grid What is the dividing line between VO and Grid? Not well defined Example UK: Astro. Grid covers VO and Grid aspects Example Germany GAVO: application layer Astro. Grid-D: middle ware 9/16/2021 3

Theory and Grid Benefits of the Grid Logistics (resource monitoring, scheduler/broker, virtual organizations, …) Virtual Surveys (Millenium simulation) Enterprise computing (access to supercomputers via grids, e. g. DEISA) Cloud computing (“task farming”) Volunteer computing (@home model) Visualization 9/16/2021 4

Stellar. IS: resource monitoring Grid-Ressource. Map Basiert auf Google. Map 9/16/2021 5

Stellaris: Job monitoring Minutes Time table for submitted Jobs hours days Webinterface for simple job monitoring 9/16/2021 6

Analysing Cosmological Simulations in the Virtual Observatory: Designing and Mining the Millennium Simulation Database Gerard Lemson German Astrophysical Virtual Observatory ARI, Heidelberg MPE, Garching bei München 9/16/2021 7

Time evolution: merger trees 9/16/2021 8

9/16/2021 9

Merger trees : select prog. * from galaxies des , galaxies prog where des. galaxy. Id = 0 and prog. galaxy. Id between des. galaxy. Id and des. last. Progenitor. Id Branching points : select descendant. Id from galaxies des where descendant. Id != -1 group by descendant. Id having count(*) > 1 9/16/2021 10

9/16/2021 11

Usage statistics Up since August 2006 (astro-ph/. . . ) ~210 registered users > 4. 4 million queries ~ 35 billion rows (since March 2007) 9/16/2021 # queries/day # rows/day # secs/day 12

Enterprise Computing: Mare Nostrum Simulations at HLRZ WMAP 3 parameters, w=0. 8 Testrun using the grid: 10243+10243 particles 9/16/2021 13

NBODY 6++ Use. Case Computer simulation of dense stellar systems Example: globular clusters Gravitational Star-Star interaction Complexity N 2 (N: number of stars) 9/16/2021 14

Enterprise Computers Recent Development: GPU – Graphics Cards Ge. Force 8800 GTX (NVIDIA) Using CUDA Library Special Interfaces and API from GRACE project ported. Berczik et al. 2008 Spurzem et al. 2008 9/16/2021 15

Cloud Computing: Use. Case Dynamo Visualization of results 2 D-Darstellung Querschnitt der durch Magnetfeldstärke Rechner 1 den auf der Sternoberfläche Rechner 2 Rechner 3 Rechner 4 9/16/2021 16

Dynamo JDSL und RSL <jsdl: Job. Definition xmlns="http: //www. gacg-grid. de/namespaces/job-mgmt/2006/08/jsdl" xmlns: jsdl="http: //schemas. ggf. org/jsdl/2005/11/jsdl" xmlns: jsdl-posix="http: //schemas. ggf. org/jsdl/2005/11/jsdl-posix" xmlns: xsi="http: //www. w 3. org/2001/XMLSchema-instance"> <jsdl: Job. Description> <jsdl: Job. Identification> <jsdl: Job. Name> Sample Dynamo run </jsdl: Job. Name> <job> <jsdl: Description> <executable>test. x</executable> Use Case Dynamo <directory>/${GLOBUS_USER_HOME}/ </jsdl: Description> dynamo</directory> <jsdl: Job. Project> <stdout>test. out</stdout> n/a </jsdl: Job. Project><max. Wall. Time>100</max. Wall. Time> <max. Memory>1</max. Memory> </jsdl: Job. Identification> <? xml version="1. 0" encoding="UTF-8"? > <jsdl: Resources> <jsdl: File. System name="HOME"> <jsdl: Description> User's home directory </jsdl: Description> </jsdl: File. System> </jsdl: Resources> […] 9/16/2021 <file. Stage. In> […] 17

Volunteer Computing: GEO 600 / LIGO Laser Interferometer Gravitational Wave Observatory 9/16/2021 18

GEO 600/LIGO Network von 4 Detectors Hanford (2000 m side length) Livingston (4000 m side length) GEO 600 ( 600 m side length) Virgo (3000 m side length) USA Germany Italy Pathfinder for LISA, Satellite mission with 3 detectors side length: 5*109 m! 9/16/2021 19

Gravitationional waves: Data analysis via the Grid Data analysis via small data packages, “embarrassingly parallel”. Einstein@Home is, like SETI@Home, suitable to exploit idle cycles on work stations. Einstein@Home is an ideal, simple Grid application, supporting many operation system. Checkpointing and Recovery allows a very accurate control of CPU-Requirements and walltime. Automatic software deployment job submission and job management, a good scalability of grid application can be obtained Current workload: 30000 CPU h per week 9/16/2021 20

GEO 600 – Resource Integration user friendly User-Management via VOMRS Resource information via MDS und Stellar. IS Grid Service Monitoring automatic job submission on D-Grid resources Job monitoring und job management using a Laptop data management on Astrogrid-D Storage Cluster 9/16/2021 21

Visualization of a galaxy merger Simulation: two galaxies on collision orbit Visualization: 2 D-projections of 3 D snapshots Submit Execution ZIB AIP Pro. C Submit Workflow start Video Workflow A ARI . . . Video Workflow B GT 4 submit exit GT 4 submit stop 9/16/2021 exit 22

Grid-Visualization Submission Host: ZIB Execution Hosts: AIP + ZAH Pro. C + Master workflow Submission of video workflows Display of videos Pi. Co + Video workflow Projection to 2 D Color coding Future: Graphics rendering at LRZ, graphics output on local host 9/16/2021 23

Theory and Grid Benefits of the Grid Logistics (resource monitoring, scheduler/broker, virtual organizations, …) Virtual Surveys (Millenium simulation) Enterprise computing (access to supercomputers via grids, e. g. DEISA) Cloud computing (“task farming”) Volunteer computing (@home model) Visualization 9/16/2021 24