The European Cross Grid Project Marcel Kunze Institute

The European Cross. Grid Project Marcel Kunze Institute for Scientific Computing (IWR) Forschungszentrum Karlsruhe Gmb. H www. eu-crossgrid. org M. Kunze, NEC 2003, Varna

Outline u Project Overview u Applications u Toolbox u Experience (Testbeds) M. Kunze, NEC 2003, Varna

Cross. Grid Collaboration 21 institutes 11 countries Ireland: Poland: TCD Dublin Netherlands: Uv. A Amsterdam Spain: Portugal: LIP Lisbon CSIC Santander Valencia & Red. Iris UAB Barcelona USC Santiago & CESGA Germany: FZK Karlsruhe TUM Munich USTU Stuttgart Austria: U. Linz Italy: DATAMAT Cyfronet & INP Cracow PSNC Poznan ICM & IPJ Warsaw Slovakia: II SAS Bratislava Greece: Algosystems Demo Athens Au. Th Thessaloniki Cyprus: UCY Nikosia M. Kunze, NEC 2003, Varna

IST Grid Project Space EGSO Applications CROSSGRID GRIA GRIP Middleware EUROGRID DATAGRIDLAB DAMIEN DATATAG Fabric Industry/Business M. Kunze, NEC 2003, Varna Science

Mission Statement Development of a Grid Environment for Interactive Applications Make the Grid more user friendly, secure and efficient Timeline: 2002 -2005 M. Kunze, NEC 2003, Varna

Workpackages u WP 1 – Application Development u WP 2 – Grid Application Programming Environment u WP 3 – New Grid Services and Tools u WP 4 – International Testbed Organisation u WP 5 – Project Management M. Kunze, NEC 2003, Varna

Architecture: Layered Structure Applications Middleware & Tools Fabric (Infrastructure) M. Kunze, NEC 2003, Varna

Technical Components WP 1 –Application Development WP 2 – Programming Environment 1. Interactive simulation and visualization of a biomedical system 2. Flooding crisis team support 3. Distributed data analysis in HEP 4. Weather forecast and air pollution modeling High Level Architecture Visualization Kernel Data Mining Grid Services: Data. Grid Gri. Phy. N. . . 1. MPI code debugging and verification 2. Metrics and benchmarks 3. Interactive performance evaluation tools WP 3 – New Services 1. 2. 3. 4. Portals and roaming access Grid resource management Grid monitoring Optimization of data access Globus Middleware WP 4 – Testbeds - Fabric Infrastructure M. Kunze, NEC 2003, Varna

Detailed Architecture Applications Supporting Tools 1. 1 Bio. Med 2. 2 MPI Verification 1. 2 Flooding 2. 3 Metrics and Benchmarks Applications Development Support App. Spec Services Generic Services Fabric MPICH-G 1. 1 User Interaction Services 3. 2 Scheduling Agents Data. Grid Job Submission Service Resource Manager (CE) CPU 1. 3 Interactive Distributed Data Access 1. 3 Data Mining on Grid (NN) 2. 4 Performance Analysis 3. 1 Portal & Migrating Desktop 1. 4 Meteo Pollution 1. 1, 1. 2 HLA and others 1. 1 Grid Visualisation Kernel 3. 4 Optimization of Grid Data Access GRAM Resource Manager (SE) Secondary Storage Grid. FTP 3. 3 Grid Monitoring GIS / MDS Resource Manager 3. 4 Optimization of Local Data Access 3. 1 Roaming Access GSI Globus-IO Data. Grid Replica Manager Globus Replica Manager Replica Catalog Resource Manager Instruments ( Satelites, Radars) Tertiary Storage M. Kunze, NEC 2003, Varna Cross. Grid Data. Grid Globus

Cross. Grid Applications M. Kunze, NEC 2003, Varna

Key Features of Cross. Grid Applications u u u Data n Data sources and data bases geographically distributed n To be selected on demand Processing n Large processing capacity required; both HPC & HTC n Interactive Presentation n Complex data requires versatile 3 D visualisation n Support for interaction and feedback to other components M. Kunze, NEC 2003, Varna

Biomedical Application CT / MRI scan Segmentation Medical DB LB flow simulation Medical DB Visualization HDB VE WD PC PDA Interaction 10 simulations/day 60 GB/simulation > 20 MB/s M. Kunze, NEC 2003, Varna

Bypass Surgery: Simulated Treatment Planning M. Kunze, NEC 2003, Varna

Interactive Treatment Planning The vascular geometry can be modified using a library of models Draw interactively Computational geometry M. Kunze, NEC 2003, Varna

Sample pulsatile Flow Simulation M. Kunze, NEC 2003, Varna

Flood Simulation Data sources Meteorological simulations Users Hydrological simulations Hydraulic simulations Output visualization M. Kunze, NEC 2003, Varna

3 D Visualization M. Kunze, NEC 2003, Varna

Flood Simulation: Flow and Water Depth M. Kunze, NEC 2003, Varna

Distributed Data Analysis in HEP u u Objectives n Distributed data analysis n Distributed data mining techniques with neural networks Issues n Typical interactive requests will run on o(TB) distributed data n Transfer/replication times for the whole data about one hour n Data transfers once and in advance of the interactive session n n Allocation, installation and set-up of corresponding database servers before the interactive session Integration of user-friendly interactive access (based on PROOF) M. Kunze, NEC 2003, Varna

Parallel ROOT Facility: PROOF Selection Parameters Tag. D B CPU Procedure PROOF RD B Proc. C DB 1 DB 2 DB 3 DB 4 DB 5 DB 6 Local CPU Remote CPU CPU http: //root. cern. ch M. Kunze, NEC 2003, Varna

Weather Forecast and Air Pollution Modeling u Distributed/parallel codes on the Grid n Coupled Ocean/Atmosphere Mesoscale Prediction System n STEM-II Air Pollution Code u Integration of distributed databases u Data mining applied to downscaling weather forecast M. Kunze, NEC 2003, Varna

Weather Forecast and Air Pollution Modeling M. Kunze, NEC 2003, Varna

Cross. Grid Toolbox M. Kunze, NEC 2003, Varna

Migrating Desktop u u u Idea n Save and resume a user grid session n Look and feel of a windows desktop Implementation n Roaming Access Server and Clients n Java Web Services (Portability) Integration of Tools n Job submission wizard n Job monitoring dialog n Grid. Explorer dialog n Grid. Commander dialog M. Kunze, NEC 2003, Varna

User Login Dialog M. Kunze, NEC 2003, Varna

Grid Commander Tool M. Kunze, NEC 2003, Varna

Grid Explorer Tool M. Kunze, NEC 2003, Varna

Job Submission Dialog M. Kunze, NEC 2003, Varna

Job Monitoring/ Visualisation Plugin M. Kunze, NEC 2003, Varna

Cross. Grid Testbeds Various instances: Production, Validation, Test Example sites: Valencia Go. G farm, Santander (Grid. Wall), FZK M. Kunze, NEC 2003, Varna

Testbed Monitoring Mapcenter grid monitoring framework. Mapcenter was developed by Data. Grid and adapted to Cross. Grid. http: //mapcenter. lip. pt M. Kunze, NEC 2003, Varna

Production Resource Broker Statistics Graphics with RB statistics available from Mapcenter Total users 61 Submitted 2903 Accepted 2716 Matching 2554 Sent by JSS 2514 Jobs run 2355 Jobs done 2299 EDG 1. 4 M. Kunze, NEC 2003, Varna

Production RB Statistics (continued) 187 Not accepted Submitted 2903 Accepted 2716 Matching 2554 Sent by JSS 2514 Jobs run 2355 Jobs done 2299 162 Matching failures 40 Not submitted 159 Didn’t run 56 Didn’t reach the end EDG 1. 4 u 604 Jobs failed Most of the failures are related with: n Authorization n Matchmaking n Testbed sites M. Kunze, NEC 2003, Varna

The Future M. Kunze, NEC 2003, Varna

1980 s: Internet 1990 s: Web 2000 s: Grid u Where do we need to get to ? n n n u u Applications to support an “e-society” (“Cyber-Infrastructure”) A Grid infrastructure which hides the complexities from the users (“Invisible Computing”) A powerful and flexible network infrastructure (GEANT 2) Where do we need to invest ? n Applications targeted at realistic problems in “e-science” n Prototypes of Grid infrastructures n Maintain and improve the GEANT network EU FP 6 proposal: EGEE n Vision: Create European e-Infrastructure n 70 Partners, 31. 5 MEuro n Start: April 2004 M. Kunze, NEC 2003, Varna