EMEA HPTC Virtual Team HighPerformance Computing on Intel

EMEA HPTC Virtual Team High-Performance Computing on Intel® Architecture EUROGRID Overview Klaus-Dieter Oertel Intel Gmb. H, Solutions & Software Group klaus-dieter. oertel@pallas. com 2 nd EUROGRID Workshop @ CGW 03 Cracow, Oct 29, 2003 *Other brands and names are the property of their respective owners © Copyright 2002 -2003 Intel Corporation

EMEA HPTC Virtual Team High-Performance Computing on Intel® Architecture Outline § § EUROGRID and Intel EUROGRID Goals Technical Basis: UNICORE EUROGRID Results *Other brands and names are the property of their respective owners © Copyright 2002 -2003 Intel Corporation

EMEA HPTC Virtual Team High-Performance Computing on Intel® Architecture EUROGRID and Intel § Pallas HPC group has developed the UNICORE Client and has participated in EUROGRID § Intel has acquired the Pallas HPC group (Sep 12) § Pallas HPC will form the Cologne Software Lab (CGSL) within the Software Solutions Group (SSG) § CGSL’s Grid Mission: Make Intel a leading player in Grid technology § CGSL takes Pallas role in EUROGRID *Other brands and names are the property of their respective owners © Copyright 2002 -2003 Intel Corporation

EMEA HPTC Virtual Team High-Performance Computing on Intel® Architecture EUROGRID Vision Build a European Grid infrastructure that gives users a seamless, secure access to High Performance Computing resources and that advances computational science in Europe *Other brands and names are the property of their respective owners © Copyright 2002 -2003 Intel Corporation

EMEA HPTC Virtual Team High-Performance Computing on Intel® Architecture EUROGRID Goals § European HPC centers setup GRID testbeds – Seamless access to HPC resources, Multi-site jobs § Interfaces for important applications – Bio-/Chemical, Meteo, CAE § Industrial applications and GRID usage scenarios – ASP services for engineers – Intranet solutions for distributed organizations § Development of UNICORE extensions – E. g. resource broker, efficient data transfer, . . . § Contribute to Global GRID development *Other brands and names are the property of their respective owners © Copyright 2002 -2003 Intel Corporation

EMEA HPTC Virtual Team High-Performance Computing on Intel® Architecture EUROGRID Partners HPC Centers § FZ Jülich (D, Admin. Coord. ) § CSCS Manno (CH) § ICM Warsaw (PL) § IDRIS Paris (F) § Univ Bergen (N) § Univ Manchester (UK) Users § DWD (D) § EADS (F) § T-Systems (D) (Assistant Partner) Integration § Pallas (D) (Project Coordinator) § FLE (UK) (Assistant Partner) Duration: Nov 2000 – Jan 2004 Volume: 33 person years, 3. 5 MEuro cost funded by European Commission Grant No. IST– 1999– 20247 *Other brands and names are the property of their respective owners © Copyright 2002 -2003 Intel Corporation

EMEA HPTC Virtual Team High-Performance Computing on Intel® Architecture EUROGRID Geography *Other brands and names are the property of their respective owners © Copyright 2002 -2003 Intel Corporation

EMEA HPTC Virtual Team High-Performance Computing on Intel® Architecture The Technical Basis: UNICORE § Vertical Grid System for (HPC) Compute Grids § Graphical client § Sophisticated security concept § Plugins for existing applications (Nastran, Fluent, CPMD, . . . ) § Object-oriented architecture, implemented in Java *Other brands and names are the property of their respective owners © Copyright 2002 -2003 Intel Corporation

EMEA HPTC Virtual Team High-Performance Computing on Intel® Architecture UNICORE Client § § § § Graphical Interface to UNICORE Grids Open Source under Pallas Community License Job Preparation, Monitoring and Control Complex Workflows File Management Certificate Handling Plug-in Interface *Other brands and names are the property of their respective owners © Copyright 2002 -2003 Intel Corporation

EMEA HPTC Virtual Team High-Performance Computing on Intel® Architecture UNICORE Job Model § Directed acyclic graph of – tasks (computational or data transfer) – sub–jobs (to be executed at another system) – temporal dependencies § Attach resource requests to tasks – – capacity (CPU time, disk, …) capability (MPI– 2, profiling, …) can do static tests of jobs can do resource brokering … LM–Job T 3 E– 1200 Import Model Data Execute LM Model If Success Transfer Data Notify user Process–LM Zampano Postprocess *Other brands and names are the property of their respective owners © Copyright 2002 -2003 Intel Corporation Export Data

EMEA HPTC Virtual Team High-Performance Computing on Intel® Architecture UNICORE Architecture § § § Client can run anywhere Gateway as single point of entry NJS incarnates and schedules jobs UDB (User Database) contains user login information IDB (Incarnation Database) contains resource information and job incarnation rules TSI (Target System Interface) interfaces to local batch system Client(s) Insecure Internet Gateway UDB Network UNICORE Network Job Supervisor IDB TSI Cray NQE PBSPro *Other brands and names are the property of their respective owners © Copyright 2002 -2003 Intel Corporation IDB

EMEA HPTC Virtual Team High-Performance Computing on Intel® Architecture UNICORE – Security Model § User authentication – permanent user certificate – stored in protected keystore on the user PC/workstation – certificate passed downstream as part of the job § Job authentication – job description is signed with the user’s private key – downstream sites can check whether job is authentic – no trust required between UNICORE sites § Software authentication – client and gateway use certificates for mutual authentication – NJS can use a special “user certificate” for authentication to secondary sites or take the role of a client § User authorization – performed by the UNICORE server according to local procedures (mostly based on local userid) *Other brands and names are the property of their respective owners © Copyright 2002 -2003 Intel Corporation

EMEA HPTC Virtual Team High-Performance Computing on Intel® Architecture UNICORE Plugins – Extend the User Interface § UNICORE client has a plugin interface – – Create GUIs that support important applications Plugin simplifies data entry for application Plugin can support application-specific resources Plugin constructs (complicated) workflows automatically § Plugin will use UNICORE Client to – Submit the application job – Monitor and control the application job § Helpful features – End-users concentrate on applications – Extended consistency checks § Existing Plugins – CPMD molecular dynamics code – Star-CD, Fluent, Nastran, … *Other brands and names are the property of their respective owners © Copyright 2002 -2003 Intel Corporation

EMEA HPTC Virtual Team High-Performance Computing on Intel® Architecture Example: Gaussian 98 Plugin *Other brands and names are the property of their respective owners © Copyright 2002 -2003 Intel Corporation

EMEA HPTC Virtual Team High-Performance Computing on Intel® Architecture Results of EUROGRID § Setup and operation of European Grids Bio-GRID, Meteo-GRID, HPC-GRID § Industrial use of Grids started CAE-GRID: EADS (Airbus), T-Systems § Grid-enabling of many applications via plugin interface § Development and integration of new technology Alternative File Transfer, Resource Broker, Billing Services, Cost Estimation, Application Coupling, Interactive Access *Other brands and names are the property of their respective owners © Copyright 2002 -2003 Intel Corporation

EMEA HPTC Virtual Team High-Performance Computing on Intel® Architecture Bio-GRID § biogrid. icm. edu. pl § GRID for biomolecular simulations § Interfaces to existing biological and chemical codes *Other brands and names are the property of their respective owners © Copyright 2002 -2003 Intel Corporation

EMEA HPTC Virtual Team High-Performance Computing on Intel® Architecture Bio-GRID: Applications § § § § § ADF: CPMD: Gamess: Gaussian 98: Molcas: Molpro: Turbomole: Amber: CHARMM: Gromos 96: IDRIS FZJ ICM, FZJ, Uo. M, Parallab, IDRIS FZJ, Parallab FZJ ICM, FZJ, Uo. M, Parallab, IDRIS ICM, Parallab ICM *Other brands and names are the property of their respective owners © Copyright 2002 -2003 Intel Corporation

EMEA HPTC Virtual Team High-Performance Computing on Intel® Architecture Bio-GRID: Plugins § Gaussian 98 § Amber 6. 0 § PDB search plugin – Search mirrors of the PDB (Protein Data Bank) database and choose database providing fastest transfer – Visualize results using Jmol, JMV (Java Molecular Viewer), Ras. Mol/Ras. Win, or user defined application § List Jobs Plugin – Checks all available Vsites – Gather information about user jobs and their status § File Filter Plugin – Access files in job directory of running jobs *Other brands and names are the property of their respective owners © Copyright 2002 -2003 Intel Corporation

EMEA HPTC Virtual Team High-Performance Computing on Intel® Architecture Meteo-GRID § Develop a relocatable version of DWD‘s weather prediction model § Goal: ‘Weather predictionon-demand‘ as an ASP solution *Other brands and names are the property of their respective owners © Copyright 2002 -2003 Intel Corporation

EMEA HPTC Virtual Team High-Performance Computing on Intel® Architecture CAE-GRID § Coupled simulations of aircrafts (e. g. structure and electromagnetics) § Goal: internal HPC portal for EADS engineers *Other brands and names are the property of their respective owners © Copyright 2002 -2003 Intel Corporation

EMEA HPTC Virtual Team High-Performance Computing on Intel® Architecture CAE-GRID § CAE-GRID at EADS is not publicly available § EADS has developed a plug-in for acoustic & electromagnetic applications – Combining 4 applications in one job *Other brands and names are the property of their respective owners © Copyright 2002 -2003 Intel Corporation

EMEA HPTC Virtual Team High-Performance Computing on Intel® Architecture HPC-GRID § Demonstration of a European HPC GRID testbed § Sharing of competence and know-how § Agreement on certification, access policies, . . . CRAY T 3 E 900 (32 PE) NEC SX 4 B/2 A Linux Cluster (4 PE) FZJ Linux Intel Cluster (36 PE) CRAY T 3 E - 600 (512 PE) CRAY T 3 E - 1200 (512 PE) SGI O 2000 (128 PE) CRAY T 3 E - 1200 (816 PE) FUJITSU VPP 300 (8 PE) SGI O 2000 (128 PE) SGI O 3000 (256 PE) IBM SP 3 (8 PE) NEC SX 5 cluster (40 PE) IBM Power 4 (256 PE, 1. 3 TFLOPS) COMPAQ Linux Cluster (24 PE) *Other brands and names are the property of their respective owners © Copyright 2002 -2003 Intel Corporation

EMEA HPTC Virtual Team High-Performance Computing on Intel® Architecture Alternative File Transfer § Scenario: multi-site job where a huge amount of data has to be transferred between sites § UNICORE internal transfer mechanism is based on “slow” Java streams § UNICORE server has been extended to use different file transfer mechanisms e. g. rcp § Use AFTGrid. FTP in EUROGRID which is: Grid. FTP as an alternative file transfer mode *Other brands and names are the property of their respective owners © Copyright 2002 -2003 Intel Corporation

EMEA HPTC Virtual Team High-Performance Computing on Intel® Architecture AFTGrid. FTP § Grid. FTP: – Developed in the Globus project – FTP (file transfer protocol) with new features for grid applications such as parallel and partial file transfer – Requires Globus proxy certificates for operation § AFTGrid. FTP is transparently used for transfers if – The job contains a Transfer Task – The sites are set-up for AFTGrid. FTP – The Client has loaded the proxy certificate plugin • Generates a proxy certificate based on the user certificate • The proxy certificate is provided to Grid. FTP in the job directory • Proxy certificate plugin has been developed in the GRIP (Grid Interoperability) project *Other brands and names are the property of their respective owners © Copyright 2002 -2003 Intel Corporation

EMEA HPTC Virtual Team High-Performance Computing on Intel® Architecture AFTGrid. FTP § Performance results depend on – File size – Form of the file: compressed or uncompressed – Number of parallel streams § AFTGrid. FTP can be 15 times faster than default UNICORE Java streams *Other brands and names are the property of their respective owners © Copyright 2002 -2003 Intel Corporation

EMEA HPTC Virtual Team High-Performance Computing on Intel® Architecture Resource Broker § User specifies job with resource requests (e. g. a certain application resource) § Send job to Broker site which might resolve to a hierarchy of Broker sites § Broker site replies with a list of offers where to run the job § Each offer is a ticket which provides a time-limited service guarantee § Accepting one offer includes the ticket in the job § Submit the job to the selected site § Valid ticket in the job indicates the acceptance of the Qo. S offer *Other brands and names are the property of their respective owners © Copyright 2002 -2003 Intel Corporation

EMEA HPTC Virtual Team High-Performance Computing on Intel® Architecture Resource Broker *Other brands and names are the property of their respective owners © Copyright 2002 -2003 Intel Corporation

EMEA HPTC Virtual Team High-Performance Computing on Intel® Architecture Resource Broker *Other brands and names are the property of their respective owners © Copyright 2002 -2003 Intel Corporation

EMEA HPTC Virtual Team High-Performance Computing on Intel® Architecture ASP Extensions - Billing Services § Software accounting system on the server – Generate accounting data (userid, used software, hostname, cpu time, …) – Record the accounting data in a database – Independent of UNICORE § Billing plugin – Retrieve the accounting data and corresponding prices – Create and provide the billing data to the user *Other brands and names are the property of their respective owners © Copyright 2002 -2003 Intel Corporation

EMEA HPTC Virtual Team High-Performance Computing on Intel® Architecture ASP Extensions - Billing Services *Other brands and names are the property of their respective owners © Copyright 2002 -2003 Intel Corporation

EMEA HPTC Virtual Team High-Performance Computing on Intel® Architecture ASP Extensions – Cost Estimation § Generic cost estimation is not possible for CAE applications in general § The LS-Dyna application supports cost estimation § LS-Dyna plugin has been extended for cost estimation – A pre-job starts the LS-Dyna simulation for some calculation cycles – The necessary simulation time is estimated based on extrapolation – Hardware and software costs are calculated and displayed in the plugin *Other brands and names are the property of their respective owners © Copyright 2002 -2003 Intel Corporation

EMEA HPTC Virtual Team High-Performance Computing on Intel® Architecture ASP Extensions – Cost Estimation *Other brands and names are the property of their respective owners © Copyright 2002 -2003 Intel Corporation

EMEA HPTC Virtual Team High-Performance Computing on Intel® Architecture Application Coupling § A plug-in has been developed which supports the execution of CORBA coupled applications under UNICORE – Query the status of a CORBA Name Server – Or remove the context references – Start the coupled applications § The applications exchange data independently from UNICORE using only the CORBA framework *Other brands and names are the property of their respective owners © Copyright 2002 -2003 Intel Corporation

EMEA HPTC Virtual Team High-Performance Computing on Intel® Architecture Application Coupling *Other brands and names are the property of their respective owners © Copyright 2002 -2003 Intel Corporation

EMEA HPTC Virtual Team High-Performance Computing on Intel® Architecture Interactive Access § An Interactive Access plugin has been developed for – Interactive execution of pre-defined short commands – Interactive steering of batch jobs § The job directory can be inspected at run time § The user can reconnect to running interactive batch jobs *Other brands and names are the property of their respective owners © Copyright 2002 -2003 Intel Corporation

EMEA HPTC Virtual Team High-Performance Computing on Intel® Architecture Interactive Access *Other brands and names are the property of their respective owners © Copyright 2002 -2003 Intel Corporation

EMEA HPTC Virtual Team High-Performance Computing on Intel® Architecture More information § EUROGRID: www. eurogrid. org § UNICORE: www. unicore. org § Pallas / Intel: www. pallas. com § GRIP: www. grid-interoperability. org *Other brands and names are the property of their respective owners © Copyright 2002 -2003 Intel Corporation

EMEA HPTC Virtual Team High-Performance Computing on Intel® Architecture Questions? *Other brands and names are the property of their respective owners © Copyright 2002 -2003 Intel Corporation