The EU Data Grid Testbed The European Data
The EU Data. Grid Testbed The European Data. Grid Project Team http: //www. eu-datagrid. org Data. Grid is a project funded by the European Union Grid Tutorial 9/3/2021 – n° 1
Contents u Grid Services u Hardware Components of an EDG Testbed u Code distribution policy Grid Tutorial - 9/3/2021 – Testbed - n° 2
Main EDG Grid Services u Authentication & Authorization u Job submission service n u u Resource Broker Replica Management n EDG-Replica-Manager n Mass storage system support Logging & Bookkeeping Grid Tutorial - 9/3/2021 – Testbed - n° 7
EDG Main Logical Machine Types u User Interface (UI) u Information u Computing Service (IS) Element (CE) n Frontend Node n Worker Nodes (WN) u Storage u Replica Element (SE) Catalog (RC, RLS) u Resource Broker (RB) Grid Tutorial - 9/3/2021 – Testbed - n° 8
Main Services per Machine Type (release 1. 4) Daemon UI IS CE WN SE RC RB Globus Gatekeeper - - - Replica Catalog - - - Grid. FTP - - Globus MDS - - - - Info-MDS - - - - Broker - - - Job submission - - - Information Index - - - Logging & Bookkeeping - - - Local Logger - - CRL Update - - Grid mapfile Update - - RFIO - - - GDMP - - - (frontend) Grid Tutorial - 9/3/2021 – Testbed - n° 9
Main Services per Machine Type (release 2. 0) Daemon UI IS CE WN SE RLS RB Globus Gatekeeper - - - RLS-LRC - - - RLS-RMC - - - Grid. FTP - - R-GMA - - - R-GMA GOUT - - - R-GMA GIN - - - Broker (Network server, job control) - - - Condor. G Job submission - - - Logging & Bookkeeping - - - Local Logger - - - CRL Update - - Grid mapfile Update - - RFIO - - - EDG-SE - - - (frontend) Grid Tutorial - 9/3/2021 – Testbed - n° 10
A Simple Testbed Configuration “CLOSE” Computing Element 1 Storage Element 1 User Interface Resource Broker Replica Catalog Information Service Storage Element 2 “CLOSE” Computing Element 2 Grid Tutorial - 9/3/2021 – Testbed - n° 11
Testbeds Gri. Dis dissemination testbed n Small tutorial and demo testbed n has access to application testbed n Cross. Grid sites take part Application Testbed: End-user Applications n Software: Stable, certified release (EDG 1. 4. x) Certification Testbed: Extended, Detailed Testing n Software: Tagged release n State: Starting…; Collaboration with Testing Group/LCG. Development Testbed: Integration & Evaluation of SW n Software: Current tagged release + new pkg. New tagged release. n State: Active use; 5 sites involved. Development Machines: Testing of Middleware in Isolation n Software: Bleeding edge versions. n State: Varied; under control of middleware work packages. Grid Tutorial - 9/3/2021 – Testbed - n° 12
Testbed Overview Students Terminals enter Grid UI Resource Broker Computing Elements Storage Elements Information Index http: //marianne. in 2 p 3. fr http: //web. datagrid. cnr. it/Gri. Dis. WP 1. html Grid Tutorial - 9/3/2021 – Testbed - n° 13
Data. Grid testbeds Application testbed: More than 1000 CPUs 5 Terabyte of storage EDG sw installed at more than 40 sites Grid Tutorial - 9/3/2021 – Testbed - n° 14
Grid Tutorial - 9/3/2021 – Testbed - n° 15
Example IS Content Site: NIKHEF ------------------------CE tbn 09. nikhef. nl: 2119/jobmanager-pbs-qlong: - PBS queue "qlong" with 96 hours time limit - Software installed: CMS-1. 0. 2 ATLAS-1. 3. 0 ALICE 3. 07. 01 LHCb-1. 1. 1 IDL-5. 4 NIKHEF D 0 MCC-0. 1 -1 - There are 0 jobs running and 0 waiting, with 16 CPUs free Close SE tbn 03. nikhef. nl with mount point /flatfiles -------------------------CE tbn 09. nikhef. nl: 2119/jobmanager-pbs-qshort: - PBS queue "qshort" with 240 minutes time limit -----------------SE tbn 03. nikhef. nl close to 2 CEs: -tbn 09. nikhef. nl: 2119/jobmanagerpbs-qshort -tbn 09. nikhef. nl: 2119/jobmanagerpbs-qlong - VOs supported: alice atlas biomedical cms earthob lhcb iteam - gridftp on port 2811 - Software installed: CMS-1. 0. 2 ATLAS-1. 3. 0 ALICE 3. 07. 01 LHCb-1. 1. 1 IDL-5. 4 NIKHEF D 0 MCC-0. 1 -1 - rfio on port 3147 - There are 0 jobs running and 0 waiting, with 16 CPUs free Close SE tbn 03. nikhef. nl with mount point /flatfiles - 31744 Mb of free space - file Grid Tutorial - 9/3/2021 – Testbed - n° 16
EDG Software Distribution u All software available as source & binary RPMs u Binaries for Red. Hat 6. 2 and Red. Hat 7. 3 u > 600 packages including u n Complete globus distribution n EDG packages (~50 packages) n Support tools (perl, ant, jdk, …) Pre-packaged for different machine types Grid Tutorial - 9/3/2021 – Testbed - n° 17
Issues when Adding new Sites to the Testbed u EDG is currently setting-up procedures explaining how to add new sites n n u Need to clarify the “minimum requirements” for a site to become a member of the testbed n n u A number of regular tasks have to be performed by the sites administrators A maximum delay needs to be defined for responding to requests/problems if the testbed is to run efficiently Sites from new countries have to identify/create a supporting CA n u Variations already tested with Taiwan and Romania Step-by-step instructions produced which we expect to become simpler over time Since CAs need mutual trust this could lead to an explosion of inspection activities Some tasks will fall on the people responsible for managing the VOs n HEP experiment secretariats already perform some level of authentication of their institutes and members. How an we get some leverage from this? Grid Tutorial - 9/3/2021 – Testbed - n° 18
Summary u Logical machine types of an EDG Testbed u Mapping of services to logical machines u Example and current EDG Testbed configuration u Code distribution strategy Grid Tutorial - 9/3/2021 – Testbed - n° 19
Further Information u EDG Testbed homepage: http: //marianne. in 2 p 3. fr/ u Fabric management: http: //hep-proj-grid-fabric. web. cern. ch/hep-proj-grid-fabric/ u LCFG on EDG Testbed information: http: //www. lnl. infn. it/datagrid/wp 4 -install/ http: //datagrid. in 2 p 3. fr/distribution/datagrid/wp 4/installation/doc/ Grid Tutorial - 9/3/2021 – Testbed - n° 20
- Slides: 16