Report on GLUE activities Flavia Donno INFNPisa EDG

Report on GLUE activities Flavia Donno, INFN-Pisa EDG Retreat 27 -28 August 2002 Flavia Donno, INFN-Pisa 1

Summary u Resource Discovery and GLUE schema u Authorization services from VO LDAP to VOMS u Common software deployment procedures u IST 2002 and SC 2002 joint EU-US demos Interoperability : between Grid domains for core grid services, coexistence of optimization/collective services 27 -28 August 2002 Flavia Donno, INFN-Pisa 2

Resource Discovery and GLUE schema Computing Resources Structure Description Entry point into queueing system Container grouping subcluster or nodes Homogeneous collection of nodes Physical computing nodes 27 -28 August 2002 Flavia Donno, INFN-Pisa 3

Resource Discovery and GLUE schema CE Schema implementation status u u u Logical structure seems to have reached agreement and is stable Recently added file system, versioning Interoperability: EDG needs one CE working on a cluster made of a single Subcluster Information about Hardware in the subcluster Information about Queue Status are in the CE Information providers have to provide Subcluster and CE info LDAP implementation defined, agreed and released! Glue Schema will be included in Globus MDS 2. 2 (September 15 th) Globus Info Providers fill the host level info EDG Info providers filling Subcluster and CE are needed !! When ? Is the 31 st of August confirmed ? It might be necessary to include the EDG info providers in VDT This implementation can be tested with MDS 2. 1 27 -28 August 2002 Flavia Donno, INFN-Pisa 4

Resource Discovery and GLUE schema SE Schema implementation status u u Logical structure almost agreed Missing agreement on how to model connection between storage services and file system LDAP prototype implementation set up Questions to EDG for release 2: n n Do we need to model multiple instances of Storage Service on an SE ? Multiple VO etc. . . Could it be possible to remove the SE concept and introduce the 'storage services' insisting on the same hardware or 'Storage Device‘? 27 -28 August 2002 Flavia Donno, INFN-Pisa 5

Resource Discovery and GLUE schema GIIS structure proposal for the GLUE testbed mds-vo-name=glue mds-vo-name=Data. TAG u mds-vo-name=i. VDGL The US side set up the top-level GIIS 27 -28 August 2002 Flavia Donno, INFN-Pisa 6

GIIS giis. ivdgl. org Gatekeeper: Padova-site mds-vo-name=glue i. VDGL Data. TAG GIIS giis. ivdgl. org mds-vo-name=ivdgl-glue GIIS edt 004. cnaf. infn. it Mds-vo-name=‘Datatag’ LSF Resource Broker Gatekeeper: US-CMS Gatekeeper grid 006 f. cnaf. infn. it Condor Gatekeeper: US-ATLAS Gatekeeper edt 004. cnaf. infn. it WN 1 edt 001. cnaf. infn. it WN 2 edt 002. cnaf. infn. it Computing Element-1 PBS Gatekeeper LSF Computing Element -2 Fork/pbs hamachi. cs. uchicago. edu dc-user. isi. edu rod. mcs. anl. gov Job manager: Fork 27 -28 August 2002 Flavia Donno, INFN-Pisa 7

Resource Discovery and GLUE schema GLUE Schema testbed on-going activity Cnaf/Bologna site status - GIIS vo=Datatag edt 004. cnaf. infn. it - - GRIS grid 006 f. cnaf. infn. it - - - EDG 1. 1. 14, MDS 2. 1 Glue Schema CE v 1. 0 Upgrade to edg 1. 2 LDAP test (no EDG software, no Globus) gummo. cnaf. infn. it - - EDG 1. 1. 14, MDS 2. 1 Glue Schema CE v 1. 0, Glue Schema SE v 0. 1 upgrade to edg 1. 2 Glue Schema CE v 1. 0, Glue Schema SE v 0. 1 Configure RB EDG 1. 2 in edt 003. cnaf. infn. it Modify the RB to query the Glue Schema Install EDG/WP 4 Information Provider Install SE EDG 1. 2 and Glue Schema SE v 0. 1 27 -28 August 2002 Flavia Donno, INFN-Pisa 8

Resource Discovery and GLUE schema GLUE Schema testbed on-going activity u US side: n n n Top level GIIS with Glue schema CE v 1. 0 GRIS with Glue Schema CE v 1. 0 and MDS 2. 1 and static info provider to be upgraded to MDS 2. 2 and Globus Info Provider Node with Glue Schema for SE ready for testing. 27 -28 August 2002 Flavia Donno, INFN-Pisa 9

Resource Discovery and GLUE schema R-GMA implementation and interoperability with MDS u u The Glue UML model is general and can have different implementations: LDAP, Relational DB, XML etc. Schema implementation in R-GMA : n n n implement the Glue UML model in R-GMA and test in the Glue testbed, with EDG WP 3 adapt the info providers to fill the relational table, this should be zero or very low impact Does MDS need to interoperate with R-GMA ? 27 -28 August 2002 Flavia Donno, INFN-Pisa 10

Resource Discovery and GLUE schema People and HW resources PEOPLE: • Data. TAG/WP 4. 2 people : INFN (1. 5 fte) Coordinator: INFN CNAF: Cristina Vistoli (40%) - Cristina. Vistoli@cnaf. infn. it Sergio Andreozzi (100%) - Sergio. Andreozzi@cnaf. infn. it Vicenzo Ciaschini (20%) - Vicenzo. Ciaschini@cnaf. infn. it Hardware Resources: Local HW at CNAF/Padova Local HW at ISI i. VDGL Contacts: B. Tierney, J. Schopf, E. Deelman, C. Kesselman, A. Shoshani EDG Contacts: M. Sgaravatto, P. Kunstz, J. Gordon, O. Barring 27 -28 August 2002 Flavia Donno, INFN-Pisa 11

Authorization services from VO LDAP to VOMS • EDG Tools proposed for use to i. VDGL collaborators • VO LDAP server • VO Group. MAN • edg-mkgridmap • VOMS under study by i. VDGL collaborators. • FNAL – Working on KCA/X 509 • Becoming aware of LCAS/LCMAPS 27 -28 August 2002 Flavia Donno, INFN-Pisa 12

Authorization services People and HW resources PEOPLE: • Data. TAG/WP 4. 2 people : INFN (2. 5 fte) Coordinator: INFN CNAF: Roberto Cecchini (30%) Luca Dell’Agnello (30%) Fabio Spataro (100%) Vicenzo Ciaschini (80%) Hardware Resources: Local HW at CNAF - Roberto. Cecchini@fi. infn. it - Luca. Dell. Agnello@cnaf. infn. it - Fabio. Spataro@cnaf. infn. it - Vicenzo. Ciaschini@cnaf. infn. it i. VDGL Contacts: Doug Olson EDG Contacts: Roberto Cecchini 27 -28 August 2002 Flavia Donno, INFN-Pisa 13

Common software deployment procedures E Goals: Ö Contribute to a distribution that: DG/W P 4 Is EDG and VDT compliant (can be included in VDT/EDG) Includes main EDG and VDT components Allows for interoperation of testbeds Enforces common flexible configuration rules Plan: Done and installed on GLUE TB ? Distribute EDG CA Done ? Distribute EDG Replica Catalogue with GSI ? Distribute UI, RB, JSS, L&B as “standalone” components In progress (End of September, edg-release 1. 2 + new schema) ? ? Distribute information providers if requested Propose a configuration strategy to both VDT and EDG In progress Work on a common EDG-VDT versioning strategy Helping experiments with software distribution (CMS/ATLAS PACMAN distribution) 27 -28 August 2002 Flavia Donno, INFN-Pisa In progress 14

EDG Structure Many light UIs Few heavy RBs UI RB Many Ces and SEs SE JSS UI RB CE CE SE JSS SE CE UI CE SE One LB, IS and RC LB 27 -28 August 2002 IS RC Flavia Donno, INFN-Pisa 15

VDT Structure Many VDT Clients UI RB Many VDT Servers SE SE One LB, IS and RC CE CE LB SE CE JSS UI RB IS RC JSS 27 -28 August 2002 Flavia Donno, INFN-Pisa 16

Common software deployment procedures People and HW resources PEOPLE: Data. TAG/WP 4. 3 people : INFN (1. 5 fte) Coordinator: INFN Milano: INFN Padova: Flavia Donno (20%) David Rebatto (30%) Luca Vaccarossa (50%) Sergio Fantinel (50%) Hardware Resources: Local HW in Milan Local HW in Padova HW in US 27 -28 August 2002 - Flavia. Donno@pi. infn. it - David. Rebatto@mi. infn. it - Luca. Vaccarossa@mi. infn. it - Sergio. Fantinel@lnl. infn. it i. VDGL Contacts: Alain Roy, Saul Youseff, Jerry Gieraltowski EDG Contacts: Enrico Ferro Flavia Donno, INFN-Pisa 17

Data Movement and Management Ö It is part of GLUE phase II. Ö Setting up of working groups in progress with involvement with EDG/WP 2. Ö The existing collaboration between the Globus Team and EDG/WP 2 is a good starting point. Ö Experiment requirements drive the effort. Data. Tag / i. VDGL People: Nikolai Smirnov, David Rebatto, Mike Wilde EDG Contacts: P. Kunstz 27 -28 August 2002 Flavia Donno, INFN-Pisa 18

Joint EU-US grid demos IST 2002 SC 2002 4 -6 November, Copenaghen 16 -22 November, Baltimore Goals: n n Basic collaboration between European and US grid projects Interoperability between grid domains for applications submitted by users from different virtual organizations Controlled use of shared resources subject to agreed policy Integrated use of heterogeneous resources from i. VDGL and EDG testbed domains Infrastructure: ? WEB site: http: //www. ivdgl. org/demo Hypernews: http: //atlassw 1. phy. bnl. gov/Hyper. News/get/intergrid. html Mailing list: igdemo@datagrid. cnr. it Archive: http: //web. datagrid. cnr. it/hypermail/archivio/igdemo ? GLUE testbed with common schema ? VO (Data. TAG and i. VDGL) LDAP Servers in EU and US ? PACMAN cache with software distribution (Data. TAG or i. VDGL) ? Planning document outline 27 -28 August 2002 Flavia Donno, INFN-Pisa 19

Joint EU-US grid demos ? GLUE testbed with Common GLUE schema and authorization/authentication tools. Ø Ø Ø EDG 1. 2 + extensions, VDT 1. 1. 3 + extensions “Old” authentication/authorization tools : VO LDAP servers, mkgridmap, etc. We rely on the availability of the new GLUE schema, RB and Information Providers ? Concentrate on visualization: CMS/GENIUS, ATLAS/GRAPPA, EDG/MAPCENTER (EDG/WP 7), Farm Monitoring tools (EDG/WP 4), i. VDGL/GANGLIA, Data. TAG/NAGIOS Ø Ø Ø Use WEB portals for job submission (CMS/Genius, ATLAS/Grappa) Provide a World Map of the sites involved (EDG-WP 7/Map. Center, Data. TAG/Nagios) Monitor job status and statistics (EDG-WP 4, Data. TAG/Nagios). In Nagios implemented “top users”, “top applications”, “total resources”, “averages over time”, “per VO”, … Monitor farms (EDG/WP 4, Data. TAG/Nagios) Developing plugins/sensors for WP 1/L&B info (only on EDG part) Services Monitoring (EDG/WP 4, Data. TAG/Nagios [using MDS info]) ? CMS and ATLAS demo Ø Ø Ø ATLAS simulation jobs, GRAPPA modified to use both RB or explicit resources Pythia & CMSIM simulation jobs submitted to intercontinental resources with IMPALA/BOSS interfaced to VDT/MOP, EDG/JDL, Genius portal. Definition of the application demos in progress. 27 -28 August 2002 Flavia Donno, INFN-Pisa 20

Joint EU-US grid demos 27 -28 August 2002 Flavia Donno, INFN-Pisa 21

Joint EU-US grid demos Hardware definition in progress. First simple sumission tests in progress. Work on infrastructure and application demos as much as possible before the middle of September. Joint Task force in Chicago 23 -30 September, 2002 for first Data. TAG/i. VDGL and ATLAS rehearsal. 27 -28 August 2002 Flavia Donno, INFN-Pisa 22

Suggestions to EDG for release 2 • Work on installation/configuration issues • Relocatability • Configuration environemnt • Service configuration and startup • Make sure “vanilla functionality” of core services is kept. • Security system with VO support • Information System • Submission services such as the Globus Gatekeeper, Job. Manager, Condor-G, . . . • Schedule with time introduction of GLUE components in EDG. Active collaboration. • Make service interfaces clear so to allow for easy software sharing and integration whenever possible. • Discuss architecture design for Job scheduling, Data Management and Storage Services. • . . . 27 -28 August 2002 Flavia Donno, INFN-Pisa 23