National Grid Initiatives in Italy INFN Grid and

National Grid Initiatives in Italy INFN Grid and Grid. it IGI and c-OMEGA INFN Grid Workshop Padova 18 -20 December 2006 Mirco Mazzucato INFN-Grid Project Manager CNAF Director mirco. mazzucato@pd. infn. it last update 21/12/2021 10: 32 1

Summary § The Italian e-Infrastructure § The national projects: ¨ INFN Grid, FIRB Grid. it, . . . . ¨ The Middleware developments § § last update 21/12/2021 10: 32 The Italian Production Grid The Italian Grid Infrastructure (IGI) Association The c-Omega consortium for Open Middleware support in commercial take up Conclusions 2

The INFN-GRID Project § February 2000: The Board of Directors approve the INFN Grid project § Large size : 20 Italian Sites, ~100 people, ~ 50 FTE’s § Collaboration between physicists, sw engineers, computer professionals and computer scientists (CS Dep. of Universities of VE, PD, BO, CT, TO, …), CNR, and Industries § Datamat SPA and Nice major contributors of the joint developments § Focused on the preparation of the INFN LHC comp. infrastructure § …but with the goal of developing a new set of “standard” services and protocols to allow resource sharing in different administrative domains as the server Http and HTML are at basis of information sharing § taking into account the requirements of other sciences ¨ Biology (PD) and Earth Observation(Esrin-ESA-Frascati) § INFN Grid has been and is the national container for INFN to coordinate the contribution to all EU and International Grid projects and to the GGF standardization § Early R&D in Italy include work done in ISUFI (University of Lecce) § ->see S-PACI § INFN Grid consider to have accomplish his main goal: All LHC experiments and many othe VOs rely on grids for their daily work last update 21/12/2021 10: 32 3

The Reference Service Stack has been and is the EGEE g. Lite CLI API Access Information & Monitoring Authorization Auditing Authentication Network Monitoring Security Services Metadata Catalog File & Replica Catalog Storage Element Data Movement Information & Monitoring Services Policies Job Provenance Package Manager Accounting Computing Element Workload Management Data Management last update 21/12/2021 10: 32 Service Discovering Workload Mgmt Services 5

§ MW summary Constant efforts have been put on the general production grid services to be able to: § Routinely support running of thousands of jobs with efficiencies >= 99% § Keep sustainable data transfer rates at Gbps in many site; thanks also to large WLCG efforts § The present Workshop has shown that important steps have been made but also a lot of issues to tackle § § § Deploy CREAM with JSDL and BES in Italy and EGEE Guarantee time performance of all services (WMS…) Guarantee Grid wide VOMS and VO policies enforcement Guarantee robust monitoring and accounting Deploy Storm and SRM 2. 2 Need to guarantee continuity in the task forces work to reach at the same time in a well focused and coordinated way both grid project and experiments objectives § Eliminate artificial barriers while keeping both general and specific scope last update 21/12/2021 10: 32 6

The Italian Grid infrastructure: The FIRB Grid. it Project launched in 2002 within FIRB: Government Fund for the Investments in the Basic Research (Ministry of Education, University and Research (MIUR)) last update: 21/12/2021 10: 32 Mirco Mazzucato Infn -Padova 7

The Grid. it National Project TORINO MILANO PAVIA PADOVA CNR & University HPC, Parallel Programming, Grid computing, Scientific libraries, Data base and knowledge discovery, BOLOGNA GENOVA ->ASSIST INFN & University PISA Grid infratsructure(INFN-Grid, Data. Tag) , e-science applications: Astrophysics, PERUGIA Bioinformatics, Geophysics, … ROMA NAPOLI BARI R&D for a National e. Infrastructure ASI MATERA LECCE CAGLIARI COSENZA Applications of Earth Observation PALERMO CNIT R&D on optical networking Ended : November 2006 last update 21/12/2021 10: 32 8

The Grid. it e-Infrastructure § Grid. it has completed the R&D phase for the National Grid Infrastructure operation and for studying and prototyping the services required by a national Grid Operation Center (GOC) (integrated with the EU Regional Operation Center (ROC)) § Located at CNAF (Bologna) but leveraging contributions from other Italian centers § Has generalized the infrastructure support from INFN to other Sciences § The Grid. it infrastructure is now currently in operation with 9 K jobs/day of more that 20 VOs § The Italian GOC/ROC provide support to all those Italian Research Communities and the operation of the Italian e. Infrastructure also as part of WLCG/EGEE II last update 21/12/2021 10: 32 9

The Italian Production Grid: a sum of grids 3000 CPUs, 500 TB 45 ‘resource centers’: All centers are accessible through Resouce Brokers and 25 sites are registered also in the EGEE/LCG infrastructure 12 sites are accessible through the italian Grid services and the italian top level BDII INFN Grid, SPACI, ENEA, provide the foundation New Enlargment: Recent PONs Tri. Grid, PI 2 S 2 Cybersar, Scope, Cresco should provide important increases and different CPU architectures http: //grid-it. cnaf. infn. it last update 21/12/2021 10: 32 10

The Italian Grid: the SPACI Consortium: a flexible, robust, secure and scalable IT infrastructure Southern Partnership for Advanced Computational Infrastructure 1. 4 Tflops ISUFI/CACT SPACI infrastructure part of the EGEE production Grid Center for Advanced Computing Technologies University of Lecce Director: Prof. Giovanni Aloisio DMA/ICAR MIUR/HPCC Dept. of Mathematics and Applications University of Naples “Federico II” & ICAR (Section of Naples) Director: Prof. Almerico Murli last update 21/12/2021 10: 32 11 Center of Excellence for High Perfomance Computing University of Calabria Director: Prof. Lucio Grandinetti

GEANT boquad. frascati. enea. it The Italian grid: The ENEA infrastructure grid 0007 power 3. frascati. enea. it ID GR EA it-Link N E ga. B Gi EGEE dafne. casaccia. enea. it infocal. trisia. enea. it CNR Tor Vergata Citrix Nfuse Citrix Metaframe WEB(ICA) ICA last update 21/12/2021 10: 32 12

The Central Management Team (CMT) § § § Guarantee Release Distribution and Site Certification in Italy § Enriched g. Lite The CMT is responsible of the certification: dynamically checking the functionalities and configuration of a site services before including it in the Italian production grid. In particular checks: § § § § Information System data consistence Local jobs submission (LRMS) Grid submission with Globus (globus-job-run) Grid submission with the EGEE Resorce Broker Replica. Manager functionalities To certificate a site the CMT uses dedicated grid services located at CNAF Very crucial activities in the next year last update 21/12/2021 10: 33 13

The INFNGRID-3. 0 release = g. Lite 3. 0 + DGAS + GPBOX Managed Core SERVICES EGEE g. Lite 3. 0 RCn in production Grid. it Production Grid: INFNGRID 3. 0 Catalogs g. Lite 3. 1 On RB and VOMS: in preview SCIENTIFIC LINUX 3. 05 One for each VO Tested by application task forces Top level monitoring Catalogs service… RLS LFC RB VOMS Gridice DGAS GPBOX Grid policies last update 21/12/2021 10: 33 My. Proxy Security GIS Authorization Accounting Information System VO: LHC, Bio, CDF, Planck, Compchem… 14

Grid. IT Production Grid: 0 perations Portal § User documentation § site managers documentation § Software repository § Monitoring § Trouble tickets system § Knowledge base http: //grid-it. cnaf. infn. it last update 21/12/2021 10: 33 15

A new key Service badly missing: The DGAS Accounting per user and VO last update 21/12/2021 10: 33 16

An old service badly needing consolidation Grid. ICE Monitoring and Notification last update 21/12/2021 10: 33 17

Another important service: GILDA-The Laboratory to try out Grid. it and EGEE Grid last update 21/12/2021 10: 33 19

§ The long term sustainability of the Italian Grid. it end in 2006 and need to be replaced by a more long term organization to guarantee the sustainability of the current Italian Grid infrastructure and its connection to EU level § The proposal made at MIUR in February 2006 for the future: § The Association for the Italian Grid Infrastructure (IGI) § Originally conceived to provide national coordination of the different pieces of the national e-Infrastructure present in EGEE II § Supported by MIUR § § Quite consensus for the expansion of IGI scope to include all Institutions involved in e-Infrastructure projects in Italy e. g. supercomputing centers and new PON projects infrastructures in view of the set up of the European Grid Initiative Constitutional document in final approval phase by involved institutions and the Minister of University and Research last update 21/12/2021 10: 33 20

The strategy for IGI and objectives § § § Start from a core Scientific Institution already involved in Grid e. Infrastructure projects at EU (EGEE, DEISA. . ) or at national level (PON) Include enlargement and representation within the scope Focus on setting up and operate a common e-Infrastructure for the Italian Science, including main public resources providers: INFN, SPACI, ENEA, ICTP, INAF, Supercomputing Centers, new PON Consortia, Regional Initiatives…. and Users CNR, INAF, INGV, …. § Industry as advisors § Initial phase should allow different grid M/W (EGEE, UNICORE, GLOBUS…) to cohexist and interoperate § IGI should provide a consistent/coordinated Italian interface and strategy towards § EU Grid infrastructure projects, e. IRG and ESFRI, EU EGI § International activities § Should provide support to actvities of a vaste range of Scientific disciplines: Physics, Astrophysics, Biology, Health, Chemics, Geophysics, Economy, Finance, and possibile extensions to other sectors as Civil Protection (GMES), e-Learning, dissemination in Universities and secondary school § Provide the best value for the available money for computing last update 21/12/2021 10: 33 21

Some INFN thoughts on EGI § § The comparison with NRENs in building EGI is very reductive NRENs Geant and DANTE have started when consolidated middleware stacks were available § Emphasis was on operation and inclusiveness § Relevance was completely internal to the research world § No serious attempt made by NRENs to promote and organize an EU leadership in the international internet specification process or related hardware and software developments § § EGI starts when the consolidation of middleware into standard fully specified services is just at its infancy Projects like EGEE III and in future EGI should guarantee, first of all, the consolidation and sustainability of the EU innovation and MW development processes which have started in 2001 with Data. Grid from the Research world and are continuously expanding to the whole EU society § e-Infrastructures are one of the major enabling factor not only for e. Science and ERA but for the whole EU society -> related projects § Europe is currently at the frontier in the world in the movement aiming at boosting the general society productivity through e-Infrastructures last update 21/12/2021 10: 33 23

EGI: Operation vs developments § Operation of an EU e-Infrastructure based on world-wide competitive services if not managed unitarily inside the same organization with the middleware innovation and development process will not be possible § Do not dream on the market. This will come when the world competition will be set and winner players clearly identified § EGI should not leave Europe out of this competition as happened for the WEB based services § § To guarantee current EU excellence, the deployed Services will need to keep evolving towards more functionalities and more standard specifications for a decade This is the necessary condition to make of the EU stacks as g. Lite a competitive reference for NESSI e. g. and in general for the EU commercial, industrial and the e-Government work § EGI has very little value for the European society as a reseller of externally developed core services in continuous evolution § Guaranteeing the continuous 24 x 7 operation of robust services for the Research together with their continuous evolution and solid expertise and support for the extension of their usage to the society is the real challenge for EGI last update 21/12/2021 10: 33 24

EGI constituency § § § Again Geant and DANTE cannot be taken as an appropriate reference because EGI has to guarantee the EU excellence not only in deployment and operation but also in a highly world-wide competitive middleware innovation process and the extension of the e-Infrastructure usage to enhance the general society productivity INFN is convinced that it is difficult for any EU Member States Institution or a simple coordination of those to get the strength, scale and organizational capability of guaranteeing in future the current EU excellence in grid infrastrctures CERN is the only EU champion which thanks to years of expertise in frontier research and developments de facto has and can continue to guarantee in future the EU excellence in the general e -Infrastructures development and operation § As ESA has guaranteed and protected the EU Galileo GPS system development against all competitive alternative actions § Strong argument to support that EGI should be constituted not only by NGI but also by CERN if we want to keep the steady innovative progress we had up to now § CERN has already shown to be able to promote and develop a general e-Infrastructures not only for HEP last update 21/12/2021 10: 33 25

EGI /projects relations § § EGI should guarantee the long term sustainability at the level of excellence of the current EU e-Infrastructures activities This is obtained defining EU (EGI) and national organizations (NGI) capable of rising national and EU funds also through a set of projects in order to guarantee 1. 3. 4. Continuous core middleware services development and generalization process through the input and strict interaction with an increased number of user communities (EGEE N) The operation of a top world class e-Infrastructures for Research based on an EU supported stack (EGEE N) – Supported also in general for the society – Operations of a grid do not in principle require much efforts outside the na The evolution of the stack towards standards (OMII EU) The development of an HPC infrastructure (DEISA +HET) § Sustainability and consolidation is in the organization (EGI) and EU project costituency (NGI) 2. – – – NGI should guarantee the funds for the operation and development of their national einfrastrcture as NRENS EGI is the long term EU Scientific Policy Committee for e-Infrastructure The work continue to be done by the projects above (as Geant at EU level)made by EGI NGIs or national § – – Continuous evaluation of relevance of proposals No need of large DANTE in first approximation since there is now no an EU backbone. Resources are national! (in first appraximation, MAY CHANGE!) Strong coordination of Regional ROCs more needed than a strong Centralized Operation Management body last update 21/12/2021 10: 33 26

The strategy towards innovation and general Grid exploitation: C-OMEGA § Focus: Exploit Italian and general achievements in global Science Grids for early grid exploitation in Industry § Releasing and supporting in Italy and Europe a platform of selected and coherent Grid Services Open Source obeying to International Standards § Supporting pilot exploitation by Industry, Business and Services § § Leveraging from the large development and standardization effort of international and national research projects like EDG, LCG, EGEE, Grid. it… A common set of Services, as the WEB and TCP/IP for ¨ ¨ § e. Sience Institutions Early commercial adopter Should guarantee evolution and adherence to international standards § Close collaboration with similar initiatives at EU level: OMII Europe § § The Italian solution: The Consortium for the Open M/W Enabling Grid Applications (C-OMEGA) last update 21/12/2021 10: 33 27

Conclusions § First generation of Grid services in LCG/EGEE, DEISA production Grid are currently in use in Italy § They are fast evolving for more functionalities, robustness and security § Application as HEp and Biomed indicate clear directions for the evolution to satisfy those communities § Some needed services are still new or missing and very important functionalities are required by user communities § Standards specs are still badly missing for many baseline services, see BES § However LHC experiments (~10 K people), as other Sciences, need to have a fully efficient infrastructure in place as soon as possible. Europe need to provide what is required in time to avoid multiplication of efforts: § Unsustainable grid infrastrctures for each VO § Together with other National Initiatives and CERN IGI need to address the long term sustainability § EGI should be finalised after an in depth debate in all countries favored by the EU EGI design study project § The Consortium c-OMEGA in collaboration with OMII-EU will favor the transfer of Science Grid achievements to Industry and society and will guarantee sustainability and evolution towards standards last update 21/12/2021 10: 33 29