Predrag Buncic Institut fuer Kernphysik Frankfurt CERN For
Predrag Buncic Institut fuer Kernphysik, Frankfurt & CERN For ALICE Collaboration The Ali. En system, status & perspective Predrag. Buncic@cern. ch
Talk Outline 1. Alice Computing model 2. Ali. En 1. 2. 3. 4. Project timeline Architecture Components File Catalogue and Resource broker 3. Outlook 4. Conclusions 6/16/2021 Predrag. Buncic@cern. ch 2
ALICE @ CERN LHC Typical next generation HEP experiment n n ~1000 scientists in ~100 of institutions CPU & data intensive simulation & reconstruction w Results in need for heavily distributed processing and event storage w Complex analysis of distributed data n Large files (one simulated event up to 2 GB) w Specific to Alice heavy ion events (but, we have to plan for 1 PB/year of p+p data as well) n Many files 10^9 files/year (x n, n>2) 2 PB/year Ali. En (ALIce ENvironment) is our attempt to gradualy approach and tackle computing problems at LHC scale and implement ALICE Computing Model 6/16/2021 Predrag. Buncic@cern. ch 3
ALICE Computing Model User Simulation, Reconstruction, Calibration, Analysis C++ Nice! I only have to learn C++ anything World Interfaces & Distributed computing environment ROOT System GUI Persistent IO Utility Libs Ali. ROOT C++ Typical HEP use cases: üSimulation & Reconstruction üEvent mixing 6/16/2021 Predrag. Buncic@cern. ch üAnalysis 4
Project Timeline 2001 2002 2003 2004 Start 2005 10% Data Challenge (analysis) Physics Performance Report (mixing & reconstruction) First production (distributed simulation) Functionality 6/16/2021 Interoperability Performance, Scalability, Standards Predrag. Buncic@cern. ch 5
4 continents, 32 sites 6/16/2021 Predrag. Buncic@cern. ch 6
Ali. En V. O. 6/16/2021 Predrag. Buncic@cern. ch 7
What is Ali. En? Main features n n Distributed file catalogue built on top of RDBMS File replica and cache manager with interface to MSS w CASTOR, HPSS, HIS… w Ali. En. FS – Linux file system that uses Ali. En File Catalogue and replica manager n n SASL based authentication which supports various authentication mechanisms (including Globus/GSSAPI) Resource Broker with interface to batch systems w LSF, PBS, Condor, BQS, … n Various user interfaces w command line, GUI, Web portal n n n Package manager (dependencies, distribution…) Metadata catalogue C/C++/perl/java API ROOT interface (TAli. En) SOAP/Web Services EDG compatible user interface n n 6/16/2021 Common authentication Compatible JDL (Job description language) based on CONDOR Class. Ads Predrag. Buncic@cern. ch 8
Ali. En Architecture External software 6/16/2021 Predrag. Buncic@cern. ch User Application Low level Package Mgr (…) V. O. Packages & Commands Logger API (C/C++/perl) SE Config Mgr CE SOAP/XML Perl Core Perl Modules External Libraries RB Interfaces User Interface Authentication File & Metadata Catalogue LDAP Database Proxy ADBI DBD RDBMS (My. SQL) Ali. En Core Components & services FS CLI GUI Web Portal High level 9
Ali. En Components Web of Ali. En Services Modules & libraries 6/16/2021 Predrag. Buncic@cern. ch 10
File catalogue Tier 1 |--. / | |--cern. ch/ | | |--user/ | |--a/ | | |--admin/ | | | | |--aliprod/ | | | |--f/ | | |--fca/ | | | |--p/ | | |--psaiz/ | | |--as/ | | |--dos/ | | |--local/ 6/16/2021 ALICE LOCAL ALICE USERS ALICE SIM |--simulation/ | |--2001 -01/ | | |--V 3. 05/ | |--Config. C | |--grun. C | | | | | |--b/ | | |--barbera/ |--36/ | |--stderr | |--stdin | |--stdout | |--37/ | |--stderr | |--stdin | |--stdout | |--38/ | |--stderr | |--stdin | |--stdout “If you're a programmer, one of the great things about Linux and Unix is that everything is a file -or at least acts like one. From devices to sockets, the "everything is a file" paradigm has served Unix well for a long, long time. ” Predrag. Buncic@cern. ch 11
Resource Broker “Pull” instead of traditional “Push” architecture Authen Broker Transfer. Optimiser IS 6/16/2021 Logger Predrag. Buncic@cern. ch 12
GRID of GRIDs? Ali. En User Interface VTD/i. VDGL stack Ali. En stack EDG stack Nice! Now I do not have to worry about ever changing GRID environment… 6/16/2021 Predrag. Buncic@cern. ch 13
Outlook • Support for analysis/PROOF interface (=> A. Peters ) • Seamless access to GRID resources from ROOT prompt • Ali. En. FS • => Ali. En. FS - a Linux File system for the Ali. En Grid services (A. Peters) • EDG interface • => Ali. En - EDG interoperability in ALICE (poster) • Optimization o o Queue optimizers Query optimization and caching • Service registration/discovery using P 2 P technology • Jabber – XML based instant messaging protocol • Ali. En Web Services => OGSI Grid services o Monitoring, modeling and simulation of Ali. En GRID o Mon. ALisa framework o Virtual datasets? 6/16/2021 Predrag. Buncic@cern. ch 14
Conclusions 2001 2002 2003 2004 2005 o After only 2 years of development, we have deployed a distributed computing environment which meets the needs of Alice experiment ü ü ü Simulation & Reconstruction Event mixing Analysis o Using Open Source components (representing 99% of the code), internet standards (SOAP, XML, PKI…) and scripting language (perl) was the key element that alllowed quick prototyping and very fast development cycles o We are now very close to finalizing Ali. En architecture and API o Open. Ali. En? 6/16/2021 Predrag. Buncic@cern. ch 15
CHEP 2003 Ali. En Resource Brokers (Pablo Saiz, poster) Ali. En. FS - a Linux File system for the Ali. En Grid services (Andreas Peters) Ali. En - EDG interoperability in ALICE (Stefano Bagniasco, poster) ALICE experience with EDG (Piergiorgo Cerello) The Mammo. Grid Project (Richard Mc. Clatchey, next talk) Use of HEP software for medical applications (Piergiorgo Cerello) 6/16/2021 Predrag. Buncic@cern. ch 16
- Slides: 16