Alice HLT Online Calibration framework High Level Trigger

Alice HLT Online Calibration framework High Level Trigger Online Calibration framework in ALICE Normal text click to edit CHEP 2007 – Victoria (02. 09. – 07. 09. 2007) Sebastian Robert Bablok (Department of Physics and Technology, University of Bergen, Norway) for the ALICE HLT Collaboration (see last slide for complete list of authors) CHEP 2007 – Victoria Sebastian. Bablok@uib. no 1/17

Alice HLT Online Calibration framework Outline High Level Triger (HLT) system Normal text click to HLT Interfaces to the Experiment Control System (ECS) to Offline to the Detector Control System (DCS) to event visualization (Ali. Eve) Synchronization Timeline Summary & Outlook CHEP 2007 – Victoria Sebastian. Bablok@uib. no edit 2/17

Alice HLT Online Calibration framework Purpose: 3/17 High Level Trigger system Online event reconstruction and analysis 1 k. Hz event rate (minimum bias Pb. Pb, pp) Normal text - click to edit Providing of trigger decisions Selection of regions of interest within an event performance monitoring of the ALICE detectors Lossless compression of event data Online production of calibration data DAQ Mass Storage CHEP 2007 – Victoria Sebastian. Bablok@uib. no analyzed events / trigger decisions raw event data HLT ECS DCS Trigger

Alice HLT Online Calibration framework High Level Trigger system Hardware Architecture: PC cluster with up to 500 computing node AMD Opterons 2 GHz (dual board, dual core) 8 GByte Ram 2 x 1 GBit ethernet Infiniband backbone Normal text - click to edit infrastucture nodes for maintaining the cluster 8 TByte AFS file server dedicated portal nodes for interaction with the other ALICE systems cluster organization matches structure of the ALICE detectors CHEP 2007 – Victoria Sebastian. Bablok@uib. no 4/17

Alice HLT Online Calibration framework High Level Trigger system Software Architecture: dynamic software data transport framework (Publish-Subscriber) Cluster management (Task. Manager) Analysis Software (Ali. Root) Normal text - click to edit CHEP 2007 – Victoria Sebastian. Bablok@uib. no 5/17

6/17 Alice HLT Online Calibration framework DCS (Detector Control System) ECS DCS values HLT - Interfaces (Experiment Control System) Control DAQ Calculated values (Data Acquisition) Processed Trigger events decisions DDL ECSproxy DCS-portal Normal text - click to edit (Pendolino, FED) Online event monitoring HLT cluster Event data FEE HOMER FEP (Front-End. Electronics) (Data Acquisition) CHEP 2007 – Victoria (ALICE Event Monitor) OFFLINE Calibration OFFLINE-portal data (Shuttle, Taxi) DAQ Ali. Eve New calculated calibration data FEP = Front-End-Processor; DDL = Direct Data Link; Sebastian. Bablok@uib. no HOMER = HLT Online Monitoring Environment including ROOT OCDB (Offline Conditions DB) Shuttle

7/17 Alice HLT Online Calibration framework HLT ECS interface ECS Proxy: Steering of the HLT via the Experiment Control System (ECS) Synchronizes the HLT with the other online and offline systems in ALICE Provides the HLT with upcoming run setup (run number, type of collision, trigger classes, …) Consists of a state machine states (implemented in SMI++) Normal text - click to edit well defined states and transition commands Task. Manager ECS states and transition commands CHEP 2007 – Victoria Sebastian. Bablok@uib. no ECSproxy control HLT cluster

Alice HLT Online Calibration framework OFFLINE HLT interface (Taxi) Taxi portal: Fetches latest available calibration settings from the OCDB Normal text - click to edit pre-produced or in former runs calculated calibration objects enveloped in ROOT-files (can be directly used by ALICE analysing framework - Ali. Root CDB access classes transparent access for analysis code if running online or offline) Caches calibration data locally in the HLT (HLT Conditions Data. Base - HCDB) Retrieval is performed asynchronously to run HCDB distributed to nodes with the Detector Algorithms (DAs) copied locally to the DA nodes before each run fixed version per run CHEP 2007 – Victoria Sebastian. Bablok@uib. no 8/17

9/17 Alice HLT Online Calibration framework OFFLINE HLT interface (Taxi) intern Cluster node (Analysis) extern Ali. Root CDB access classes Cluster node (Analysis) DA DA Normal text click to edit DA_HCDB Task. Manager updated before each run ECSproxy current run number DA = Detector Algorithm Framework components CHEP 2007 – Victoria Sebastian. Bablok@uib. no Taxi. HCDB portal-taxi Taxi 1. Detector responsibility OCDB OFFLINE DA DA

Alice HLT Online Calibration framework DCS HLT interface (Pendolino) Purpose: Normal text - click to edit Requests current run conditions (like temperature, voltages, …) permanently measured by the Detector Control System (DCS) frequently request of “current” values (three different time intervals) providing of latest available values to the Detector Algorithms (DAs) preparation / enveloping of the data transparent access to data for the DAs (online and offline) CHEP 2007 – Victoria Sebastian. Bablok@uib. no 10/17

11/17 Alice HLT Online Calibration framework DCS HLT interface (Pendolino) Detector responsibility intern DCS Normal text - Archive click to edit Framework components PVSS DB DA = Detector Algorithm portal. Pendolino Cluster node preparation / enveloping (Analysis) Pendolino HCDB DA DA DA_HCDB CHEP 2007 – Victoria Sebastian. Bablok@uib. no Pendolino frequent update during run extern

12/17 Alice HLT Online Calibration framework HLT DCS interfaces (FED) intern Cluster node extern (Analysis) FED-portal Normal text - click to edit DA DA FED API DCS PVSS Detector responsibility Framework components DA = Detector Algorithm CHEP 2007 – Victoria Sebastian. Bablok@uib. no Archive DB FED portal: collects produced data inside the HLT during the run writing back data to the DCS (TPC drift velocity, …) uses the Front-End-Device (FED) API common among all detectors in ALICE PVSS panels to integrate data into DCS

13/17 Alice HLT Online Calibration framework HLT OFFLINE interface (Shuttle) Purpose: Cluster node DA DA Normal text - click to edit Collects online calculated calibration settings from the DAs inside the HLT calibration objects stored in a File EXchange Server (FXS) meta data stored in a My. SQL DB (run number, detector, timestamp, …) Feeding back new calibration data to the OCDB notifies: “collecting Task. Manager finished” ECSproxy ECS portal-shuttle (FXS-Subscriber) FXS Shuttle OFFLINE synchronisation CHEP 2007 – Victoria Sebastian. Bablok@uib. no My. SQL OCDB

Alice HLT Online Calibration framework 14/17 HLT Ali. Eve interface (Alice Event visualization) DA DA • integrates the HLT in existing visualization infrastructure of Ali. Eve (M. Tadel) connection to the HLT from anywhere within CERN network ONE visualization infrastructure for all detectors Normal • text - click to edit Homer • Ali. Eve Visualization of results CHEP 2007 – Victoria Sebastian. Bablok@uib. no

15/17 Alice HLT Online Calibration framework Synchronization timeline Sequence: Before Run At Start-of. Run Offline Taxi (asynch) HLT ECSproxy Distribute HCDB Normal text - click to edit During Run FEEdata Initialize cluster Init DCSPendolino outstream to DAQ DCSFED Ali. Eve. Homer After End-of-Run . . . collect data for Shuttle Offline Shuttle analyze events / calculate calibration data So. R contact with the various other systems in ALICE CHEP 2007 – Victoria Sebastian. Bablok@uib. no Eo. R Complete

Alice HLT Online Calibration framework Summary & Outlook HLT consists of a large computing farm matching the structure of the ALICE detectors. Normal text - click to edit Interfaces with the other systems of ALICE (online and offline) enables data exchange with the connected parts. HLT allows for detector performance- and physics monitoring and calibration online. With the presented mechanism the HLT will be able to provide all required conditions and calibration settings to the Analysis Software for first physics in ALICE 2008. CHEP 2007 – Victoria Sebastian. Bablok@uib. no 16/17

Alice HLT Online Calibration framework 17/17 ALICE HLT Collaboration Sebastian Robert Bablok, Oystein Djuvsland, Kalliopi Kanaki, Joakim Nystrand, Matthias Richter, Dieter Roehrich, Kyrre Skjerdal, Kjetil Ullaland, Gaute Ovrebekk, Dag Larsen, Johan Alme (Department of Physics and Technology, University of Bergen, Norway) Torsten Alt, Volker Lindenstruth, Timm M. Steinbeck, Jochen Thaeder, Udo Kebschull, Stefan Boettger, Sebastian Kalcher, Camilo Lara, Ralf Panse Normal text - click to edit (Kirchhoff Institute of Physics, Ruprecht-Karls-University Heidelberg, Germany) Harald Appelshaeuser, Mateusz Ploskon (Institute for Nuclear Physics, University of Frankfurt, Germany) Haavard Helstrup, Kirstin F. Hetland, Oystein Haaland, Ketil Roed, Torstein Thingnaes (Faculty of Engineering, Bergen University College, Norway) Kenneth Aamodt, Per Thomas Hille, Gunnar Lovhoiden, Bernhard Skaali, Trine Tveter (Department of Physics, University of Oslo, Norway) Indranil Das, Sukalyan Chattopadhyay (Saha Institute of Nuclear Physics, Kolkata, India) Bruce Becker, Corrado Cicalo, Davide Marras, Sabyasachi Siddhanta (I. N. F. N. Sezione di Cagliari, Cittadella Universitaria, Cagliari, Italy) Jean Cleymans, Artur Szostak, Roger Fearick, Gareth de Vaux, Zeblon Vilakazi (UCT-CERN, Department of Physics, University of Cape Town, South Africa) CHEP 2007 – Victoria Sebastian. Bablok@uib. no

Alice HLT Online Calibration framework Backup slides Normal text - click to edit CHEP 2007 – Victoria Sebastian. Bablok@uib. no 18/17

Alice HLT Online Calibration framework 19/17 HLT interfaces ECS: Controls the HLT via well defined states (SMI++) Provides general experiment settings (type of collision, run number, …) DCS: Normal text - click to edit Provides HLT with current Detector parameters (voltages, temperature, …) Pendolino, (DCS HLT) Provides DCS with processed data from the HLT (TPC drift velocity, …) FED-portal, (HLT DCS) OFFLINE: Interface to fetch calibration settings from the OCDB Taxi, (OFFLINE HLT) Feeding back new calibration data to OCDB Shuttle-portal, (HLT OFFLINE) HOMER: HLT interface to Ali. Eve for online (event) monitoring FEE: stream in of raw event data via H-RORCs DAQ: stream out of processed data and trigger decisions to DAQ-LDCs CHEP 2007 – Victoria Sebastian. Bablok@uib. no

20/17 Alice HLT Online Calibration framework Synchronization timeline Timeline (1) Before Run (continuously): Normal text - click to edit Taxi requests latest available calibration settings from the OCDB (asynchronously to actual run) At Start-of-Run (So. R) - Initial Setting: ECS HLT: initialization of the HLT cluster (run number, beam type, trigger classes, …) fixating the HCDB for a run, distribution of the HCDB to the DA nodes Offline Taxi (asynch) HLT ECSproxy Distribute HCDB FEEdata Initialize cluster Init DCSPendolino outstream to DAQ DCSFED Ali. Eve. Homer . . . collect data for Shuttle Offline Shuttle analyze events / calculate calibration data So. R contact with the various other systems in ALICE CHEP 2007 – Victoria Sebastian. Bablok@uib. no Eo. R Complete

Alice HLT Online Calibration framework 21/17 Synchronization timeline Timeline (2) During Run (after So. R): Normal text - click to edit DCS HLT: current environment/condition values fetched via the Pendolino update of the HCDB on the DA nodes with the new DCS data HLT DCS: processed, DCS relevant values are written back via the FED-portal (e. g. TPC drift velocity) HLT DAQ: analyzed events go to permanent storage (includes a certain time period after the End-of-Run) HLT Ali. Eve: online event monitoring in the Alice Control Room After End-of-Run (Eo. R): HLT OFFLINE: collecting of fresh produced calibration objects in the FXS of the Shuttle portal OFFLINE Shuttle fetches these objects from the HLT (synchronization via ECS) CHEP 2007 – Victoria Sebastian. Bablok@uib. no

22/17 Alice HLT Online Calibration framework ECS Proxy implicit transition INITIALIZE OFF DEINITIALIZING << intermediate state>> implicit transition ECS interface Preparing of HCDB for DAs Normal text - click to edit SHUTDOWN INITIALIZED CONFIGURE + params CONFIGURING RESET << intermediate state>> implicit transition CONFIGURED ENGANGE implicit transition Offline Shuttle can fetch data DISENGAGING << intermediate state>> DISENGAGE READY implicit transition START COMPLETING Collecting data for Shuttle CHEP 2007 – Victoria Portal Sebastian. Bablok@uib. no STOP Pendolino fetches DCS data for DAs RUNNING Event processing

TAXI Ali. En 23/17 Alice HLT Online Calibration framework TAXI-portal TAXIHCDB HLT - cluster AFS OCDB release TAXI-HCDB before SOR ECSproxy set links from TAXI-HCDB to HCDB TAXIHCDB (read-only) Normal text - click to edit Init HCDB script Task Manager links from TAXI-HCDB to HCDB control Pendolino DCS Archive DB release HCDB after update Amanda Pendolino HCDB (read-only) Pendolino-portal Sebastian. Bablok@uib. no DA HCDB Prediction processor inform about new release CHEP 2007 – Victoria DA notify Pub. Sub of new release AFS ship update info through chain DA

Alice HLT Online Calibration framework Introducing the HLT system HLT counting room in ALICE pit Normal text - click to edit CHEP 2007 – Victoria Sebastian. Bablok@uib. no 24/17

Alice HLT Online Calibration framework HLT overview ALICE data rates (example TPC) ● Central Pb+Pb collisions ● event rates: ~200 Hz (past/future protected) Mbyte Normal~75 text - click to edit event sizes: (after zero-suppression) ● data rates: ● ~ 15 Gbyte/sec TPC data rate alone exceeds by far the total DAQ bandwidth of 1. 25 Gbyte/sec Detectors ● ● Event selection based on software trigger Efficient data compression CHEP 2007 – Victoria Sebastian. Bablok@uib. no HLT DAQ Mass storage TPC is the largest data source with 570132 channels, 512 timebins and 10 bit ADC value. 25/17

Alice HLT Online Calibration framework Architecture Normal text - click to edit CHEP 2007 – Victoria Sebastian. Bablok@uib. no 26/17

Alice HLT Online Calibration framework 27/17 Architecture Mass Storage ECS Normal text - click to edit DAQ raw event data CHEP 2007 – Victoria Sebastian. Bablok@uib. no analyzed events / trigger decisions HLT DCS Trigger