A software framework for Data Quality Monitoring in

A software framework for Data Quality Monitoring in ATLAS S. Kolos, A. Corso-Radu University of California, Irvine, M. Hauschild CERN, B. Kehoe, H. Hadavand Southern Methodist University September 2007 CHEP 07 Conference 1

Outline • The ATLAS Data Quality challenge • Data Quality Framework – Design & Implementation – Experience during ATLAS commissioning • Conclusion September 2007 CHEP 07 Conference 2

ATLAS DQ Challenge Inner Detector Pixel SCT TRT Calorimeter Tile. Cal LAr Muon Spectrometer MDT CSC RPC TGC • The Data Quality Challenge: – Fast and reliable detection of DQ problems during data taking and reconstruction September 2007 • 3 main detectors • 10 sub-detectors + LVL 1 Trigger • 33 sub-systems • Each sub-system can be used independently • 140 M channels • Bunch crossing rate 40 MHz • 3 Trigger levels reducing rate by 105 CHEP 07 Conference 3

The Model of Data Quality Framework Inner Detector Pixel SCT TRT Calorimeter Tile. Cal LAr Muon Spectrometer MDT CSC RPC TGC DQ Configuration DQM Config Histograms, Graphs, Rates, etc. DQ Result: • {Red, Yellow, Green} • Optional Tags • TObject DQM Input DQM Core DQM Output • DQM Core: – Read DQ Configuration – Algorithm execution is data-driven: • Execute DQ algorithms as soon as input data becomes available – Produce DQ Result September 2007 CHEP 07 Conference 4

DQ Configuration Atlas Inner Detector Calo Tile Calo Pixel Muon Module 1 Num. Of. Events Module N Num. Of. Events Energy Eta. Phi Histogram NOT Empty Gaus. Fit Chi 2 September 2007 SCT TRT Calorimeter Tile. Cal LAr Muon Spectrometer MDT CSC RPC TGC DQ Region: • Produces summary result for given sub-system • It is based on the children results Liquid Argon Calo Worst Result Summary Weighted Summary Inner Detector CHEP 07 Conference DQ Parameter: • Produces DQ result for a given quantity • Analyses input data with specific algorithm 5

DQ Algorithm • C++ function which analyses histogram and produce the DQ Result for the corresponding DQ Parameter • Special type of DQ Algorithms is Summary. Maker: – Produces DQ Result for a DQ Region – This result is based on the DQ Results of the Region’s children • Custom algorithms can be easily plugged in by: – Inheriting from the Algorithm base class – Implementing two abstract methods: clone and execute September 2007 CHEP 07 Conference 6

Data Quality Algorithms • DQMF provides shared algorithms repository: – contains a set of predefined algorithms: • Reference Histogram comparison – Chi 2, Kolmogorov, Bin comparison (using ROOT) • Fits: – Gaus, Landau, Pol 1 (using ROOT) • Bin Threshold: <, >, <=< >= • Basic Histogram checks – All_Bins_Filled, Histogram_Not_Empty, No_Under. Flows, No_Over. Flows • Basic Stat Checks – Check. Histo_Mean, Check. Histo_RMS • Etc. – Contains 2 summary makers: • Worst result • Weighted result • Repository is filled up by ATLAS sub-system experts: – Share experience between sub-systems – Avoid duplication of work September 2007 CHEP 07 Conference 7

DQ Algorithm Data Flow DQ Configuration • Parameters • Thresholds • References Names Reference Histograms (several references for different conditions) 3 Input Data (Histogram) September 2007 2 1 DQ Algorithm CHEP 07 Conference 4 DQ Result 8

DQM Workbench • DQMF provides set of Root macros which can be used to run DQ Algorithms in Root shell: – For algorithms development and debugging – For preparing configuration root [0]. x Workbench. C root [1]. x Test. Bin. Content. Comp. Algorithm. C Number of bins 2 Sigma away from reference is 16 Green threshold: 33 bin(s); Red threshold : 1 bin(s) Result 2 September 2007 CHEP 07 Conference 9

On-line vs. Off-line Online Histogramming Service On-line Data Taking Online Configuration Service Online Information Service DQM Config DQM Input DQM Core DQM Output Conditions DB ROOT File Off-line Reconstruction • DQMF is used for DQ assessment during the online data taking as well as during the off-line reconstruction September 2007 CHEP 07 Conference 10

On-line DQMF Scalability Node A Atlas DQMF Agent Node B DQMF Agent Inner Detector Calo Muon Node C Tile Calo Liquid Argon Calo DQMF Agent • Online environment implies high performance requirements for the DQ Assessment • The DQMF functionality is provided by a process called DQMF Agent – Any DQRegion can be processed by individual DQMF Agent on a separate node September 2007 CHEP 07 Conference 11

DQMF at the ATLAS Commissioning • Online DQMF configuration for the last Commissioning run includes ~2000 histograms: – Signals distributions, energy distributions, noise distributions, Trigger rate, etc. – This configuration has been handled by a single DQMF agent process • Lessons learned: – The JAIDA which we were using for the DQMF display prototype does not fulfill our performance & scalability requirements – The final version will be implemented using ROOT/Qt/C++ September 2007 CHEP 07 Conference 12

Online DQMF Display (1/2) September 2007 CHEP 07 Conference 13

Online DQMF Display (2/2) September 2007 CHEP 07 Conference 14

Summary & Conclusion • DQMF is framework for DQ analysis which is: – Efficient – implemented in C++ using ROOT – Flexible - fully configurable via simple interface – Simple – no code development is necessary if standard algorithms can be used – Scalable – can be naturally distributed over any number of machines • It is being successfully used now in both On-line and Off-line environments for the ATLAS commissioning September 2007 CHEP 07 Conference 15

Backup September 2007 CHEP 07 Conference 16

Software development process At this stage the scope of the framework has been extended to the Off-line reconstruction Requirements collection Discussion Design has been influenced by the feedback to the Requirements document • The standard software development model has been found extremely useful September 2007 CHEP 07 Conference Design Discussion Implementation C++ 17