Smart Monitoring TDAQ view o Background o TDAQ

Smart Monitoring: T/DAQ view o Background o T/DAQ services: o o o Smart Monitoring Discussion MWG role Milestones Requirements o o 30 th November, 2005 Data sources Status and Plans o o Distributed information Online histograms General thoughts Past experience Stephen Hillier 1

Reminder: distributed information o o Mature part of Online Software Data of many sorts published/retrieved over network o o o Simple parameters, warnings, alarms Histograms, analysis results Scalable, Flexible, Extensible o Performance Measured in Large Scale Tests Information Server 30 th November, 2005 Stephen Hillier 2

Reminder: current histogram manipulation tools o Basic Histogram Display o Histogram ‘Gatherer’ o o o Combines multiple sources Performs further analysis Histogram Presenter o Includes control of analysis Gatherer 30 th November, 2005 Stephen Hillier 3

Reminder: online data sources Event Based Monitoring DCS input ROD Detector Front End Monitoring SFI Athena PT 30 th November, 2005 ROS Stephen Hillier Event Dataflow GNAM Framework Machine Parameters Online Data and Histograms EF 4

Smart Monitoring o Discussion held at Mainz T/DAQ week o o o Automatic histogram checking o o o Led by Michael Hauschild See agenda. cern. ch/full. Agenda. php? ida=a 051610 Used in OPAL and CDF Sorely missed at other experiments Consensus: ATLAS online needs automated checking o o o MWG to provide framework for checking Preferably soon, to avoid divergence But manpower is an issue 30 th November, 2005 Stephen Hillier 5

MWG role in automated checks o MWG will provide online smart monitoring framework o This will provide a complete basis for data quality assessment o o Methods for configuring checking algorithms Responsible for execution Common methods for reporting warnings and errors Detectors are responsible for providing: o o o Algorithms Reference entities Configuration details o Offline algorithms (run at Point 1) also supported o Need to assess requirements from all groups 30 th November, 2005 Stephen Hillier 6

Monitoring Milestones: extract o Q 4 2006 o MWG will provide as part of the TDAQ software release: o o Detectors will: o o o First prototype of the Smart Monitoring Start using Smart Monitoring framework Start making and using reference histograms Q 2 2007 o MWG will provide: o o A fully functional Online Monitoring framework Data quality assessment tools Detectors will use Online Monitoring for data quality studies Detectors will provide: o o o Algorithms for the online monitoring The necessary configuration (i. e. reference histograms, algorithms, alarms, etc. ) for the smart monitoring facility Detectors will use the data quality assessment tools for storing the data quality decisions together with run data 30 th November, 2005 Stephen Hillier 7

Histogram checking requirements o Reference histograms o o o Robust against changes in (eg) lumi, trigger menu… …or provide configurable sets Flexible checking mechanism o Ability to configure: o o Different types of checks o o Bounds, non-zero, fit parameters Result archiving o o Regularity, alarm levels, ‘dead’ periods And flexible browsing User Friendly 30 th November, 2005 See Michael’s Talk Stephen Hillier 8

Past (Trigger) Experience o o Trigger monitoring is the most important (!) Often also quite unpredictable o o o Luminosity dependant Trigger menu dependant Background dependant o Non-filling conditions easy to spot o Would also like to flag: o o Hot channels Flaky channels Calibration drifts Distinguishing odd conditions from genuine problems requires experience o And eventually very clever software 30 th November, 2005 Stephen Hillier 9

Conclusions o MWG intend to implement smart monitoring through histogram comparison o Very early stage of development o Willing to look at whatever tools are available o Flexibility and configurability are probably the toughest problems 30 th November, 2005 Stephen Hillier 10