DAQ software for SND detector Budker Institute of
DAQ software for SND detector Budker Institute of Nuclear Physics, Novosibirsk M. Achasov, A. Bogdanchikov, A. Kim, A. Korol SND DAQ software
VEPP-2000 collider In the past: SND @ VEPP-2 M Years: 1995 - 2000 Energy (2 E): 0. 36 – 1. 4 Ge. V Today: SND @ VEPP-2000 Project started: 2000 Energy: 1. 4 - 2 Ge. V Luminosity: 1032 (2 Ge. V) Perimeter: 24. 4 m Collision time: 82 nsec Beam current: 0. 2 A Bunch length: 3. 3 cm Energy spread: 0. 7 Me. V ACAT 05, Zeuthen 2/25/2021 Session 1, SND DAQ software 2
VEPP-2000 collider ACAT 05, Zeuthen 2/25/2021 Session 1, SND DAQ software 3
SND DAQ requirements n Loose electronic selection conditions (FLT), strong software (TLT) filtration (ratio to 10%). n Readout process should work on the stand-alone equipment, separately from processes which consume a lot of computational power (TLT). n Event selection using fully reconstructed events – L 3 farm. ACAT 05, Zeuthen 2/25/2021 Session 1, SND DAQ software 4
Main data flow 1 k. Hz Readout and events building 4 KB 1 k. Hz Events packing 4 KB 1 k. Hz 1 KB 100 Hz Events filtering Storage 1 KB Expected rates: n Events fragments: 4 МB/s are read from IO processors over Ethernet; n Event building: 4 MB/s; n Event packing: 1 МB/s; n Events filtering (90% screening): 100 KB/sec. ACAT 05, Zeuthen 2/25/2021 Session 1, SND DAQ software 5
Raw events transfer (alternatives) Readout L 3 L 3 Sum Storage Readout ACAT 05, Zeuthen 2/25/2021 L 3 L 3 Session 1, SND DAQ software Storage 6
Raw events transfer (software layers) n Abstract layer n n System layer n n Container capture Read events from container Write events to container Global File System (GFS) – tested Shared raw disks Network File System (NFS) Hardware layer n n Shared SCSI disk - tested Network disk ACAT 05, Zeuthen 2/25/2021 Session 1, SND DAQ software 7
DAQ architecture Detector Off-line Front-end electronics KLUKVA Visualization Backup X 12 X 16 Filtered events CAMAC Readout & Event Building Buffer Database TLT computers Calibration process System support ACAT 05, Zeuthen 2/25/2021 Session 1, SND DAQ software 8
Main online processes n Events treatment : n n n Readout & event building Scalers (statistics from electronic subsystems) L 3 (packing, reconstruction, filtration, statistics) Calibration on events Control : n n n Operator interface Distributed processes management Context recovery after process crash Informer (message analyzer) Electronics sub-system control ACAT 05, Zeuthen 2/25/2021 Session 1, SND DAQ software 9
Readout process state machine Initialization Idle Full stop Monitoring n Monitoring Reject Stop Note: error branches are not shown, all errors cause Idle state. Start Configuration Success Stop Pause Readout Pause Auto-start ACAT 05, Zeuthen 2/25/2021 Session 1, SND DAQ software 10
Process starter service Database TLT 1 Process A: 2 Process B: 1 Process C: 3 TLT 2 Process A: 1 Process D: 3 Online Computer TLT 1 Online Computer TLT 2 Starter service Process A Process child B Process A OK, one session ID Starter service Process C Process A Process D Process C needs to stop Process D needs to start ACAT 05, Zeuthen 2/25/2021 Session 1, SND DAQ software 11
Processes states and recovery n After start processes should receive domain name from “domain distributor server“ (DD). Each domain contains fixed name variable to store a path to recover program. When process crashes (tears connection), DD starts recover program with domain name as parameter. n Current process state data are stored in the Onl. State database and may be accessed by other processes. n State variable type: int, float, string, arrays; access index: domain_name/variable_name n Atomic domain access. ACAT 05, Zeuthen 2/25/2021 Session 1, SND DAQ software 12
Process context recovery Domain states: Domain request Free Busy Process finished Start recovery program d crashe s s e c Pro Context recovery Re-register domain Distributor server crashed and restarted Confirmation timeout Confirmation ACAT 05, Zeuthen 2/25/2021 Session 1, SND DAQ software 13
Online Databases Online state DB n Conditions DB n Configurations DB n Bookkeeping DB n } Closely tiered. To simplify access integrated interface is supported. Implementation: RDBMS (My. SQL) ACAT 05, Zeuthen 2/25/2021 Session 1, SND DAQ software 14
Configurations & Bookkeeping API Table description (SQL format) table. sql parser C++ description and implementation table. h table. cc Application ACAT 05, Zeuthen 2/25/2021 Session 1, SND DAQ software 15
Operator interface Run control Web-log Database access Process control Communication Personal settings Status Selective views Run conditions Occupied resources Users contacts User info Start Stop Pause Message search Configurations Processes start list Chat UI details Message add UI Calibrations Silence Extended messages E-mail notifications Statistics ACAT 05, Zeuthen 2/25/2021 UI colors Scalers selection Session 1, SND DAQ software 16
Operator interface components HTTP server HTML javascript PHP scripts Configurations PHP API Calibrations PHP API Online States PHP API Recovery PHP API RDBMS Web-log PHP API Online Process Configurations API Calibrations API Online States API Recovery API Web-log API ACAT 05, Zeuthen 2/25/2021 Session 1, SND DAQ software 17
Web-log messages The web-log toolkit consists of several interfaces written in languages PHP and C ++, which allow to choose, add and expect the message. Each message has attributes: n n n n Alarm – is on when required to draw attention of the operator immediately. Date and time of the message. Topic of the message (what field it belongs to). Severity of the message (fatal mistake, error, warning, info, debug, trace). Sender: user, host, process name. Subject – the summary of the message. This is a basic field. Message attachments of any MIME-type. Full description of the problem, pictures, diagrams can be presented in the attachment. Attachment has attributes: name, size and inline. The attribute inline is a recommendation for browsers, whether to include attachment in the message directly or display just a link. ACAT 05, Zeuthen 2/25/2021 Session 1, SND DAQ software 18
Statistics view (example) ACAT 05, Zeuthen 2/25/2021 Session 1, SND DAQ software 19
Design n n High level robustness: raw events deep buffering; High quality of filtering: full events reconstruction before filtering; Reuse of off-line framework software; Relational databases to store experiment configurations, states, calibrations; Operator interface: web technologies. ACAT 05, Zeuthen 2/25/2021 Session 1, SND DAQ software 20
Status n DAQ system software status: Event transfer mechanism were tested on the maximum possible rates; n The most important functionality were realized; n Advanced features were added; n 2003 -2004: system were used to receive events from the drift chamber prototype. n ACAT 05, Zeuthen 2/25/2021 Session 1, SND DAQ software 21
- Slides: 21