LHCb Trigger and Data Acquisition System Beat Jost

LHCb Trigger and Data Acquisition System Beat Jost Cern / EP Presentation given at the 11 th IEEE NPSS Real Time Conference June 14 -18, 1999 Santa Fe, NM

Outline q Introduction q General Trigger/DAQ Architecture q Trigger/DAQ functional components ã Selected Topics å Level-1 Trigger å Event-Building Network Simulation q Summary RT 99, Santa Fe, 15 June 1999 Beat Jost, Cern Slide 2

Introduction to LHCb q Special purpose experiment to measure precisely CP violation parameters in the BB system q Detector is a single-arm spectrometer with one dipole q Total b-quark production rate is ~75 k. Hz q Expected rate from inelastic p-p collisions is ~15 MHz q Branching ratios of interesting channels range between 10 -5 -10 -4 giving interesting physics rate of ~5 Hz RT 99, Santa Fe, 15 June 1999 Beat Jost, Cern LHCb in Numbers Slide 3

LHCb Detector RT 99, Santa Fe, 15 June 1999 Beat Jost, Cern Slide 4

Typical Interesting Event RT 99, Santa Fe, 15 June 1999 Beat Jost, Cern Slide 5

Trigger/DAQ Architecture LHC-B Detector VDET TRACK ECAL HCAL MUON Data rates RICH 40 MHz Fixed latency 4. 0 ms Level 1 Trigger 40 TB/s 1 MHz Timing L 0 & Fast 40 k. Hz L 1 Control Front-End Electronics Front-End Multiplexers (FEM) 1 MHz Front End Links Variable latency <1 ms Throttle RU RU RU Read-out units (RU) Read-out Network (RN) SFC Variable latency L 2 ~10 ms L 3 ~200 ms RT 99, Santa Fe, 15 June 1999 Storage Beat Jost, Cern SFC 1 TB/s LAN Level 0 Trigger 4 GB/s 2 -4 GB/s Sub-Farm Controllers (SFC) CPU CPU Trigger Level 2 & 3 Event Filter Control & Monitoring 20 MB/s Slide 6

Front-End Electronics q Data Buffering for Level-0 latency q Data Buffering for Level-1 latency q Digitization and Zero Suppression q Front-end Multiplexing onto Frontend links RT 99, Santa Fe, 15 June 1999 Beat Jost, Cern Slide 7

Timing and Fast Control q Provide common and synchronous clock to all components needing it q Provide Level-0 and Level-1 trigger decisions q Provide commands synchronous in all components (Resets) q Provide Trigger hold-off capabilities in case buffers are getting full RT 99, Santa Fe, 15 June 1999 Beat Jost, Cern Slide 8

Level-0 Trigger q Large transverse Energy (Calorimeter) Trigger q Large transverse momentum Muon Trigger q Pile-up Veto q Implemented in FPGAs/DSPs basically hard-wired RT 99, Santa Fe, 15 June 1999 Beat Jost, Cern Input rate: 40 MHz Output rate: 1 MHz Latency: 4. 0 s (fixed) Slide 9

DAQ Functional Components q Readout Units (RUs) ã Multiplex Front-end links onto Readout Network links ã Merge input fragments to one output fragment q Subfarm Controllers (SFCs) Note: There is no central event manager ã assemble event fragments arriving from RUs to complete events and send them to one of the CPUs connected ã Load balancing among the CPUs connected q Readout Network ã provide connectivity between RUs and SFCs for event-building ã provide necessary bandwidth (4 GB/sec sustained) q CPU farm ã execute the high level trigger algorithms å Level-2 (Input rate: 40 k. Hz, Output rate: 5 k. Hz) å Level-3 (Input rate: 5 k. Hz, Output rate: ~100 Hz) ã ~2000 processors (à 1000 MIPS) RT 99, Santa Fe, 15 June 1999 Beat Jost, Cern Slide 10

Control System q Common integrated controls system ã Detector controls (classical ‘slow control’) å High voltage å Low voltage å Crates å Temperatures å Alarm generation and handling å etc. ã DAQ controls å Classical RUN control å Setup and configuration of all components (FE, Trigger, DAQ) å Monitoring ã Same system for both functions RT 99, Santa Fe, 15 June 1999 Beat Jost, Cern Slide 11

Level-1 Trigger q Purpose ã Select events with detached secondary vertices q Algorithm ã Based on special geometry of vertex detector (r-stations, -stations) ã Several steps å track reconstruction in 2 dimensions (r-z) å determination of primary vertex å search for tracks with large impact parameter relative to primary vertex å full 3 dimensional reconstruction of those tracks ã Expect rate reduction by factor 25 q Technical Problem: 1 MHz input rate, 3 GB/s data rate, small event fragments, Latency RT 99, Santa Fe, 15 June 1999 Beat Jost, Cern Slide 12

Level-1 Trigger (2) q Implementation ã ~32 sources to switching network ã Algorithm running in processors (~200 CPUs) ã Basic idea is to have a switching network between data sources and processors ã In principle very similar to DAQ, however the input rate of 1 MHz poses special problems. RT 99, Santa Fe, 15 June 1999 Beat Jost, Cern Slide 13

Event-Building Network q Requirements ã ã ã 4 GB/s sustained bandwidth scalable expandable ~100 inputs (RUs) ~100 outputs (SFCs) affordable and if possible commercial (COTS, Commodity? ) q Readout Protocol ã Pure push-through protocol of complete events to one CPU of the farm Ú Simple hardware and software Ú No central control perfect scalability Ú Full flexibility for high-level trigger algorithms Ø Large bandwidth needed Ø Avoiding buffer overflows via ‘throttle’ to trigger RT 99, Santa Fe, 15 June 1999 Beat Jost, Cern Slide 14

Event-Building Network Simulation q Simulated technology: Myrinet ã Nominal 1. 28 Gb/s ã Xon/Xoff flow control ã Switches: å ideal cross-bar å 8 x 8 maximum size (currently) å wormhole routing å source routing å No buffering inside switches q Software used: Ptolemy discrete event framework q Realistic traffic patterns ã variable event sizes ã event building traffic RT 99, Santa Fe, 15 June 1999 Beat Jost, Cern Slide 15

Network Simulation Results don’t depend strongly on specific technology (Myrinet), but rather on characteristics (flow control, etc) q. FIFO buffers between switching levels allow to recover scalability q 50 % efficiency “Law of nature” RT 99, Santa Fe, 15 June 1999 Beat Jost, Cern Slide 16

Summary q LHCb is a special purpose experiment to study CP violation q Triggering poses special challenges ã Similarity between inelastic p-p interactions and events with B-Mesons q DAQ is designed with simplicity and maintainability in mind ã Push readout protocol Simple, e. g. No central event manager ã Harder bandwidth requirements on readout network ã Simulations suggest that readout network can be realized by adding FIFO buffers between levels of switching elements q Unified approach to Controls ã Same basic infrastructure for detector controls and DAQ controls ã Both aspects completely integrated but operationally independent RT 99, Santa Fe, 15 June 1999 Beat Jost, Cern Slide 17