SBS VME DAQ SBS collaboration Meeting Alexandre Camsonne

SBS VME DAQ SBS collaboration Meeting Alexandre Camsonne July 7 th 2014

Outline • • SBS DAQ overview GEM readout Fastbus readout HCAL readout Plan Manpower Conclusion 9/5/2021 Super. Bigbite DAQ and Electronics Alexandre Camsonne 2

SBS DAQ Overview • Calorimeter – ECAL : Fastbus – HCAL • SBS GEM – APV 25 INFN MPD • Big. Bite – Scintillator – Shower preshower • Coordinate detector 9/5/2021 Super. Bigbite DAQ and Electronics Alexandre Camsonne 3

Super. Bigbite Spectrometer Focal Plane Polarimeter setup 9/5/2021 Super. Bigbite DAQ and Electronics Alexandre Camsonne 4

Calorimeter Rates (CDR section 5. 1. 7) Most demanding HCAL ECAL Hadron rate estimate using SLAC & DESY data, Wiser code: w/4. 5 Ge. V threshold: ≈ 1. 5 MHz From Hall A Real Compton Scattering experiment NB: Good resolution ≈ 16%/ÖE Electron rate estimate w/2. 5 Ge. V threshold (73% of Eelas): ≈ 200 k. Hz Background rate vs. cut on deposited energy (MC studies in progress) 9/5/2021 Super. Bigbite DAQ and Electronics Alexandre Camsonne ≈ 9 k. Hz coincidence rate w/ 30 ns window 5

DAQ concept • Hybrid Fastbus and pipelined electronics • Level 1 – ≈100 ns latency by analog summing and discrimination – Generated by electron arm (≈200 k. Hz rate) – Gate for Fastbus & non-pipelined VME for Big. Cal • Level 2 : coincidence proton in HCAL and electron in ECAL – Assume up to ≈1. 8 μs latency ( L 2 800 ns max + Fast Clear 1 μs) – 9 KHz with 30 ns coincidence windows – FPGA-based coincidence logic using geometrical constraints reduction by factor 5 ≈ 2 k. Hz physics DAQ rate – Fast Clear FB & VME after L 2 timeout Þ ≈ 13% Electronics Dead Time 9/5/2021 Super. Bigbite DAQ and Electronics Alexandre Camsonne 6

(CDR section F. 3) e’-p Kinematic Correlation 11 x 22 HCAL blocks 20 x 76 ECAL blocks Using geometric correlations from elastic kinematic one can reduce final rate by a factor of 5 and tracker data by at least a factor of 3 Super. Bigbite DAQ and Electronics Alexandre Camsonne 9/5/2021 7

Data flow 100 Mb/s VME CPU VME digitizer APV 25 GEM Gig. E CODA Event builder Event recorder L 1 trigger E Gig. E Big. Cal Bigbite TS Gig Summing Fastbus VME CPU L 1 trigger L 2 trigger Logic Digital Sum HCAL FADC 9/5/2021 SAS RAID VME CPU Super. Bigbite DAQ and Electronics Alexandre Camsonne Data file Compression + 10 Gig. E Silo 8

APV 25 GEM FEC SRU SRS PC 10 Gig. E 100 Mb/s VME CPU VME digitizer APV 25 GEM Gig. E Event builder Event recorder L 1 trigger E Gig. E Big. Cal Bigbite Fastbus VME CPU L 1 trigger L 2 trigger Logic Digital Sum HCAL FADC 9/5/2021 SAS RAID TS Gig Summing CODA VME CPU Super. Bigbite DAQ and Electronics Alexandre Camsonne Data file Compression + 10 Gig. E Silo 9

GEM readout • INFN MPD – – VME 64 X board New version with fast VME protocol Working with CAEN controller Adapting software package to Intel VME CPU to use with CODA • SRS readout – Using Mississippi State SRS – Running with DATE, starting to look integration into CODA – Standard UDP based protocol • Request to use GEM for PREX and Tritium in addition to A 1 n 9/5/2021 Will have baseline performance number before November Super. Bigbite DAQ and Electronics Alexandre Camsonne 10

Front Tracker layout 2048 channel 1 MPD 9/5/2021 Super. Bigbite DAQ and Electronics Alexandre Camsonne 11

Trackers layout 1 MPD 2048 Front tracker Region of Interest from Big. Cal position 1 MPD 2048 Back tracker Region of Interest from HCal position Middle tracker Interpolated from both front and back information 9/5/2021 Super. Bigbite DAQ and Electronics Alexandre Camsonne 12

Trackers layout Worst case configuration 9/5/2021 Super. Bigbite DAQ and Electronics Alexandre Camsonne 13

Suppression schemes • Several algorithm can be implemented in FPGA for further data reduction – Thresholds – Timing – Fitting c 2 – slope 9/5/2021 Super. Bigbite DAQ and Electronics Alexandre Camsonne 14

FPP Tracker layout 2560 2048 512 9/5/2021 Super. Bigbite DAQ and Electronics Alexandre Camsonne 15

Tracker event size with 3 samples readout Detector Rate Channels Occupancy Time window Hits Geometr ical factor Data size Effective data size Data rate 3 KHz Mb/s Front Tracker 400 49000 13. 5% 75 ns 6615 3 52 Kb 14. 3 Kb 43 Second Tracker 130 13600 7. 4% 50 ns 1010 5 8 Kb 1. 6 Kb 5 Third Tracker 64 13600 3. 6% 50 ns 490 5 4 Kb 0. 8 Kb 2 Electron arm GEM 173 12000 2. 4% 50 ns 288 1 2. 3 Kb 6 0. 5 Kb 1. 5 67. 8 Kb 19. 5 Kb 58. 5 Mb/s Calorimeters 125 Total 9/5/2021 Super. Bigbite DAQ and Electronics Alexandre Camsonne 16

Data rates Detector Rate Channels Occupancy Hits Data size Bytes Data rate MB/s 5 KHz Front Tracker 400. 00 49000. 00 0. 50 6615. 00 161929 809. 65 3. 00 269. 88 134. 94 Second Tracker 130. 00 61440. 00 0. 50 1010. 00 203040. 1015. 20 5. 00 203. 04 101. 52 Third Tracker 64. 00 61440. 00 0. 50 490. 00 203040 1015. 20 5. 00 203. 04 101. 52 8115. 00 568009 2840. 05 675. 96 337. 98 Total 9/5/2021 171880. 00 Super. Bigbite DAQ and Electronics Alexandre Camsonne Geometrical Data size factor no sup Data rate MB/s 17

Fastbus readout • Time : 1877 S • Amplitude 1881 M or MQT+1877 S • Fastbus max transfer speed : 40 MB/s can use either Intel or Old vxworks VME CPU • Test Lab : 4 sets of 3 crates, will be able to test performance 9/5/2021 Super. Bigbite DAQ and Electronics Alexandre Camsonne 18

HCAL readout • • • 288 channels 2 VXS crates , 18 FADCs 1. 5 MHz singles 16 block clusters FADC 250 MHz 12 bit = 2 bytes 10 samples : 320 bytes • Need HCAL occupancy 9/5/2021 Super. Bigbite DAQ and Electronics Alexandre Camsonne 19

Tape price Experiment Days Rate (MB/s) seconds Total TB Double DLO 5 DLO 6 GEp 45 250 3888000 972 1944 97200 58320 GEn 50 250 4320000 1080 2160 108000 64800 GMn 25 250 2160000 540 1080 54000 32400 SIDIS 64 250 5529600 1382. 4 2764. 8 Total 138240 397 K$ 82944 239 K$ 9/5/2021 Super. Bigbite DAQ and Electronics Alexandre Camsonne 20

Man power • A. Camsonne : General , MPD readout • S. Abraham , M. Jones : Fastbus • Students – Jessica Campbell (SMU) : Fastbus – Jessie Twigger (FIT SULI) , ? : SRS CODA readout • HCAL trigger readout : ? 9/5/2021 Super. Bigbite DAQ and Electronics Alexandre Camsonne 21

Plan summary • Setup on 3 Fastbus crate setup and test performance • APV 25 performance with SRS and MPD and integration with CODA • Getting started with FADC for HCAL • Start write-up about DAQ 9/5/2021 Super. Bigbite DAQ and Electronics Alexandre Camsonne 22

Conclusion • Need good ways to reduce data • Fastbus setup almost ready for testing • Need to start thinking about SIDIS • Start writeup 9/5/2021 Super. Bigbite DAQ and Electronics Alexandre Camsonne 23

Backup

VETROC JLAB Electronics group 9/5/2021 Super. Bigbite DAQ and Electronics Alexandre Camsonne 25

VXS Crate CPU TD TD TD SD SD TD TD TD TS CPU SSP SSP SSP GTP SD SSP SSP SSP TI L 1 Trigger Diagram Fiber Optics FADC 250 VXS Serial Link • 16 bit @ 250 MHz: 4 Gbps • 64 bit @ 125 MHz CPU FADC FADC CTP SD FADC FADC TI • 12 bit @ 250 MHz, 16 ch • Sums amplitude from all channels • Transfer total energy or hit pattern to CTP VXS Crate CTP Crate Trigger Processor VXS Crate • Sums energies from FADCs • Transfer total energy or hit pattern to SSP Super. Bigbite DAQ and Electronics Alexandre Camsonne 9/5/2021 26

CPU TD TD TD SD SD TD TD TD TS CPU SSP SSP SSP GTP SD SSP SSP SSP TI L 1 Trigger Diagram SSP VXS Crate Copper Ribbon Cable Sub-System Processor VXS Serial Link • 32 bit @ 250 MHz: 8 Gbps CPU FADC FADC CTP SD FADC FADC TI • Consolidates multiple crate subsystems • Report total energy or hit pattern to GTP VXS Crate Global Trigger Processor • Collect L 1 data from SSPs • Calculate trigger equations • Transfer 32 bit trigger pattern to TS Super. Bigbite DAQ and Electronics Alexandre Camsonne 9/5/2021 27

Level-1 Trigger Electronics Custom Designed Boards at JLAB Detector Signals (16) (1) Fiber Optic Links Clock/Trigger (16 bits @ 62. 5 MHz (1) f. ADC 250 (8) CTP Crate Trigger Processor (2) (1) SD Signal Distribution VXS Backplane 9/5/2021 (1) Fiber Optic Link (~100 m) (64 bits @ 125 MHz) ( ) – Number in parentheses refer to number of modules TI Trigger Interface (12) Pipelined detector readout electronics: f. ADC Super. Bigbite DAQ and Electronics Alexandre Camsonne Copper Ribbon Cable (~1. 5 m) (32 bits @ 250 MHz) • Trigger Latency ~ 3 μs 28

Pipelined Hall D DAQ FADC Calorimeter FADC Light Gas Cerenkov SSP GTP C T P CTP C T P Heavy Gas Cerenkov 9/5/2021 FADC Super. Bigbite DAQ and Electronics Alexandre Camsonne 29

Pipelined Hall D DAQ Electron shower accidental 3 us latency Above threshold 9/5/2021 Super. Bigbite DAQ and Electronics Alexandre Camsonne 30

L 3 Farm blocked event fragments partially recombined event fragments full events • All nodes connected with 1 GB/s links ROC • Switches connected with 10 GB/s fiber optics EB 1 Event Builder stage 1 EB 2 ROC ROC Event Builder stage 2 EB 1 L 3 Farm Event Builder stage 1 ROC EB 2 ROC ROC Front-End Crates ~60 crates ~50 MB/s out per crate 9/5/2021 node Event Builder stage 2 EB 1 node Raid Disk Read Out Controllers ROC ER Event Recorder Event Builder stage 1 Staged Event Building N x M array of nodes (exact number to be determined by available hardware at time of purchase) Super. Bigbite DAQ and Electronics Alexandre Camsonne Level-3 Trigger and monitoring Event Recording 300 MB/s in 300 MB/s out 31

Hall D L 1 Trigger-DAQ Rate • • • Low luminosity (107 /s in 8. 4 < E < 9. 0 Ge. V) – 20 k. Hz L 1 High luminosity (108 /s in 8. 4 < E < 9. 0 Ge. V) – 200 k. Hz L 1 – Reduced to 20 k. Hz L 3 by online farm Event size: 15 k. B; Rate to disk: 3 GB/s SC Detectors which can be used in the Level-1 trigger: Energy Start Counter (SC) Hits Time of Flight (TOF) Hits Photon Tagger Hits Electromagnetic background Basic Trigger Requirement: EBCAL + 4 ∙ EFCAL > 2 Ge. V and a hit in Start Counter Hadronic E < 8 Ge. V Energy FCAL (Ge. V) Super. Bigbite DAQ and Electronics Alexandre Camsonne Hadronic E > 8 Ge. V Energy BCAL (Ge. V) Barrel Calorimeter (BCAL) Energy BCAL (Ge. V) Forward Calorimeter (FCAL) Energy FCAL (Ge. V) 9/5/2021 Energy FCAL (Ge. V) 32

Custom Electronics for JLab • • • VME Switched Serial (VXS) backplate – 10 Gbps to switch module (J 0) – 320 MB/s VME-2 e. SST (J 1/J 2) All payload modules are fully pipelined – FADC 125 (12 bit, 72 ch) – FADC 250 (12 bit, 16 ch) – F 1 -TDC (60 ps, 32 ch or 115 ps, 48 ch) Trigger Related Modules – Crate Trigger Processor (CTP) – Sub-System Processor (SSP) – Global Trigger Processor (GTP) – Trigger Supervisor (TS) – Trigger Interface/Distribution(TI/D) – Signal Distribution (SD) FADC 125 F 1 -TDC Super. Bigbite DAQ and Electronics Alexandre Camsonne 9/5/2021 33

CPU SSP SSP SSP GTP SD SSP SSP SSP TI CPU TD TD TD SD SD TD TD TD TS L 1 Trigger Diagram VXS Crate Trigger Distribution Fiber Optics • 16 bit @ 62. 5 MHz: 1 Gbps CPU FADC FADC CTP SD FADC FADC TI • Distribute trigger, clock and synchronize signals to TI in each Crate VXS Serial Link VXS Crate Trigger Supervisor • Calculate 8 bit trigger types from 32 bit trigger pattern • Prescale triggers • Transfer trigger and sync signal to TD (16 bit total) Super. Bigbite DAQ and Electronics Alexandre Camsonne 9/5/2021 34

CPU SSP SSP SSP GTP SD SSP SSP SSP TI CPU TD TD TD SD SD TD TD TD TS L 1 Trigger Diagram VXS Crate TID Trigger Interface VME Readout Controller CPU FADC FADC CTP SD FADC FADC TI • Gigabit ethernet • Receive trigger, clock and sync signals from TD • Make crate trigger decision • Pass signals to SD Signal Distribution • Distribute common signals to all modules: busy, sync and trigger 1/2 VXS Crate VXS Serial Link • 4 bit @ 250 MHz: 1 Gbps Super. Bigbite DAQ and Electronics Alexandre Camsonne 9/5/2021 35

The Glue. X Detector 2. 2 Tesla Solenoid • 2. 2 T superconducting solenoidal magnet • Fixed target (LH 2) • 108 tagged /s (8. 4 -9. 0 Ge. V) • hermetic TOF time of flight SC start counter Charged particle tracking • Central drift chamber (straw tube) • Forward drift chamber (cathode strip) Calorimetry • Barrel Calorimeter (lead, fiber sandwich) • Forward Calorimeter (lead-glass blocks) PID • Time of Flight wall (scintillators) • Start counter • Barrel Calorimeter Super. Bigbite DAQ and Electronics Alexandre Camsonne 9/5/2021 36

Glue. X Data Rate BNL Glue. X CLAS 12 3 GB/s 0. 1 GB/s 15 k. B 20 k. B ALICE ATLAS CMS LHCb STAR PHENIX 500 GB/s 113 GB/s 200 GB/s 40 GB/s 50 GB/s 0. 9 GB/s 2, 500 k. B 1, 000 k. B 40 k. B 1, 000 k. B ~60 k. B L 1 Trigger Rate Bandwidth to mass Storage 200 k. Hz 300 MB/s 10 k. Hz 100 MB/s 200 k. Hz 75 k. Hz 1000 k. Hz 0. 6 k. Hz ~ 15 k. Hz 200 MB/s 300 MB/s 100 MB/s 450 MB/s private comm. Event Size CHEP 2007 talk Sylvain Chapelin LHC JLab Front End DAQ Rate * ** * Jeff Landgraf Private Comm. 2/11/2010 ** CHEP 2006 talk Martin. L. Purschke Super. Bigbite DAQ and Electronics Alexandre Camsonne 9/5/2021 37

CODA 3 – What’s different CODA 2. 5 CODA 3 Run Control (X, Motif, C++) (rc. Server, runcontrol) Experiment Control – AFECS (pure JAVA) (rc. Platform, rcgui) Communication/Database (msql, cdev, dptcl, CMLOG) c. Msg – CODA Publish/Subscribe messaging Event I/O C-based simple API (open/close read/write) EVIO – JAVA/C++/C APIs Tools for creating data objects, serializing, etc… Event Builder / ET System / Event Recorder (single build stream) EMU (Event Management Unit) Parallel/Staged event building Front-End – vx. Works ROC (Interrupt driven – event by event readout) Linux ROC, Multithreaded (polling – event blocking) Triggering: 32 ROC limit, (12 trigger bits -> 16 types) TS required for buffered mode 128 ROC limit, (32 trigger bits -> 256 types) TI supports TS functionality. Timestamping (4 ns) Super. Bigbite DAQ and Electronics Alexandre Camsonne 9/5/2021 38

FADC Encoding Example Super. Bigbite DAQ and Electronics Alexandre Camsonne 9/5/2021 39

GTP Trigger Bit Example Super. Bigbite DAQ and Electronics Alexandre Camsonne 9/5/2021 40