DUNE FD DAQ Firmware Status October DAQ Sprint

DUNE FD DAQ Firmware Status (October DAQ Sprint Summary) David Cussans DUNE Upstream DAQ Meeting 29/Oct/19

Outline • What did we try to do? • How much did we get done? • Outstanding items? - What can we get done in Monday “DAQ Days”? • How long will it take? 2 David Cussans | DUNE DAQ Firmware

Activities in DAQ Sprint? • Integration of hit finding code with Felix: • Build environment • Interface to central router • Debugging hit finding • Be able to merge pipelines • State save/restore for pedestal subtraction and filtering • Be able to find hits in all time-ticks 3 David Cussans | Firmware Overview

Embarrassingly Parallel • Ten fibres per APA @ 9. 6 GBit/s - 12 -bit ADC samples @ 2 MSample/s - Each fibre 2560 wires. - Induction and collection wires on same fibre • Treat each fibre in parallel - How safe is this? Is it sufficient to perform common mode noise subtraction on a per-fibre basis? 4 David Cussans | Firmware Overview

Firmware: What’s in it? 40 pipelines Per APA 5 David Cussans | Firmware Overview

Firmware: Receive Data • Data on fibres has complicated encoding. • Francesco Gonnella: Birmingham, David Cussans: Bristol, Alessandro Thea: RAL • Now synthesizes for Felix ( chasing timing issues. ) • Needs more testing 6 David Cussans | Firmware Overview

Firmware Framework • Dave Newbold, RAL • 2 MSample/s from ADC, but logic can run much faster - Run processing logic ~ 200 MHz - Data arrives packets containing all channels for a single “time tick” - Rearrange into packets with multiple time ticks for a single channel. - Save/restore state so processing transparent to block size 7 David Cussans | Firmware Overview

Firmware: Pedestal Subtraction, Filter • Kostas Manoulopolous: RAL • Can now cycle through wires 8 David Cussans | Firmware Overview

Firmware: Hit Finding • Kunal Kothekar: Bristol • Issues located and fixed (2 days ago) 9 David Cussans | Firmware Overview

Firmware: Interface to Felix Firmware • Simone Ponzio: CERN , Filiberto Bonini: BNL, Alessandro Thea: RAL • Extremely active. • Working with Alessandro Thea towards hit finding integrated with Felix • Felix code not written as framework for user code • AT has written IPBus / Wupper bridge • SP has written interface from AXI 4 S to Central Router. • Adding ability to filter empty packets 10 David Cussans | Firmware Overview

Firmware: Configuration, Debugging • Dave Newbold: RAL, Simone Ponzio: CERN, Alessandro Thea: RAL • DMN has written code to allow channel-by-channel configuration • e. g. wire-by-wire threshold • Needs testing and debugging • SP, AT working on adding ability to debug hit finding pipeline over IPBus/Wupper • Will be needed to fix issues. 11 David Cussans | Firmware Overview

Firmware: Build Environment • Alessandro Thea, RAL • Challenging to build firmware: - Traditional approach: One engineer and their Workstation • We have distributed team • Mix of framework , user code • Multiple hardware platforms • Tools developed within LHC to facilitate collaborative firmware development - CMS IPBus-build (ipbb) • Felix firmware build tools not yet configured to facilitate distributed development ( e. g. separation of framework and processing code) 12 David Cussans | Firmware Overview

Firmware - a work in progress • Starting from standalone hit finding in ZCU 102 , KU 105 eval boards - No input from WIB (yet) • Integrated with FLX 712 - Issues with timing closure - Debugging in progress - Colour code: - magenta: mgts - blue: central router - green: wupper - pale: regmap - forest: dme if/pcie - red: to. Host fifos - yellow: ipbus infrastructure - light orange: hitfinder data-reception - deep orange: hitfinder - everything else 13 David Cussans | Firmware Overview

Interface to Readout Software • Pierre Lasorak: Sussex , Phil Rodrigues: Oxford • Assistance from Roland Sipos • Writing Art. DAQ Board Reader • Will read hits produced by firmware and pass on in same format as s/ware hit finder • Current status: Can load data into DPR buffer and produce hits on host. • Still a bit "fragile" • Will soon move onto critical path. 14 David Cussans | Firmware Overview

Other Work in Parallel Buffering O(10 s) RAM Non-volatile ( for SN burst storage)

Firmware: Lossless Compression • Patrick Dunne, Abbey Waldron: Imperial 16 David Cussans | Firmware Overview

Firmware: Buffer Management • Erdem Motuk: UCL One DRAM interface Per APA 17 David Cussans | Firmware Overview

Firmware: Non-Volatile Buffer • Roy Wastie: Oxford Two NVMe SSD Per APA 18 NVMe Interface David Cussans | Firmware Overview

Summary • Have not yet demonstrated hit finding on live data at Proto. DUNE • Still possible before LAr emptied. • … but running out of slack: There will be system integration issues. • Working on buffering in parallel. 19 Presenter Name | Presentation Title