DUNE FD DAQ Firmware David Cussans DUNE US

DUNE FD DAQ Firmware David Cussans DUNE US DAQ Meeting 7/Jan/19

Outline • What are the functional blocks? • Who is writing them? - (Timing firmware not covered) • Current challenges • (Outline) plan for 2020 • Summary 2 David Cussans | DUNE DAQ Firmware

Functional Blocks 3 David Cussans | Firmware Overview

Felix Framework • Provides - Handling of WIB fibre input - Data routing ( "central router" , FIFOs , etc. ) - PCIe Interface ( "Wupper" ) • Hit finding chain integrated with Felix framework - Filiberto Bonini, Simone Ponzio, Alessandro Thea • Integration challenging - Felix not set up for easy integration of "user code" - IPBus/Wupper addressing philosophy different • AT has written an IPBus<-->Wupper bridge • Trying to move closer to ATLAS Phase 2 developments - FB - Fed improvements from DUNE back to Felix master 4 David Cussans | Firmware Overview

Data Reception/ Reordering • Turns WIB packets ( multiple channels, single time-tick ) into processing packets ( single channel, multiple time-ticks) • Francesco Gonnella, Birmingham, David Cussans, Bristol • Status: - Working - Needs optimization ( uses ~ 5 – 10 x more logic than necessary – can move to RAM ) 5 Presenter Name | Presentation Title

Hit Finding • Filtering, Pedestal Subtraction - Kostas Manolopoulos, RAL - Status: Working • Hit finding - Kunal Kothekar, Bristol - Status: Works in isolation. Undergoing system level debugging 6 Presenter Name | Presentation Title

Compression • Abbey Waldron, Patrick Dunne, Imperial - Status: • Core works. • Compression factor 2. 6 on Proto. DUNE data - Needs further work on wider range of data • Needs integration with AXI 4 S based framework • Stalled due to Near Detector work 7 Presenter Name | Presentation Title

Buffering – 10 second • FIFO using external DDR 4 RAM - Need ~ 1 DIMM per APA • Receives event selection requests and returns event fragments • Erdem Motuk, UCL • Status: - Core written - Needs integration with synthesizable test bench - Needs integration with Felix framework - “Parked” while Erdem helps with hit finding - Format of selection requests probably needs iteration 8 David Cussans | Firmware Overview

Buffering – 100 second • Writing to NVMe attached SSDs • No file system - Index data by block number • Need 2 NVMe SSDs per APA • Two approaches - Write from Zynq Processor Subsystem - Write directly from FPGA fabric 9 David Cussans | Firmware Overview

Buffering – 100 second, Zynq CPU • Roy Wastie, Oxford • Achieved 1. 6 GBytes/s sustained write • Need to show PL PS NVMe chain and integrate with 10 s buffer • Need to show writing with > 1. 8 GBytes/s per drive to 2 drives 10 David Cussans | Firmware Overview

Buffering – 100 second, FPGA Fabric • Given contract to FPGA consultancy to write Open Source FPGA Fabric NVMe driver - BEAM Ltd • Have done work for CERN accelerator division • Very close to Bristol • Setting up contract suffered significant delays • “Kick off meeting with BEAM” 10/Jan/19 • Not due to finish until ~ May 11 David Cussans | Firmware Overview

Current Challenges • Hit finding chain integrated with Felix exists - … but is buggy - Data corruption, lock-up - Adding monitoring and probes to chain • Alessandro Thea - Debugging individual components • Kunal Kothekar, Kostas Manolopoulos, and others - Integrating firmware hit finding with Art. DAQ • Pierre Lesorak, Sussex, Phil Rodrigues, Oxford - Getting ready to verify that hits produced by firmware hit finding are correct. • Raul Stein, Pieter Keenan, Bristol 12 Presenter Name | Presentation Title

Current Challenges • Hit finding chain integrated with Felix exists - Writing IPBus attached test-bench to write data in WIB format and receive data in Central router format • Erdem Motuk • Test on dev boards rather than Felix - Numerous dev boards available - Felix has proven tricky to set up - Testing/verifying firmware • Tony Earle, Sussex, et. Al. • Time consuming • Helps to enforce documentation if separate “development” and “verification” teams…. 13 Presenter Name | Presentation Title

Towards Proto. DUNE 2 • Demonstrate functionality in Proto. DUNE 2 • Aiming to build relationship with ATLAS Phse-2 upgrade Felix developers - Felix firmware already builds for Ultrascale+ - Join Felix firmware team to enhance Felix tools • ATLAS Phase-2 upgrade framework better able to host project specific functionality (? ) • Positive response to far from FELIX team. 14 David Cussans | Firmware Overview

Summary • Working to demonstrate Upstream DAQ functionality - Trigger primitive production • Moving effort to this to recover from slippage - O(10 s) Buffer - O(100 s) Buffer - Suitable firmware design flow • Integrating project planning with Jim B’s top level US DAQ plan - See talk on 6/Jan/20 in General DAQ meeting • Focus on hit finding in Proto. DUNE at DAQ sprint • Then focus moves to Proto. DUNE 2 - Demonstration of DUNE functionality. 15 Presenter Name | Presentation Title