Hall D Level 1 Trigger Dave Doughty 1102008

Hall D Level 1 Trigger Dave Doughty 1/10/2008 Hall D Collaboration Meeting

Outline I. III. IV. Intro to the Trigger Photon Energy (Tagger) Track Count Processing Subsystem Processors and Global Trigger Crate V. Global Trigger Processors VI. Trigger Timing VII. Conclusions Christopher Newport University

L 1 Trigger – Current plans • Five separate subsystems – – – Barrel Calorimeter - compute energy Forward TOF - compute number of tracks Start Counter - compute number of tracks (? ) Tagger – compute photon energy (useful at lower lum) • Each subsystem computes continuously – Goal - At speed of the FADC pipelines - 250 MHz • Global Trigger Processor “combines” all five subsystems – 4 level hierarchy: Board ->Christopher Crate -> Subsystem Global Newport-> University

Level 1 Trigger Block Diagram FADC FADC -VXS- -VXS- -Fiber links 12 Crates BCAL SUM -Fiber links 12 Crates FCAL SUM TOF TRACK COUNT TAGGER ENERGY START COUNTER TRACK COUNT FADC PAIR -VXS- ENERGY SUM PROCESSOR SUM/TIME (8 INPUTS) SPECTROMETER -Fiber links 2 Crates * Longest Link * -Fiber links 2 Crates ENERGY SUM PROCESSOR SUM/TIME (8 INPUTS) ENERGY SUM PROCESSOR** SUM/TIME (8 INPUTS) ** Process Track Counts -Fiber link 1 Crates GTP Select FCAL Energy, BCAL Energy, Photon Energy, AND Track Counts F(x) TRIGGER SUPERVISOR --------CLOCK TRIGGER SYNC ROC CONTROL Signal distribution to Front End Crates (Fiber Links) Christopher Newport University

The (VXS) Serial Backplane • Convert Board Data to serial data – 16 bits @250 MHz -> 4 Gbit/s • Link speed of 3. 125 Gbps => data rate => 2. 5 Gbps – Packet overhead also! • Multiple “lanes” likely necessary – But two lanes will work fine – With four lanes - redundancy for free! – Higher speed in the future will also help • Use VXS “Switch Fabric” – VXS has “familiarity advantage” of VME at JLAB Christopher Newport University

VXS Crate Christopher Newport University

VXS Serial Routing Christopher Newport University

Tagger in the Trigger • broad-band focal plane hodoscope photon beam exits from tagger inside vacuum, continues 70 m down to collimator cave and Hall D – 144 readout channels – short scint. rods – max rate/chan: 2 x 106 (107) • focal plane microscope – 120 readout channels – 600 scint. fibers – max rate/chan: 2 x 105 (106) • hodoscope exit window is 1 mm Al • other walls of vacuum chamber are 1 cm Al • exposed side of electron exit channel is 5 mm Al Christopher Newport University

Microscope scintillating fiber design n Design parameters q square scintillating fibers q size 2 mm x 20 mm q clear light guide readout q aligned along electron direction for reduced background sensitivity Si. PM sensors clear light fibers scintillating fibers focal plane electron trajectory q readout with silicon photomultipliers (Si. PM devices) devices Christopher Newport University

Microscope electronics q Si. PM “bases” Ø provides bias voltage regulation: regulation Ø Ø regulation range: 48 ± 8 V regulation stability: 0. 1 V regulation control: 0. 1 V steps reasonable packing density: 600 channels on 20 x 103 cm 2 q Programmable constant fraction discriminators: 120 channels q High resolution tdc (F 1 TDC): 120 channels q Pulse height needed: FADC is a good choice: 120 channels q q essential for setup useful for monitoring performance rates alone give incomplete information USED TO PRODUCE TRIGGER! q Scalers counting on all channels: 120 channels Christopher Newport University

Fixed array electronics q Conventional 1” phototubes, bases q High voltage for phototubes: 144 channels q Timing needed: High resolution tdc (F 1 TDC): 144 channels q for measuring coherent bremsstrahlung spectrum q time is compared with pair spectrometer trigger q at low intensities (107) can run concurrently with experiment q at high intensities (108) only for special low-intensity runs q Pulse height needed: FADC may not be a good choice: 144 channels (YES IT IS – if you want to ever trigger on it!) q essential for setup q at high intensities, monitor spectrum in integrating mode q Scalers counting on all channels: 144 channels Christopher Newport University

Tagger trigger electronics • Tagger. OR – fast logical OR of 120 channels from the microscope – sent over fiber to Glue. X trigger electronics • Too primitive! We can do better! – Select channels in microscope – Optionally include main hodoscope – Without any extra hardware! Christopher Newport University

Microscope - “Board Tag Word” • For microscope, not interested in a “sum, ” we want an energy “tag” (or two) • Have one crate of FADCs for the microscope – @16 Flash ADC channels/board -> 8 boards • Use “hit bit” on each FADC channel as energy tag! – 16 bits – every 4 ns! – These 16 bits are exactly like the 16 bit board sum produced by the FADC for the calorimeters, except for the interpretation! • For calorimeters - 16 bit “board sum” every 4 ns • For microscope – 16 bit “board tag word” every 4 ns • Board tag words from all boards sent to “crate tagger processor” – 120 bits into crate tagger (at 250 MHz) Christopher Newport University

Microscope Photon Energy Tag • Use one “Switch Slot” for Photon Crate Trigger Processor – FADCs in payload slots – All “board tag words” sent serially to Photon Crate Energy Tagger – (Note that Photon Crate Trigger Processor is same hardware as Crate Energy Summer – both are now called Crate Trigger Processors) • Photon Crate Trigger Processor (CTP) produces a 120 bit composite energy tag word from all microscope channels • Can download an energy “mask” to Photon CTP – CTP board is loadable via TI board! • Address of two “highest” or “lowest” energy tag bits are sent to Tagger Subsystem Processor – Need 7 bits to encode 120 bits, with 16 bit word can encode 2 tag bits – The “rate conundrum” Christopher Newport University

For Hodoscope – Duplicate the System? • With 144 channels need 8 bits to encode each energy tag. • Two hits still fit into one 16 bit word! • No new development! • Option to include in trigger (or use by itself) Christopher Newport University

Track Processing – at Front End • Easy in principle - just add up discriminators – – – How many layers (are there multiple layers)? Which disc setting (is there a separate for trigger? ) Programmable time window At pipeline resolution of 250 MHz - 4 ns time stamp Only one crate of electronics for TOF? Use for Start Counter as well? • Create crate track count – Use VXS backplane for communication – Can send any data desired over backplane – Track CTP computes total track count in crate • Time stamped track count data sent to Track Subsystem Processor (TSP) Christopher Newport University

From Crates to Global Processor • Must move crate data (energy, track counts, photon energy) to the global trigger processor crate. • Use four lanes of fiber optics! Christopher Newport University

HFBR-7934 Fiber Module - POP 4 compliant Fiber Optic Module. Uses 12 fiber MTP/MPO, 50/125μm multi-mode. - 8 Used Fibers: 4 Tx, 4 Rx @ 3. 125 Gbps each for a total bandwidth of 12. 5 Gbps in each direction - Small footprint allows 8 modules to fit VME 6 U slot - up to 150 m range with 500 Mhz-Km fiber, or up to 350 m range with 2000 Mhz-Km fiber - Interfaces directly with Xilinx/Altera gigabit transceivers on FPGA Christopher Newport University

Crate Sum -> Energy Sum Processor Christopher Newport University

The Global Trigger Crate • Global Trigger Crate has eight Sub. System Processors (SSPs) on one logical “Side” • Global Trigger Crate has eight Global Trigger Processors (GTPs) on other logical “Side” • SSPs are connected to the GTPs via a custom backplane – “VXS-Like” – 8 x 8 Fully Connected • Each SSP talks to each GTP via a four-lane Aurora Backplane Link • Each SSP sources eight four-lane links to the backplane • Each GTP sinks eight four-lane links from the backplane Christopher Newport University

The Global Trigger Crate (logical view) VME/ VXS bcal ESPs fcal tof strt phot ESPs TSPTSPESP SSP Array GTP Array Clk/ Trig Out In Christopher Newport University

The Global Trigger Crate (physical view) VME/ VXS GTP Array bcal ESPs fcal ESPs tof strt phot TSPTSPESP GTP Array Clk/ Trig Out In SSP Array. Christopher Newport University

Global Crate Backplane Layout • In Logical Layout 64 4 -lane signals must be routed across center • In Physical Layout 32 4 -lane signals must be routed across center • In “Normal” VXS 18 4 -lane signals are routed to each of two center boards • Will route! Christopher Newport University

A Similar but Smaller Backplane (Bustronic) Christopher Newport University

Subsystem Processors (SSPs) • • Each of the subsystem processors resides in Global Trigger Crate All subsystem processors are same physical PC boards! Each SSP can receive up to eight four-lane “crate data links” Each SSP has multiple Xilinx FPGAs for receiving and processing data from crates • Some SSPs use two boards (because of crate count) – If so – both board “Partial Results” sent to global processor • Eight SSPs are needed: – – – Two for BCAL – Energy Subsystem Processor (ESP) Two for FCAL – Energy Subsystem Processor (ESP) One for TOF – Track Subsystem Processor (TSP) One for Start Counter – Track Subsystem Processor (TSP) ? ? One for Tagger – Photon Energy Subsystem Processor One spare! • Each subsystem processor sends time-stamped Subsystem Event Reports (SER) to all Global Trigger Processors Christopher Newport University

Global Trigger Processors (GTPs) • Run at the same 250 MHz pipeline speed as the subsystems • Receive (via backplane) data from all subsystems – – – – TTOF - Tracks Forward TOF EFCal 1 - Energy Forward Calorimeter (partial sum 1) EFCal 2 – Energy Forward Calorimeter (partial sum 2) EBCal 1 – Energy Barrel Calorimeter (Partial sum 1) EBCal 2 – Energy Barrel Calorimeter (Partial sum 2) Tagger – Photon Energy STRT – Track count in Start Counter ? • Adds partial sums (but can use independently) • Adjusts data to match time-stamps • Does “global trigger computation” – Z >= TFM*TTOF + EFM*EFCal + RM*((EFCal +1)/(EBCal + 1)) – Or something similar Christopher Newport University

GTPs - II • Each GTP has multiple internal processors (FPGAs), each looking for one of four triggers • With eight GTPs allows 32 different triggers • Output to electronics is “Programmable Delay” after computed trigger event time – Produces a result with constant latency (relative to timestamped data) – Simplifies pipeline “lookback” for front end electronics - it’s fixed • Interfaces with TS Christopher Newport University

Trigger Timing – Board sum 64 ns – Transfer to crate sum board 512 ns – Crate sum 64 ns – Link to subsystem 512 ns – Subsystem trigger processing 256 ns – Transfer SER to GTP 512 ns – GTP 256 ns – Level 1 output to FEE 256 ns TOTAL = 2. 432 m. S - design FEE for 3. 5 ms Christopher Newport University

Conclusion – Where is Level 1? • In relatively good shape! – Thanks to electronics group here at JLAB – Have plans and/or proof of concept for most of system – Minimal number of boards to be developed (but of course more “software” versions) • Crate Trigger Processor (CTP – simple prototype done) • Sub. System Processor (SSP) • Global Trigger Processor (GTP) • Things we still need – Re-do of L 1 simulation • Current simulation dates from 2001 • Chinese post-doc here at the lab – Track Count Processing Christopher Newport University

SSP ->Energy Sum Processors (ESPs) • Used for both BCAL and FCAL • Logic of both ESPs is essentially same as crate sum • Each ESP uses data from all (up to eight) its crates • Matches time-stamps and adds data – Knows time-stamp offsets for its subsystem (if any) • Passes energy sum results to all trigger processors – SER looks like: <Hdr><Time><Energy> – Time resolution is 4 ns Christopher Newport University

Where the ESP sits in the Pipeline Christopher Newport University

ESP Block Diagram (from Ben) Christopher Newport University

SSP ->Track Subsystem Processor (TSP) • • Used for TOF and perhaps Start Counter Logic of both TSPs is essentially same as crate sum Each TSP uses data from all its crates Matches time-stamps and adds data – Knows time-stamp offsets (if any) • Passes total track counts when over threshold to trigger processors • SER looks like: <Hdr><Time><Count> – Time resolution is 4 ns Christopher Newport University

SSP -> Tagger Energy Processor (TEPs) • Process data from Microscope and Hodoscope (? ) • Matches time-stamps to correlate (if desired) data • Pass tagger energy results to global processor (in same crate) – SER looks like: <Hdr><Time><Photon Energy> – Time resolution is 4 ns Christopher Newport University