Belle II ECL DAQ subsystem ZHULANOV Vladimir Budker

Belle II ECL DAQ subsystem ZHULANOV Vladimir Budker Institute of Nuclear Physics Novosibirsk, Russia Beijing 2011. 01. 25

ECL electronics Infrastructure (1) analogue signals ~10 m PA PA PA 12 Ecl. Shaper. DSPb connected to one Ecl. Connector custom digital channel Ecl. Collector ~1 m analogue signals, clock FAM JTAG over LVDS 100 Mbit Ethernet LVDS levels ~ 1 m 16 crystals & pre. Amps connected to each Ecl. Shaper. DSPb Ecl. Shaper. DSP Belle 2 link VME backplane LVDS levels TTD

ECL electronics Infrastructure (2) Ecl. Shaper. DSPb Ecl. Shaper. DSPb T T D Ecl. Collector VME crate There are 52 ECL VME crates We don’t plan to install VME controller at the crates F A M Modules: • Ecl. Shaper. DSPb – shaping pulses from Cs. I pre. Amps, digitizing and digital processing. • Ecl. Collector – merges data from Ecl. Shaper. DSPb modules and sends the data to the DAQ system • TTD – provides timing & trigger information for Ecl. Collector • FAM – provides arguments for global trigger from ECL. The crates have non-standard power supplies! ± 7. 5 V (instead of ± 5. 0 V) and ± 15 V (instead of ± 12 V)

Ecl. Shaper. DSPb (1) • Continuous digitizing shaped pulses, fitting with reference waveform and restoration of the amplitude and time of the pulse. • Sample frequency is 1. 77 MHz = 508. 9 MHz/288 and is synchronous with electron bunches. • It is important for calculation of pulse time to have trigger time information referenced to the sample frequency and to have the restored peak time at the same clock domain. • There a lot of precomputed coefficients used in DSP and being stored in SDRAM chips The computed data for one channel is packed into one 32 -bit integer: 18 bits for amplitude, 12 bits for time, 2 bit for quality flags

Ecl. Shaper. DSPb photo

Ecl. Shaper. DSPb (2) – digital part Two 133 MHz 16 bit DDR chips (1 GByte/s) SDRAM 1 SDRAM 2 16 x 18 -bit ADCs @ 1. 76 MHz ALTERA CPLD Xilinx FPGA ADC SCK=42. 33 MHz, TKN CONF LVDS => CMOS DESERIALIZER 66 MHz XTAL DAC DAC 16 x 64 -taps DCPs (Digitally controlled potentiometer) C O L STP cable L (1 m) E C T O R

Ecl. Shaper. DSPb (3) – FPGA design SDRAM 1 ADC DATA PROCESSOR + buffer for 512 samples 16 channel (290 mks) SDRAM 2 DSP PACKAGER Control logic F(clock) =132 MHz=66 MHz*2 PLL ADC control sequence Control logic F(clock)=127 MHz=42. 33*3 S e r i a l i z e r D e s e r i / l a z e r ADC data is stored in cyclic buffer for 290 mks and is overwritten with the new data after that!!! Output can be one of three cases: • only processed data, • only raw ADC data, • both ADC and processed data SCK (42. 33 MHz) TKN (synchronization signal) 60 samples used in processing, and 22 of them are already get before trigger signal. 60*567 ns = 34 mks = 12 mks (before trigger) + 22 mks(after trigger) Up to 8 trigger events can be processed simultaneously The processing of one trigger can be delayed for upto 290 mks Data transferred to Ecl. Collector @ 528 Mbps (66 MHz *8) 60 ns for 32 bit word

Ecl. Shaper. DSPb(4) – output data packat Offset High 16 bit Low 16 bit 0 Magic number = 0 x 5 AA 5 FF 00 1 B 15. . 8 = 79 (packet ID) B 7. . 0 = 16 (packet type – Trigger command) 67 (packet length) 3 B 22. . B 16 = 60 – number of ADC samples per channel B 28. . 24 = 1 – number of channels with ADC data • B 7. . 0 = 14 – (0. . 143) trigger time • B 12. . 8=2 – number of hit channels (length of DSP data) 4 B 31. . 16 = 0 x 00 C 0 – hit channels mask B 15. . 0 = 1839 Trigger TAG 5 0 B 31. . 16 = 0 x 0400 – ADC data mask 6 B 31. . 30 - quality flag , B 29. . 18 – time, B 17. . 0 – amplitude (ch. 7) 7 B 31. . 30 - quality flag , B 29. . 18 – time, B 17. . 0 – amplitude (ch. 8) 8. . 67 ADC samples (60 samples, ch. 11) 68 ch. 16 Hit channel mask 0 x 00 C 0 No raw ADC data: data packet size = 6… 22 32 -bit words data latency min = 22. 4 mks = 22 mks (ADC data ready) + 0. 4 mks (transfer to Ecl. Collector) data latency max = 279. 3 mks 278 mks (ADC data ready, processing & transfer inside FPGA) + 1. 3 mks (transfer to Ecl. Collector) Raw ADC data present: data packet size = 966… 972 32 -bit words data latency max = 336 mks 278 mks (ADC data ready, processing & transfer inside FPGA) + 58 mks (transfer to Ecl. Collector) CRC 32 ch. 1 0 0 0 0 1 1 0 0 0 ch. 16 ADC data mask 0 x 0400 ch. 1 0 0 0 0 Data latency to Ecl. Collector = 22… 336 mks. Expected average latency is 30 mks

Tasks of the ECL collector module. 1. 2. 3. 4. 5. 6. Configure Xilinx FPGA on Shaper. DSPb modules Synchronize sampling process of Shaper. DSPb modules Collect data from 12 connected Shaper. DSPb modules Calibration signal generation Interface with remote PC for stand-alone operation Provide interface with Belle II TTD and DAQ 9

Ecl. Collector(1) Block diagram calib DAC Ethernet PHY PC SER/DESER LVDS drivers, SCK, RCK FPGA Xilinx Virtex 5 Rocket. IO LVDS Clock FAM Flash memory 512 MByte DAQ TTD

Ecl. Collector(2) clock domains The requirement for TTD: provide any clock frequency synchronous to el. bunches & trigger signal referenced to this frequency FLASH memory Merge of large data patches can consume a lot of time (next slide) FGPA Ecl. Shaper. DSPb data Processing unit (FPGA logic now, Embed. CPU planned) FIFO 8 k. Byte x 12 Rocket. IO Ethernet several clocks asynchronous to el. bunches SCK (42. 33 MHz) & TKN for Ecl. Shaper. DSPb DAC TTD clock PLL DAC sequence generator 254 MHz synchronous to el. bunches or on-board OSC 42. 33 MHz

Ecl. Collector(3) data merge FIFO 8 k. Byte FIFO 8 k. Byte x 12 Data merger Continuous buffer Belle 2 link IFC DAQ Even data locations info FIFO map Even data locations info Data transfer time is upto 22. 2 mks (12 ch. * 976 words/ch. * 3. 79 ns/cycle * 0. 5) in case the raw ADC data are read out This time can be saved if the second option is selected So, the requirement for Belle 2 link interface: exclude any data transfers from buffer to buffer – transfer data from FIFO buffers to optical link directly

Data latency summary units are mks (min/avg/max) no raw ADC data present Data latency in Ecl. Shaper. DSP module 22/30/278 29. 3/37. 3/278 Data transfer to Ecl. Collector 0/0. 7/1. 3 58/58. 7/59. 3 Data transfer inside Ecl. Collector 0/0. 2/0. 4 21. 8/22/22. 2 Data transfer through optical link (2. 5 Gbit/sec for data) 0/1/2. 5 147. 5/148. 4/149. 9 Total: 22/31. 9/282. 2 256. 6/266. 4/509. 4

Ecl. Collector(4) Ethernet interface 216 virtual 16 -bit registers 232 virtual 16 -bit memory cells

Ecl. Collector(5) Network processor – other FPGA logic interface reg_num 16 reg_wdata 16 reg_we 1 reg_rdata 16 mem_addr 32 mem_wdata 16 mem_we 1 mem_rdata 16 register file Network processor control signals, commands, status signals buffer 1 memory map buffer 2 buffer 3 Network processor doesn’t know anything about register and memory designations 4 Ethernet actions are enough for the interface : read/write register, read/write memory region I would be happy if Belle 2 link provide means for control and setup ECL (and maybe other) system in a manner like this

FPGA configuration and Startup Ecl. Collector configuration from flash memory OK failed Ecl. Collector configuration via JTAG-over -LVDS interface, flash memory update Ecl. Shaper. DSPb configuration and DSP coefficients load from flash memory OK failed Remote data is used, flash memory update Ready for operation

Status �Ecl. Shaper. DSPb has been developed and the restoration algorithm was checked with it on Belle detector in 2009/2010 season. �Ecl. Collector prototype is manufactured and most of its functions have been tested: Ethernet interface, interface with Ecl. Shaper. DSPb modules, calibration mechanism, Rocket. IO link. �We need to update the Rocket. IO and TTD connection schematics and check them �We need to add JTAG-over-LVDS interface �We need to implement embedded processor �We need to check flash-memory read/write �And finally we need to implement Belle 2 link and TTD logic design and check the overall system perfomance

Than you for your attention!