STAR HFT The STAR Heavy Flavor Tracker PXL

  • Slides: 19
Download presentation
STAR HFT The STAR Heavy Flavor Tracker PXL detector readout electronics J. Schambach (University

STAR HFT The STAR Heavy Flavor Tracker PXL detector readout electronics J. Schambach (University of Texas at Austin) G. Contin, L. Greiner, T. Stezelberger, C. Vu (LBNL) X. Sun (CCNU) M. Szelezniak (IPHC)

Talk Outline STAR HFT • Introduction: STAR, HFT, PXL • Electronics: Hardware • Electronics:

Talk Outline STAR HFT • Introduction: STAR, HFT, PXL • Electronics: Hardware • Electronics: Firmware • Interfaces: DAQ, USB, Slow Controls • Summary J. Schambach University of Texas at Austin TWEPP 2015, Sep 28 – Oct 2, 2015 2

Heavy Flavor Tracker @ STAR @ RHIC STAR HFT Magnet HFT TPC PXL IST

Heavy Flavor Tracker @ STAR @ RHIC STAR HFT Magnet HFT TPC PXL IST SSD TPC – Time Projection Chamber: main STAR tracking detector HFT – Heavy Flavor Tracker SSD – Silicon Strip Detector IST – Intermediate Silicon Tracker PXL – Pixel Detector R (cm) SSD Tracking inwards with graded resolution: r = 22 = ~300 m IST r = 14 PXL r 2 = 8 r 1 = 2. 8 J. Schambach University of Texas at Austin = ~1 mm TWEPP 2015, Sep 28 – Oct 2, 2015 = ~250 m = <30 m 3

HFT Subsystems STAR HFT Silicon Strip Detector (SSD) • Double sided silicon strip modules

HFT Subsystems STAR HFT Silicon Strip Detector (SSD) • Double sided silicon strip modules with 95 µm pitch • Existing detector with new faster electronics • Radius: 22 cm Intermediate Silicon Tracker (ITS) • Single sided double-metal silicon pad with 600 µm x 6 mm pitch • Radius: 14 cm Pi. Xe. L detector (PXL) • MAPS sensors with 20. 7 µm pitch pixels • Radius: 2. 8 and 8 cm First MAPS based vertex detector at a collider experiment J. Schambach University of Texas at Austin TWEPP 2015, Sep 28 – Oct 2, 2015 4

PXL Detector Design STAR HFT 2 layers 10 sectors 2 detector halves 5 sectors

PXL Detector Design STAR HFT 2 layers 10 sectors 2 detector halves 5 sectors / half 4 ladders / sector MAPS Sensor “Ultimte-2” Ladder with 10 MAPS sensors (~ 2× 2 cm each) 20 cm RDO Buffers / Drivers MAPS 2 -layer kapton flex cable with Al traces • • • PXL Sensor 928 rows * 960 columns =~ 1 M pixel Rolling shutter, column-parallel readout through end-of-column discriminators 185. 6 us integration time MIP Signal ~ 1000 e-, S/N ~ 30 Configuration via JTAG J. Schambach University of Texas at Austin • • In-pixel Correlated Double Sampling (CDS) On-chip zero-suppression and run-length encoding on rows (up to 9 hits / row) 2 memory banks of 1500 words each for frame readout in ping-pong configuration 2 LVDS data outputs @ 160 MHz TWEPP 2015, Sep 28 – Oct 2, 2015 5

Sector Readout Electronics Chain 11 m (24 AWG TP) STAR HFT Clock, config, data,

Sector Readout Electronics Chain 11 m (24 AWG TP) STAR HFT Clock, config, data, power 2 m (42 AWG TP) Clk, config, data Mass Termination Board signal buffering + latch-up protected power RDO board with Xilinx Virtex-6 FPGA 100 m (fiber optic) PXL built events Trigger, Slow control, Configuration, etc. Existing STAR infrastructure J. Schambach University of Texas at Austin PXL Sector DAQ PC with fiber link interface board (ALICE “RORC”) Highly parallel system Ø Ø 4 ladders per sector 1 Mass Termination Board (MTB) per sector 1 sector per RDO board 10 RDO boards total in the PXL system TWEPP 2015, Sep 28 – Oct 2, 2015 6

USB USB USB JTAG JTAG USB JTAG SIU SIU SIU Daughter Card Daughter Card

USB USB USB JTAG JTAG USB JTAG SIU SIU SIU Daughter Card Daughter Card Daughter Card TCD Interface PXL Readout J. Schambach University of Texas at Austin STAR HFT Networking USB Hub Fiber-Optics Hub Power Supplies 356 M pixel readout in a single 9 U size crate TWEPP 2015, Sep 28 – Oct 2, 2015 7

Detailed Sector Readout Architecture STAR HFT Ladder x 4 RDO board x 1 x

Detailed Sector Readout Architecture STAR HFT Ladder x 4 RDO board x 1 x 4 LU prot. power FPGA MTB x 1 J. Schambach University of Texas at Austin SRAM Power Supplies DAQ RDO PC fiber SIU USB TCD I/F Control PC Black – cfg, ctl, clk. path Blue – data path Red – power / gnd path TWEPP 2015, Sep 28 – Oct 2, 2015 Trigger 8

Mass Termination Board (MTB) J. Schambach University of Texas at Austin TWEPP 2015, Sep

Mass Termination Board (MTB) J. Schambach University of Texas at Austin TWEPP 2015, Sep 28 – Oct 2, 2015 STAR HFT 9

Readout Board Block Diagram Events, control JTAG SIU Fiber module control, events FTDI USB

Readout Board Block Diagram Events, control JTAG SIU Fiber module control, events FTDI USB module Xilinx Virtex-6 FPGA trigger J. Schambach University of Texas at Austin Trigger interface BUSY VME P 1 x 4 configuration Flash config STAR HFT Clock, control, data drivers TWEPP 2015, Sep 28 – Oct 2, 2015 VHDCI 68 pin trigger BUSY To MTB 10

Readout Board (RDO) SIU STAR HFT TCD I/F Status LEDs Power Ladder I/Fs V

Readout Board (RDO) SIU STAR HFT TCD I/F Status LEDs Power Ladder I/Fs V 6 USB Clk Config J. Schambach University of Texas at Austin Ø Virtex-6 LX 240 T-FF 1759: • 15 Mb RAM • 720 User I/O Busy • high-performance Select. IO technology Ø FPGA Configuration via either Platform Flash XL (DS 617) or front panel JTAG header Ø USB Interface: FTDI FT 2232 H daughter card Ø DAQ I/F: ALICE DDL “SIU” daughter card Ø Trigger (TCD) interface via daughter card Ø Ladder interfaces on the back of the board via VHDCI connectors: • Data (LVDS) from sensors • Clock to sensors • JTAG for sensor config • I 2 C for MTB monitoring & control Power TWEPP 2015, Sep 28 – Oct 2, 2015 11

Firmware: Readout Requirements STAR HFT 40 sensors per RDO = 80 data LVDS pairs

Firmware: Readout Requirements STAR HFT 40 sensors per RDO = 80 data LVDS pairs at 160 MHz Data consists of bit-serially transmitted run-length encoded hit addresses Average trigger rate in excess of 1 k. Hz Data generation speed (40 sensors x 160 MHz x 2 = 12. 8 Gb/s) must be reduced to fiber speed (2 Gb/s) • RDO burst (buffer) capability must match TPC RDO capability (1 event every 50 μs in bursts up to 8 events) in order to not increase the DAQ dead time • Each trigger will result in a separate event containing a full frame to DAQ: • • frame 1 Trigger time J. Schambach University of Texas at Austin frame 2 frame 3 Event data TWEPP 2015, Sep 28 – Oct 2, 2015 12

Firmware Architecture Trigger STAR HFT Sensor output Clock, JTAG & Control IOdelay Ladder Interface

Firmware Architecture Trigger STAR HFT Sensor output Clock, JTAG & Control IOdelay Ladder Interface Event Readout Fiber Interface J. Schambach University of Texas at Austin Configuration Temperature, Voltage, Current & Latch-up Monitoring Slow Controls USB Interface TWEPP 2015, Sep 28 – Oct 2, 2015 13

Firmware: Event Readout Module Sensor Output STAR HFT Sensor Output 160 Mhz Serial To

Firmware: Event Readout Module Sensor Output STAR HFT Sensor Output 160 Mhz Serial To Parallel Converter . . . 20. . . Serial To Parallel Converter valid(1: 8) & data(32) valid( 1) Trigger & Readout Controller data(32 ) Event Buffer . . . 8. . . Serial To. . . 20. . . Parallel Converter 10 Mhz valid(1: 8) & data(32) Mux (40 X 20) : 40 Trigger Serial To Parallel Converter Mux (40 X 20) : 40 valid( 8) valid( 1) Event Buffer . . . 8. . . Event Buffer Busy feedback Mux (32 X 8) : 32 Mux (32 X 2) : 32 Event FIFO J. Schambach University of Texas at Austin 200 Mhz 50 Mhz TWEPP 2015, Sep 28 – Oct 2, 2015 Fiber (SIU) 14

PXL DAQ Data Format STAR HFT Header token Header Trigger word temperature Event Firmware

PXL DAQ Data Format STAR HFT Header token Header Trigger word temperature Event Firmware version Header Hardware ID RHIC counter system status Hit Block Length ……. reserved Hit Addresses Separator (0 x. CCCC) High Level Trigger Info CRC of all preceding words Ender (0 x. BBBB) J. Schambach University of Texas at Austin Sensor Hit Data TWEPP 2015, Sep 28 – Oct 2, 2015 15

USB Protocol STAR HFT • FT 2232 H chip’s FIFO interfaces to FPGA Firmware

USB Protocol STAR HFT • FT 2232 H chip’s FIFO interfaces to FPGA Firmware • Two levels of protocol: USB-FPGA low level, FPGA High Level Protocol • Low Level Protocol: Separate “WRITE” and “READ” transaction to 2 Firmware FIFOs WRITE Transaction Start of READ Transaction • High Level Protocol: • 32 bit “Command words” interface to “memory-mapped” 16 -bit registers • “WRITE” command word: set configurations or start actions • “READ” command word: results in Data in “READ” FIFO, which can then be read with a READ lowlevel transaction Command Word Format: 31 -28 4 bit control 31 READ/NOT-WRITE 30 -28 not defined J. Schambach University of Texas at Austin 27 -16 15 -0 12 bit address 16 bit data TWEPP 2015, Sep 28 – Oct 2, 2015 16

USB Software Interface STAR HFT • Lowest level interface via lib. D 2 xx

USB Software Interface STAR HFT • Lowest level interface via lib. D 2 xx or lib_ftdi on either Linux or Windows • Four types of memory mapped registers: • • “Configuration” (R/W) “Status” (read-only) “Action” (write-only) – start FPGA transactions like JTAG or I 2 C “Indirect” (R/W) – memory or FIFO interfaces (“Count”, “Address”, “Data”) • PXL library provides convenience functions in C++ and Python • open_ftdi, close_ftdi • read. Reg, write. Reg • read. Mem, write. Mem • Python used for scripting and control GUIs • Interface to STAR Slow controls (EPICS) via EPICS “soft-IOCs” that C++ and Python can read/write J. Schambach University of Texas at Austin TWEPP 2015, Sep 28 – Oct 2, 2015 17

PXL Slow Controls (EPICS and Python) Power Supply and Cooling Monitor STAR HFT PXL

PXL Slow Controls (EPICS and Python) Power Supply and Cooling Monitor STAR HFT PXL control GUI Power up, configure and check for errors Controlled shut down Fast reset Status indicator Error indicator Process monitoring terminal J. Schambach University of Texas at Austin TWEPP 2015, Sep 28 – Oct 2, 2015 18

Concluding Remarks STAR HFT • Readout Performance: • Readout through DAQ tested up to

Concluding Remarks STAR HFT • Readout Performance: • Readout through DAQ tested up to ~3 k. Hz (limit of trigger system) • Typical event rate during 200 Ge. V Au-Au ~ 1 k. Hz with <5% PXL dead time • Typical Au-Au PXL data rate about 120 MB/s • PXL operation highly scripted, very little shift operator intervention needed (mostly just turning PXL on and off) • Latch-up events cleared automatically by RDO firmware • Electronics was used successfully during STAR Run 14 and Run 15 • More than 1 B events taken each run, which allowed STAR to perform direct topological reconstruction of charmed hadrons Thank you for your attention J. Schambach University of Texas at Austin TWEPP 2015, Sep 28 – Oct 2, 2015 19