The optical readout chain for the ALICE Transition

The optical readout chain for the ALICE Transition Radiation Detector Presentation at IRTG Seminar Felix Rettig 29. Juni 2007

Outline • The ALICE Experiment • The Transition Radiation Detector (TRD) • Optical Readout Interface Board (ORI) – features, high-speed laser modulation – error rates, irradiation test results • Readout datapath components – Global Tracking Unit (GTU) and its sub-components – low-latency transmission for triggering – high-bandwidth buffering of raw data – multi-event data buffering • Current Status of GTU development and assembly • Outlook Felix Rettig – KIP – ALICE TRD Global Tracking Unit – IRTG Seminar 2007 -06 -29 p. 3

ALICE Experiment • Pb-Pb collisions at up to 1150 Te. V • 8000 collisions per second • Goal: Quark-Gluon. Plasma • Various detectors: TPC, TRD, . . . A Large Ion Collission Experiment (ALICE) Felix Rettig – KIP – ALICE TRD Global Tracking Unit – IRTG Seminar 2007 -06 -29 p. 4

Transition Radiation Detector • 1. 2 million channels • 65664 MCMs Felix Rettig – KIP – ALICE TRD Global Tracking Unit – IRTG Seminar 2007 -06 -29 p. 5

Transition Radiation Detector Ampl. Cathode Pads • 1. 2 million channels • 65664 Anode MCMs Wires Cathode Wires Drift Chamber Radiator z y Felix Rettig – KIP – ALICE TRD Global Tracking Unit – IRTG Seminar 2007 -06 -29 p. 6

Local Tracking deflection 10 time bins ADC Readout Tree 10 . . . ADC Network Interface Data Buffers Tracklet Processor (TRAP) Filter Preprocessor PASA . . . 18 x. . . Analog signals from 18 cathode pads TRD Front-End Electronics (FEE) Multi-Chip Module (MCM) y • Multi-Chip Module (MCM) – 10 -bit ADC for each of the 18 channels – up to 32 time bins, 10 MHz sampling rate • ADC raw data reduction to stiff "tracklets" characterized by y-, z-position and deflection (compound 32 -bit value) Felix Rettig – KIP – ALICE TRD Global Tracking Unit – IRTG Seminar 2007 -06 -29 p. 7

TRD FEE Readout Tree z y 2 x 72 Channels C C C C Board Merger HC Merger ORI-Board Optical laser link to GTU located outside of L 3 magnet 2 x per Module 12 or 16 Pad Rows The full TRD comprises • 65664+ MCMs on 4104 ROBs • 540 modules in 90 stacks • 1080 optical readout links • Readout Board (ROB): – 16 MCMs in readout tree – 4 column mergers – 1 board merger • Module: – 2 x 4 (3) Readout Boards – 2 Half-Chamber Mergers – 2 Optical Readout Interface Boards (ORI) • half chamber data concentrated by HC Merger • optical transmission from ORI to Global Tracking Unit outside of L 3 magnet Felix Rettig – KIP – ALICE TRD Global Tracking Unit – IRTG Seminar 2007 -06 -29 p. 9

Optical Readout Interface Board HC-MCM LVDS 120 MHz CPLD 125 MHz 2. 5 GHz 8 B/10 B Serial. Laser Driver Laser Diode I²C bus ROB PROM Key features: • conversion 120 MHz 8 -bit DDR to 125 MHz 16 -bit SDR • 8 B/10 B encoding and serialization to 2. 5 Gbit/s • high-speed modulation of laser driving signals • 850 nm Laser Diode (VCSEL) with Monitoring Diode Felix Rettig – KIP – ALICE TRD Global Tracking Unit – IRTG Seminar 2007 -06 -29 p. 10

Optical Readout Interface Board Connectors to Readout Board LVDS Transceivers CPLD Laser Driver Laser Diode 8 B/10 B Encoder Serializer Felix Rettig – KIP – ALICE TRD Global Tracking Unit – IRTG Seminar 2007 -06 -29 p. 11

Laser Diode Iforward • Vertical Cavity Surface Emitting Laser Diode (VCSEL) C • built-in monitoring diode for power regulation feedback Felix Rettig – KIP – ALICE TRD Global Tracking Unit – IRTG Seminar 2007 -06 -29 p. 12

Laser Modulation - Shunt Switching Laser Driver APC Mode CURRENT ATTENUATOR LOG AMP ADC IMD IM Rng - IMD Nom IMD TC 1, 2 Temp Sensor ADC TC IIMM tc 1/tc 2 1, 2 IM Nom diff. data signal S + Temp Comp + - APC Gain DAC Servo Controller Source Current RT TNom Temp Comp IM Gain + - C IS Servo Controller Modulation Current S IM ILD DAC Laser Diode + ADC Peaking DAC IM Rng Felix Rettig – KIP – ALICE TRD Global Tracking Unit – IRTG Seminar 2007 -06 -29 p. 13 Monitor Diode

Laser Modulation - Eye Diagrams 250 ps eye closed 8 m. A Laser Forward Current eye closed clear eye opening • big parameter space: 15 setup parameters in combination with various properties of board components determine low- and high-speed characteristics of laser driving signals • hard to find a common setup to be used for all 1080 boards • very fast signals to be optimized, intricate to measure Felix Rettig – KIP – ALICE TRD Global Tracking Unit – IRTG Seminar 2007 -06 -29 p. 14

Laser Modulation - Eye Diagrams 250 ps eye closed 125 ps 6 m. A 8 m. A Laser Forward Current eye closed clear eye opening • huge parameter space: 15 setup parameters in combination with various properties of board components determine low- and high-speed characteristics of laser driving signals • hard to find a common setup to be used for all 1080 boards • signals to optimize are very fast, intricate to measure final setup with "peaking" Felix Rettig – KIP – ALICE TRD Global Tracking Unit – IRTG Seminar 2007 -06 -29 p. 15

First Supermodule installed at CERN minimum power for BER <10 -10 Number of Optical Links ORIs in first Supermodule at CERN Optical Power [µW] • high reliability of data transmission achieved: Bit Error Rates < 10 -15 • broad safety margin ensures sufficient optical power in case of laser degeneration • laser diodes operating in lower region of forward current and optical power output Felix Rettig – KIP – ALICE TRD Global Tracking Unit – IRTG Seminar 2007 -06 -29 p. 16

ORI Irradiation Tests • Irradiation tests carried out at OCL • 27. 7 Me. V proton beam • All components meet safety margin of factor 4 life-time Assumptions about TRD radiation environment: DIP ~ 1. 6 Gy, Dtot ~ 1. 8 Gy in 10 years of operation Felix Rettig – KIP – ALICE TRD Global Tracking Unit – IRTG Seminar 2007 -06 -29 p. 17

Full TRD Readout Chain 18 x GTU Racks 5 x 6 x Tracklets only Half Chamber ORI Module 12 fibres ORI Track Concentr. Trigger Design Trigger Handling & Control RX Tracklets & Raw Data TGU Event Buffering Design DCS Board TTCrx DDL SIU D-RORC TMU Central Trigger Processor SMU DATE Software Stack 5 x GTU Supermodule Segment Front-End Electronics within L 3 magnet Mass Storage Racks below Muon System DAQ System • Global Tracking Unit (GTU) • Track Matching Unit (TMU) • Supermodule Unit (SMU), Trigger Generation Unit (TGU) Felix Rettig – KIP – ALICE TRD Global Tracking Unit – IRTG Seminar 2007 -06 -29 p. 18

ALICE Trigger Timing Collision Pre Trigger Level-1 Trigger Level-0 Trigger Drift Time Fit Calculation Tracklet Building Raw Data Shipping Tracklet Shipping Global Tracking TRD Level-1 Trigger Contribution Shipping 0 1 2 3 4 5 6 7 8 Time after Collision [µs] Collision 1. 2 Level-0 Trigger 6. 2 µs Level-1 Trigger Level-2 Trigger Window 73. 8 µs Accept/Reject 493. 8 µs Tracklets Data Forward to DAQ Raw Data Time after Collision Felix Rettig – KIP – ALICE TRD Global Tracking Unit – IRTG Seminar 2007 -06 -29 p. 19

GTU Requirements • 1080 optical links deliver both – tracklets: low-latency required, small volume, high rate – raw data: big volume, latency subordinate, lower rate • huge bandwidth – each opical link: 1. 94 Gbit/s – links into one TMU: 23. 3 Gbit/s – full detector: 2. 1 Tbit/s = 244 GByte/s • fast track reconstruction and decision taking for TRD's level-1 trigger contribution • uni-directional optical links, no flow control – buffering of incoming raw data with full bandwidth • forward buffered event data to DAQ system or discard Rettig – KIP for – ALICE TRD Globalinterlacing, Tracking Unit – IRTGbuffering Seminar 2007 -06 -29 p. 20 • Felix support trigger of multiple events

TMU Requirements • low-latency deserialization and 8 B/10 B decoding of optical data received from front-end electronics • 12 independent 16 -bit data streams at 125 MHz clocks • single 4 -Mbit SRAM with 128 -bit interface at 200 MHz • multi-event buffering of up to five events, appropriate data forwarding or discarding on L 2 messages • error detection / handling for each separate link • aborting of incomplete events, discarding associated data • delicate balance between speed and ressource utilization to accomodate trigger and buffer designs in one FPGA → complex FPGA design, many clock domains, highly pipelined data path design needed to allow for operation at 200 MHz Felix Rettig – KIP – ALICE TRD Global Tracking Unit – IRTG Seminar 2007 -06 -29 p. 21

10 GBASE-R Block Sync PMA PCS Clock Control 10 GBASE-R Decode RX Data Flags Clock Dividers Phase Align 16 x 52 ring buffer Clocks 8 B/10 B Decode 8 B/10 B Descram Comma Detect Fabric Interface SIPO 2. 5 GHz serial data Receiving of Optical Data • optical-electrical converter modules • Virtex-4 FX 100 FPGA • 16 Multi-Gigabit Transceivers – highly configurable, ~100 parameters – clock reconstruction – deserializing – comma detection, comma aligning – 8 B/10 B decoding – rate matching • incoming 16 -bit data streams synchronized to one 125 MHz clock Felix Rettig – KIP – ALICE TRD Global Tracking Unit – IRTG Seminar 2007 -06 -29 p. 22

Event Buffering Design SRAM Controller data Scheduling Event Info FIFO Buffer Memory Management Event Shaper SMU Control (L 0 -/L 1 -Trigger) status Data Formatting read address write Event (n+1, 11) . . . 12 x. . . Event (n, 11) empty Event (n+1, 0) Data Block, Link 11 TMU/SMU Interface Read Address Logic & Control Readout Unit SMU Control (L 2 -Message) Felix Rettig – KIP – ALICE TRD Global Tracking Unit – IRTG Seminar 2007 -06 -29 p. 23 Supermodule Unit 16/128 Data stream Merging 16/128 . . . 12 x. . . Links from one Stack 4 -Mbit SRAM Event (n, 0) Data Block, Link 0

Data Stream Merging • 12 independent 16 -bit data streams at 125 MHz with gaps • single SRAM with 128 -bit wide interface at 200 MHz raw data commas . . . 12 links. . . END 128 END L 0 L 1 16 128 -bit Line tracklet data . . . 12 blocks. . . 16 SRAM • collation of 16 -bit values to 128 -bit lines for each link • padding at the end of datastreams • 12 data path clock domain crossings Felix Rettig – KIP – ALICE TRD Global Tracking Unit – IRTG Seminar 2007 -06 -29 p. 24

Data Collation • Block-RAM primitives Link Data 16 BRAM 0 – 16 -bit / 32 -bit collation 32 – safe datapath clock domain crossing BRAM 1 Data Valid 128 Line Data BRAM 2 write enable BRAM 3 End Marker = write address ring counter read address FIFO 125 MHz clock domain Aligning Buffer Last Line Advance 200 MHz clock domain Line Valid – buffering of filled 128 -bit lines until scheduled forward to SRAM write • commas discarded, only valid data words are collated • write enable logic performs padding without need for extra endmarker write cycles • end of data flag stored explicitly, no 32 -bit compares necessary in later stages Felix Rettig – KIP – ALICE TRD Global Tracking Unit – IRTG Seminar 2007 -06 -29 p. 25

Buffer Organization • • SRAM divided into separate parts, one for each link blocks independently organized as ring buffers data of multiple events dynamically double write pointers allow discarding of incomplete events, directly freeing occupied memory locations • overrun protection, anticipatory busy signaling . . . 12 x. . . end (n+1) end (n) Event (n+1, 11) Data Block, Link 11 Event (n, 11) empty memory Event (n+1, 0) 4 -Mbit SRAM Event (n, 0) Data Block, Link 0 end (n+1) end (n) Felix Rettig – KIP – ALICE rp TRD Global Tracking Unit – IRTG Seminar 2007 -06 -29 rp p. 26

Buffer Address Management Event n (incomplete) Event n-1 Event n-2 Data Block, Link 0 . . . 12 x. . . A B Write pointers Read Pointer base address R 18 offset + SRAM write address Write Pointer A 0 Write Pointer A 1 1 0 Bank A + event end pointer Write Pointer B 0 Write Pointer B 1 0 Bank B Block RAM write address read address • 2 write pointers – first word offset – current offset • normal operation: – first constant – increment current • event complete: – pointers swap meaning • discard incomplete event: – use first word pointer, copy to current pointer • only one RAM block needed for all pointers, no 12: 1 multiplexers Felix Rettig – KIP – ALICE TRD Global Tracking Unit – IRTG Seminar 2007 -06 -29 p. 27

Event Buffering Design - Detailed Felix Rettig – KIP – ALICE TRD Global Tracking Unit – IRTG Seminar 2007 -06 -29 p. 28

Event Buffering Design - Detailed Felix Rettig – KIP – ALICE TRD Global Tracking Unit – IRTG Seminar 2007 -06 -29 p. 29

Event Buffering Design - Detailed Felix Rettig – KIP – ALICE TRD Global Tracking Unit – IRTG Seminar 2007 -06 -29 p. 30

Event Buffering Design - Detailed Felix Rettig – KIP – ALICE TRD Global Tracking Unit – IRTG Seminar 2007 -06 -29 p. 31

Tests and Integration Progress • GTU segment running in Münster for super-module assembly and testing • all segments being assembled and tested currently • transfer to CERN and installation scheduled for end of july • some CTP signal handling and DDL/DAQ issues open • trigger design not yet integrated Felix Rettig – KIP – ALICE TRD Global Tracking Unit – IRTG Seminar 2007 -06 -29 p. 34

Prospective Work • Finish development of GTU – a lot of open detail issues – multi-event buffering under full CTP control • Installation at CERN • Beam and performance tests, refinements • Monitoring system, GTU event display • Physics applications of GTU – various trigger schemes based on comprehensive track and pt information, e. g. Jet Triggers Felix Rettig – KIP – ALICE TRD Global Tracking Unit – IRTG Seminar 2007 -06 -29 p. 35

Thank You for Your Attention Questions? Felix Rettig – KIP – ALICE TRD Global Tracking Unit – IRTG Seminar 2007 -06 -29 p. 36

TMU Board - Final Layout Felix Rettig – KIP – ALICE TRD Global Tracking Unit – IRTG Seminar 2007 -06 -29 p. 37

SMU Board - Final Layout Felix Rettig – KIP – ALICE TRD Global Tracking Unit – IRTG Seminar 2007 -06 -29 p. 38

TMU-SMU-Hybrid Felix Rettig – KIP – ALICE TRD Global Tracking Unit – IRTG Seminar 2007 -06 -29 p. 39