DCM II DCM function DCM II design conceptual

DCM II • DCM function • DCM II design ( conceptual ? ) – DCM II system bock diagram – DCM II block diagram – Data flow • Possible PHENIX bandwidth upgrade path • Data volume for some of the upgrade detectors

DCM II Function • Collect FEM data • Zero suppressed data • Data Error/ Event Alignment checking – Alignment checking via L 1 data or/and adjacent data link • Pocketsized data • Possibility for data processing • First stage of event building – we need to handle possible link errors associate with increasing radiation. It may not be practical any more to stop run when one link drop out during the run.

DCM II system diagram Custom Backplane BUSY DOWNLOAD READBACK DATA L 1 Partitioner II DCM II JSEB II FEM DCM II INTERFACE PC GTM L 1 PC SEB Download/reback 40 Mbytes/sec DCM data link 640 Mbytes/sec DCM II/ JSEB II link 6. 4 Gbits/sec. FEM/DCM 16 bits/80 MHz clock The DCM II system is similar to the DCM – the JSEB II will use PCI express bus (2. 5 Gbits/sec/per lane) in PC. The new JSEB will have by-directional optical links.

Conditions for building DCM II • We will use new optical serializer/de-serializer for the FEM to DCM II link. – a) faster speed, b) more reliable protocol and c) GLINK is probably will be phase out some point in the future – Penalty is the data word is only 16 bits wide. • We will use FPGA to replace DSP function. – The old zero suppression code is running at speed while DSP program still being improving – FPGA can process data in parallel with pipeline operation. • It is easier to build speed into the system than the increasing buffer memory size – New chip set just run faster… – Running faster require less memory to buffer the data • DCM I is using VME crate with semi-custom dataway. – The VME market is getting smaller, crate controller run slower than PC and cost more (5 K vs 1 K ? ) and require special operating system (for example Vx. Work) – Except for busy fanout, nobody else use the DCM crate. • VME 64 X backplane mechanical structure has problem bend pins – The DCM II will use complete custom dataway with proper 2 mm HM and 2 mm HM ZD connectors for keying. – We will use PC + DCM interface board to replace VME crate controller

DCM II Block Diagram ALTERA STRATIX I EP 1 S 30 Optics+ De-serializer 5 event buffer Zero suppression Event buffer 320 Mbytes/sec Alignment Zero suppression Event buffer ALTERA STRATIX I EP 1 S 30 Alignment 640 Mbytes/sec Data link out 5 event buffer L 1 data Event Buffer Optics+ De-serializer Data processing Event buffer Zero suppression receiver 5 event buffer MUX Optics+ De-serializer MUX FEM DCM link 80 MHz 16 bits/word 1. 6 Gbits/sec 8 b/10 b encoding method choose Ti’s TLK 2501 as de-serializer STRATIX has 2 Mbits for event data buffer 1 Mbits for processing buffer 30 K logical element old DCM has 1 K Logical element per optical link ALTERA STRATIX II EP 2 S 60 Data link in Event buffer Zero suppression receiver 5 event buffer MUX Optics+ De-serializer 8 optical links per DCM STRATIX II has 1 Mbits for event data buffer 1. 5 Mbits for processing buffer 20 K - 60 K logical element faster than STRATIX I

Power Possible floor plan for the DCM II De-serializer Control/ download Optical transceiver Zero Suppression FPGA High speed ZD connector FPGA To merge data

DCM II parameters • FEM link run 80 MHz 16 bits mode – 50 microsec/12. 5 ns = max 4 K words per events • Assume at least factor 10 zero suppression factors. • 8 optical ports per DCM. – We could use half the optical port if detector has less than factor 10 zero suppression rate. • The available buffer size will increase by factor 2 • Input 8*80 Mwords (16 bits per word)/sec, max output rate is 160 Mwords (32 bits/per word) /sec. • Large data buffer at Partition III, ½ MWords. • Paritioner III to JSEB II link will be running at around 4 -6 Gbits/sec. • DCM II board size currently is 6 Ux 160 mm, slightly smaller than size of A 4 paper.

JSEB II interface to PC through PCI express bus. The PCI express has 2. 5 Gbits/sec per lane. Max. 12 lanes per interface. The current local bus to PCI express bus only support one lane. Serve both slow control path and DCM II path. Use optical cable to both paths. Smaller size and less distance constrain. DCM data optical Download Control/ Slow readback Serializer De-serilaizer FPGA Altera Arria 4 lanes 10 Gbits/sec

Future DAQ upgrade choices • WE have the following choices – A) Restore the bandwidth by factor of 2 • Max 10 KHz Max 20 KHz. – B) Make sure everything could run up to 10 KHz trigger rate. – C) Make everything run as fast as it can without big investment.

Current DCM count • 74 DCM with FE 1 ( 20 bits wide data word). – – TEC 42. PC 10. DC 20. Mu. ID 2, • 93 DCM with FE 2(16 bits wide data word). – – Muon Tracking EMCAL RICH other detectors 56. 24. 4. 9. • Exclude RXN, HBD and test stands. • Excluding spares.

Cheapest (minimum change) solution to improve bandwidth Matt’s 6/23/07 mail Comments: • • • • Packet 22006 3145 3011 3125 22011 23001 22010 21031 21032 21022 1003 1002 Mean Sigma 318. 2 230. 9 294. 4 49. 1 288. 6 41. 8 288. 3 42. 3 273. 4 198. 9 258. 0 0. 0 238. 9 209. 8 233. 0 86. 3 226. 8 74. 8 219. 2 85. 0 215. 0 0. 0 Detector HBD DC DC DC HBD RXNP HBD MPC MPC BBC Bank swapping run at 6. 4 KHz Without HBD and DC Next year we will have rebuilt HBD, data size could be different. We could also double no. of HBD fibers or reduce number of samples. Can we cover the GLINK optical system to 8 b/10 b system? readout through DCM II If we solve the problem with DC, MPC, RXNP and BBC, will DAQ run fast enough?

Solution to achieve 10 KHz rate in DCM (even with non-zero suppressible data) • Reuse the DCM daughter cards • Build a new card with DCM II backend architectures. – use FPGA to merge 4 inputs to one 320 Mbytes/sec link. • Need to build 180+ new cards or 720+ links – Assume $200 -$300 per links + EDIA etc.

Possible upgrade path to restore the full trigger rate • What we know – DCM run $1 K per link. • 16 bits word size for FE 2, 20 MHz data rate. • 20 bits word size for FE 1, 40 MHz date rate. – DCM II is about. 5 K -. 7 K • 16 bits word size, 80 MHz data rate. • The DCM II is about factor 4 faster and about factor 2 cheaper in price compare to DCM equib with FE 2. – As time goes by, it will harder and harder to rebuild DCM. – Possible methods to restore the bandwidth • Rebuild some detector with 16 bits readout mode to DCM II to free up old DCM boards for other detectors. • Use those free up DCM boards to readout other old detectors.

Possible upgrade path to restore the full trigger rate (II) • Muon Tracking use 56 DCM with FE 2. – The cost double DCM for Mu. Tr will be 224 K + cost on Mu. Tr FEE copper to fiber crate. – The cost to readout Mu. Tr completely through DCM II will be readout through 56 *4 ports per DCM * factor 2 / 4 (number data packet could stuff through DCM II) = 112 optical ports • Cost will be. 6 k*112= 67. 2 K + cost plus the cost of rebuild Mu. Tr copper to fiber crates. • This would free up 56 FE 2 DCM boards. • To free up FE 1 DCM is more complicated. Possible solutions could include – Rebuild DC FEM optical output interface. – Build optical data format exchange board. • GLINK in and 8/10 b optical out.

First look at New detector data volume

VTX barrel pixel • Pixel data – – – – – 5 header words +4 SPIRO words +5 trailer word 2 groups 2048 16 bits data word (y/n, 1 bits per pixel) Pixel calculated occupancy for Au-Au central is 0. 53% and. 16%. assume 1% noise rate --- estimate zero suppress data volume Split data into 8 bits group, for un-zero byte, attached with 6 bits label to form 16 bits data word, additional word every 64 bytes. 1 % occupancy mean => every 64 bytes will have 7 16 bits word So average data volume for pixel data is = (7+1)*64 groups * 2 sets / (32/16 packing) = 512 words +7 = 519 words compare to 4. 1 K words un-zeo suppressed data volume 8 optical fiber per DCM => 4136 words 32 bits word per events => 41 Mwords per sec for 10 KHz trigger rate. Pixel has 60 fibers.

VTX barrel strips • • Silicon strips is through SVX chip. 128 channel per SVX 4. 12 SVX per RCC. 5/6 RCC per ladder. 18 ladder for R 3 and 26 ladder per R 4. SVX has one board zero suppression (one threshold level). ~4. 5% occupancy in R 3 and ~2. 5% occupancy in R 4. Assume 1% noise. For every RCC, there are 2 “identify” words. For every SVX 4 words there is 1 address word. 10 header+trailer word per packet. Data size for Au-Au central – = 10+{5 (RCC) *2 + 60(SVX)* (1 header + 7 data words) (~128*. 0. 055)}/2 = 255 32 bits words per ladder R 3 – = 10+{6 (RCC) *2 + 72(SVX)* (1 header + 5 data words) (~128*. 0. 035)}/2 = 232 32 bits words per ladder R 4 • Total data volume is – (255*18+232*26) =10. 6 K words per Au-Au events.

FVTX • Channel counts = (73. 7+160+160)K =553. 7 K, 48 wedges per disk. – Assume one fiber per wedge. • 3 (5) bits FADC per channel = only readout hit channel = 1 16 bits word per channel • 2. 8 % occupancy in au-au central plus 1 % noise. • Data volume = 3. 8% * 553. 7 K channel = 21 K 16 bits words + all header + address words…

RPC, NCC and HBD • RPC measures time. The RPC 2 and RPC 3 should have similar hit rates as Muon ID, 20 hits for Au-Au mini bias. We need to look at RPC 1 occupancy for Au-Au events • NCC has 2 EM, 1 HAD sections plus 2 strips plans readout by SVX 4. – EM and HAD readout through multi-samples ADC. – I would expect data volume at factor 2 above the FVTX > 21 K words • HBD has 2. 3 K channel. – The data word per channel is 7. Without zero suppression, the data volume is 16 K. The problem is not size of total events but data word per fiber. One could double number fibers or only output summary information.