TTC upgrade The TTCPON project S Baron on
TTC upgrade The TTC-PON project S. Baron - on behalf of the TTC-PON team Sophie Baron – TTC Upgrade for TTC-PON – BE/BI Technical Board – 10 Nov 2016 1
2 • Defining ‘TTC upgrade’ for this talk • Upgrade Requirements • From TTC to TTC-PON Sophie Baron – TTC Upgrade for TTC-PON – BE/BI Technical Board – 10 Nov 2016 Outline
Definition CANDIDATE FOR TTC UPGRADE Sophie Baron – TTC Upgrade for TTC-PON – BE/BI Technical Board – 10 Nov 2016 3
TTC system - overview P 2 CCR P 8 TTC backbone TTC off-detector TTC on-detector Sophie Baron – TTC Upgrade for TTC-PON – BE/BI Technical Board – 10 Nov 2016 P 5 P 4 – RF SOURCE Bunch Clocks, Orbits Introduction 4
Introduction 5 TTC backbone TTC off-detector TTC on-detector RF_Rx RF 2 TTC TTcrx Central Trigger Unit Local Trigger Unit TTCex Sophie Baron – TTC Upgrade for TTC-PON – BE/BI Technical Board – 10 Nov 2016 Generic TTC in experiments
Introduction 6 Trigger unit TTCex 1 10 1 16/32 DAQ TTCrx FE Controller TTCrx FE driver TTCrx Det. Module TTCrx DAQ FE driver OFF Detector Det. Module Sophie Baron – TTC Upgrade for TTC-PON – BE/BI Technical Board – 10 Nov 2016 Generic TTC in experiments – block diagram
Introduction 7 Trigger unit Proposed Upgrade: TTC-PON TTCex 1 10 1 16/32 DAQ TTCrx FE Controller TTCrx FE driver TTCrx Det. Module TTCrx DAQ FE driver OFF Detector Det. Module Proposed Upgrade: GBT Versatile LInk Sophie Baron – TTC Upgrade for TTC-PON – BE/BI Technical Board – 10 Nov 2016 Generic TTC in experiments – block diagram
Requirements TTC OFF DETECTOR UPGRADE Sophie Baron – TTC Upgrade for TTC-PON – BE/BI Technical Board – 10 Nov 2016 8
• • • Bi-directionality Increased bandwidth Backward compatibility with legacy TTC system Common system Scalability Partitioning flexibility Sophie Baron – TTC Upgrade for TTC-PON – BE/BI Technical Board – 10 Nov 2016 Upgrade Requirements TTC OFF Detector: Basic Requirements 9
• Downstream, broadcast/unicast: – Machine Timing Signals with programmable phase, low, deterministic and fixed latency: • Bunch Clock (40. 079 MHz) to be distributed and recovered with extremely low jitter to be used as a reference clock for high speed links • Orbit/Frev (11. 2 k. Hz) commonly used as BCR – Experiments’ Synchronous Signals with programmable latency: • Triggers to be transmitted at maximum 40. 078 MHz with low and fixed latency, together with minimum 8 bits of Trigger Type • Event Counter resets • Calibration Pulses – Low to medium rate Asynchronous Signals: • Region of Interest Readout • Event routing information (IP address? ) • Slow control Sophie Baron – TTC Upgrade for TTC-PON – BE/BI Technical Board – 10 Nov 2016 Upgrade Requirements Bi-directionality 1 0
• Upstream: – Feedback and acknowledgment for control – Busy/throttle signals transmitted with short and bonded latency – Fiber latency monitoring Sophie Baron – TTC Upgrade for TTC-PON – BE/BI Technical Board – 10 Nov 2016 Upgrade Requirements Bi-directionality 1 1
PON for TTC: introduction and basics Downstream Path Upstream Path System Performance Overview Next steps FROM TTC TO PON Sophie Baron – TTC Upgrade for TTC-PON – BE/BI Technical Board – 10 Nov 2016 12
• • • Unidirectional system 1: 32 split ratio 1310 nm 40 Mb/s per channel Busy/throttle on a separate link Sophie Baron – TTC Upgrade for TTC-PON – BE/BI Technical Board – 10 Nov 2016 Current TTC system 13
• Introduction to PONs (Passive Optical Networks) – Bidirectional – Two wavelengths (1/direction) • Downstream (OLT->ONU) – High bandwidth • Upstream (ONU->OLT) – TDMA (shared bandwidth) Sophie Baron – TTC Upgrade for TTC-PON – BE/BI Technical Board – 10 Nov 2016 Introduction to the TTC-PON system 15
• TTC-PON system Sophie Baron – TTC Upgrade for TTC-PON – BE/BI Technical Board – 10 Nov 2016 1: 64 15 Introduction to the TTC-PON system
• OLT →ONUs • Broadcast and continuous serial stream • LHC Bunch Clock (BC) synchronous 1: 64 • • ONUs →OLT Time Domain Multiplexing between ONUs Shared bandwidth, lower line rate LHC Bunch Clock (BC) synchronous (BC Sophie Baron – TTC Upgrade for TTC-PON – BE/BI Technical Board – 10 Nov 2016 Introduction to the TTC-PON system 16
• System – Split ratio/Power Budget – Level of customization wrt standards • Downstream – Trigger latency – Bandwidth – Clock quality • Upstream – Busy latency – Dynamic range – Payload Sophie Baron – TTC Upgrade for TTC-PON – BE/BI Technical Board – 10 Nov 2016 Main figures of merit of TTC-PON 17
1577 nm 9. 6 Gbps 1270 nm 2. 4 Gbps Sophie Baron – TTC Upgrade for TTC-PON – BE/BI Technical Board – 10 Nov 2016 TTC-PON 2016 18
OLT Sophie Baron – TTC Upgrade for TTC-PON – BE/BI Technical Board – 10 Nov 2016 ONU 1 ONU 2 ONU 3 19
PON for TTC: introduction and basics Downstream Path Upstream Path System Performance Overview Next steps FROM TTC TO PON Sophie Baron – TTC Upgrade for TTC-PON – BE/BI Technical Board – 10 Nov 2016 20
Sophie Baron – TTC Upgrade for TTC-PON – BE/BI Technical Board – 10 Nov 2016 Downstream Path • Principle and Bandwidth • Performance • Power Budget and split ratio • Jitter map • Latency • PON-GBT bridge 21
• • OLT →ONUs Broadcast LHC Bunch Clock (BC) synchronous 9. 6 Gbps serial link 8 B 10 B encoded, K 28(. 1, . 5) comma Full Payload: 24 bytes (200 bits) per BC including trigger and control. User Field: 20 bytes (160 bits) Bunch clock period (25 ns) 10 G PON-TTC D D K 28 Ad C 1 C 2 D D D D 24 bytes Current TTC 1 0 Channel A (Trigger field, 1 bit) Channel B (Control, 1 bit) 2 bits 22 D D K 28 Ad Sophie Baron – TTC Upgrade for TTC-PON – BE/BI Technical Board – 10 Nov 2016 Downstream Protocol
Split Ratio & Power Budget Upstream 1 splitters Measured Minimum Tx Power 3. 68 d. Bm (OMA) Target BER : 10 -12 Confidence Level: 0. 95 Minimum Receiver Sensitivity (OMA) -19. 2 d. Bm Power Budget 22. 88 d. B Splitters (1: 64) -20. 5 d. B Connectors - 1 d. B Fiber length -0. 05 d. B Margin 1. 33 d. B 4 d. B A FEC will help gaining some margin 23 Sophie Baron – TTC Upgrade for TTC-PON – BE/BI Technical Board – 10 Nov 2016 Downstream BER & Split Ratio
Errors observed seem to have random nature (different from GBT environment) o Binary BCH codes are good for correcting random errors with a relatively low complexity [5] n-bits o Systematic encoding: BCH(n, k) INFORMATION – k bits o Four shortened-BCH codes were evaluated: o BCH(40, 34) Single-error o BCH(80, 73) correcting o BCH(120, 113) Double-error o BCH(120, 106) correcting o Main Figures of merit: o Efficiency (k/n) o Coding gain o Latency o Timing o Complexity (area) o Scrambling: signal randomizer o Self-synchronous scrambling: no sync. overhead but error multiplication… Scrambler FEC encoder Channel FEC decoder Descrambler 24 PARITY Sophie Baron – TTC Upgrade for TTC-PON – BE/BI Technical Board – 10 Nov 2016 Downstream path
o Measurements Results: Gain of 3 d. B of power budget + Better efficiency Double-error correcting Single-error correcting * Decoder parameters considered (latency, slice luts, timing slack on Kintex 7) 25 Sophie Baron – TTC Upgrade for TTC-PON – BE/BI Technical Board – 10 Nov 2016 Downstream path
Sophie Baron – TTC Upgrade for TTC-PON – BE/BI Technical Board – 10 Nov 2016 Downstream Jitter Map – PON only KC 705 Kintex 7 OLT 240 MHz gen - CG 635 1: 8 1577 nm 9. 6 Gbps 1270 nm 2. 4 Gbps ONU PLL -Si 5338 40 MHz / PLL-Si 5345 KC 705 Kintex 7 120 MHz phase-noise analyzer 26
PON for TTC: introduction and basics Downstream Path Upstream Path System Performance Overview Next steps FROM TTC TO PON Sophie Baron – TTC Upgrade for TTC-PON – BE/BI Technical Board – 10 Nov 2016 27
Upstream Path Sophie Baron – TTC Upgrade for TTC-PON – BE/BI Technical Board – 10 Nov 2016 • Principle and Bursts composition • Challenges • Synchronizing all ONUs • Recovering data burst by burst • Fine alignment of bursts • Performance • BER & Power Budget • Dynamic Range • Temperature • Latency 28
Sophie Baron – TTC Upgrade for TTC-PON – BE/BI Technical Board – 10 Nov 2016 Upstream Principle Based on Burst Mode… Gap Preamble Comma/Addr Payload • Major difficulties for burst-mode optical receivers (OLT_Rx) • Short settling time • Large dynamic range • Major Challenge for the system • Short gaps alignment • Major challenge for FPGA • Short Bursts synchronization @Rx 10/02/2016 GPON OLT Module Rx Reset Pulse during gap • Synchronize all ONUs Tx, • perform fine calibration No CDR fast enough, use of x 4 oversampling *not in scale E. Mendes 29 33
Dealing with short gaps and high dynamic range…. ONU 1 ONU 2 Kintex 7 OLT Rx_Reset KC 705 10 G GPON specs: • Minimum Gap: 25 ns • Minimum Reset Pulse Width: 25 ns • Maximum Setting time: 52 ns 30 Sophie Baron – TTC Upgrade for TTC-PON – BE/BI Technical Board – 10 Nov 2016 Upstream Principle
Upstream Principle Sophie Baron – TTC Upgrade for TTC-PON – BE/BI Technical Board – 10 Nov 2016 Adjusting each part of the burst… Comma 25 ns 58. 3 ns Gap Preamble Comma+Addr Payload Total Gap+Burst Latency (64 onu’s) Average data-rate 10/02/2016 /Addr Payload 8. 3 ns 33. 3 ns 25 ns 58. 3 ns 33. 3 ns (64 b)* 125 ns 8 us** 8 Mbps (64 b/8 us) E. Mendes Busy waiting time *or 25 ns if we decide to extend the preamble ** for 64 ONUs. 4 us for 32 ONUs 31 33
• Link synchronization => Clock recovery and re-use for transmit path (@ONU level) • Phase changing between bursts → classical CDR is not an option => Oversampling scheme (@ OLT RX level) • TDM arbitration (token is automatically passed between ONUs) Sophie Baron – TTC Upgrade for TTC-PON – BE/BI Technical Board – 10 Nov 2016 Upstream Challenges => Requires a calibration procedure 10/02/2016 E. Mendes 32 11
Sophie Baron – TTC Upgrade for TTC-PON – BE/BI Technical Board – 10 Nov 2016 Upstream Challenges Link synchronization @ ONU … • Clock recovery and re-use for upstream transmission 10/02/2016 E. Mendes 33 12
Sophie Baron – TTC Upgrade for TTC-PON – BE/BI Technical Board – 10 Nov 2016 Upstream Challenges …Oversampling scheme @ OLT Principle x 4 -oversampling 10/02/2016 E. Mendes 34 13
Sophie Baron – TTC Upgrade for TTC-PON – BE/BI Technical Board – 10 Nov 2016 Upstream Challenges Oversampling scheme @ OLT… Architecture Blind. Over. Sampler (BOS) 35
Upstream Challenges Sophie Baron – TTC Upgrade for TTC-PON – BE/BI Technical Board – 10 Nov 2016 L TDM arbitration & Calibration… Suppose that we have a 7 time units burst length (6 u. burst + 1 u. gap) L From OLT – broadcast a reset signal (K 28. 1) every T units (T=n_onu * burst_length; Here T=21 u. ) OLT_TX … ONU 1_RX ONU 2_RX ONU 3_RX cntr rst … OLT_RX *arbitrary units for illustration 6 u. 1 u. 36
BER & Power Budget … 1: 64 with a lot of margin Upstream 1 Splitter Measured Minimum Tx 2. 79 d. Bm Power Minimum Receiver Sensitivity -22 d. Bm Power Budget 24. 79 d. B Splitters (1: 64) -20. 5 d. B -20. 5 Connectors -1 d. B -2 Fiber length -0. 05 0 Margin 3. 04 d. B 6 d. B 37 -24 d. Bm Sophie Baron – TTC Upgrade for TTC-PON – BE/BI Technical Board – 10 Nov 2016 Upstream Performance
DR Dynamic Range… ONU 2 ONU 1 …no penalty up to 12 d. B !! 38 Sophie Baron – TTC Upgrade for TTC-PON – BE/BI Technical Board – 10 Nov 2016 Upstream Performance
PON for TTC: introduction and basics Downstream Path Upstream Path System Performance Overview Next steps FROM TTC TO PON Sophie Baron – TTC Upgrade for TTC-PON – BE/BI Technical Board – 10 Nov 2016 39
• System: – Split ratio: 1: 64 / >3 d. B margin – Dynamic Range: 12 d. B (!) – Immune to temperature variations • Downstream: – Bunch Clock Synchronous – Low latency – 85 ns – 9. 6 Gbps, 6. 4 Gbps User Bandwidth maximum • Upstream – – Bunch Clock Synchronous 2. 4 Gbps, bandwidth shared between max 64 ONUs Waiting Time per ONU: Nx 125 ns, with N<=64. 8 us for 64, 4 us for 32. Payload: 64 bits per ONU every cycle of Nx 125 ns. 40 Sophie Baron – TTC Upgrade for TTC-PON – BE/BI Technical Board – 10 Nov 2016 1: 64 System Performance Overview
Manager (Software) • Core design – Similar structure for OLT/ONU* – Currently implemented in Kintex 7 CTRL TX RX – Software for: Configuration Calibration Diagnostics PHY-CTRL – Firmware delivered to users MGT • • • * ONU can be also controlled via OLT 41 Sophie Baron – TTC Upgrade for TTC-PON – BE/BI Technical Board – 10 Nov 2016 TTC-PON Core - firmware
Sophie Baron – TTC Upgrade for TTC-PON – BE/BI Technical Board – 10 Nov 2016 System Performance Overview Resource & Costing… • Resources (of a middle range Kintex 7): – OLT =~ 1% Slices LUT – ONU =~0. 5% Slices LUT • Cost – SFP+ OLT: 965/660 USD – XFP+ OLT: 1075/660 USD – SFP+ ONU: 258/110 USD 42
• FIRMWARE: – First version of TTC-PON core IP has been delivered for Kintex 7 and Kintex Ultrascale (for ALICE CTP) • HARDWARE: – TTC PON FMC designed & produced (Generic for OLT & ONU) • Next steps: – – – Migration to Arria 10 for LHCb & ALICE CRU Long BER tests Temperature tests on OLT/ONU and Si 5344 Device selection & Market survey Investigate new PON standards & proprietary solutions for Phase-2 upgrades • • XGS PON, NG-PON 2 … Xilinx BCDR 43 Sophie Baron – TTC Upgrade for TTC-PON – BE/BI Technical Board – 10 Nov 2016 Next steps for TTC-PON
Sophie Baron – TTC Upgrade for TTC-PON – BE/BI Technical Board – 10 Nov 2016 THANK YOU
- Slides: 44