Achieving Deterministic Latency in a JESD 204 B

Achieving Deterministic Latency in a JESD 204 B Link June 1, 2014

Abstract JESD 204 B links are the latest trend in data-converter digital interfaces. These links take advantage of high speed serdes technology to offer many compelling benefits including improved channel densities and simplified board layouts. JESD 204 B expands upon existing JESD 204 standards by including methods for achieving deterministic latency. This presentation addresses one of the challenges of adopting the new interface - understanding and designing the link latency. The concepts and examples presented will enable a system designer to: – Understand the tradeoff between link latency and tolerance to link delay variation – Guarantee deterministic latency across the JESD 204 B link – Use a formulaic and procedure-based approach to design the link latency 2

Table of Contents • Defining latency in a JESD 204 B link • Deterministic latency intro • Guaranteeing deterministic latency • Steps to achieve deterministic latency • Verifying deterministic latency • Bonus!

Latency in a JESD 204 B Link

What is Latency? • Latency is defined as the total time (in seconds or clock cycles) it takes a signal to travel from Point A to Point B • In a system: – Point A might be the input to the ADC – Point B might be the output of the DAC • In a JESD 204 B link: – Point A is the input to the JESD 204 B transmitter – Point B is the output of the JESD 204 B receiver’s elastic buffer

JESD 204 B Latency Influencers • Device dependent – SYSREF propagation delay – JESD link layer – Serdes implementation • Board level – Channel (trace length, vias, connectors) – SYSREF propagation delay – Clock propagation delay

JESD 204 B System Diagram (Subclass 1) • Data is sent over serdes lanes • Each device receives a “SYSREF” signal to align LMFCs in each device • Each device receives a “device clock” which is essentially the sample clock • The device clock captures the SYSREF signal so setup and hold times must be met

Latency in a JESD 204 B Link

Latency Definitions Link Delay — delay from when the sampled parallel data is input to the serializer at the transmitter (ADC/FGPA) until the same data is presented at the input to the elastic buffer in the receiver (FPGA/DAC)

Latency Definitions Link Latency — the latency from when the sampled parallel data is input to the serializer at the transmitter (ADC/FPGA) until the same data is available in parallel form at the output of the elastic buffer in the receiver (FPGA/DAC)

Which one is Deterministic Latency referring to?

Which block enables Deterministic Latency? Why?

Deterministic Latency Intro

What is Deterministic Latency? • Wikipedia: ”A deterministic system is a system in which no randomness is involved in the development of future states of the system. A deterministic model will thus always produce the same output from a given starting condition or initial state. ” • The most important aspect of deterministic latency is that the latency should stay constant from system startup to startup. • Having “deterministic latency” does not necessarily mean the latency is known • How do we achieve deterministic latency? – JESD 204 B defines two mechanisms for deterministic latency called “subclasses”

JESD 204 B Subclasses Subclass Mechanism for Deterministic Latency Subclass 0 No support for deterministic latency (backward compatible with JESD 204 A). Subclass 1 “SYSREF” signal is used to align LMFCs within all TX and RX devices. Subclass 2 “SYNC” signal is used to align LMFCs, eliminating the need for “SYSREF”. LMFC is aligned based on when the SYNC signal is received from each data converter device. Complex timing means that this will only work below 500 MSPS.

Which Systems May Need Deterministic Latency? • Applications sensitive to latency – Digital pre-distortion (DPD) control loops – Automatic gain control loops (AGC) – Defensive counter measures • Any system that requires multi-device synchronization – Multi-antenna communications systems – Phased array radar – Magnetic Resonance Imaging (MRI)

How Does JESD 204 B Achieve Deterministic Latency? • Sequence of events for subclass 1: 1. SYSREF is sent to all devices at the same point in time to align their local multi-frame clocks 2. All transmitters issue an /A/ or /R/ character at the same point in time based on the LMFC clock edge. - Channel and device variations will skew the lanes relative to each other 3. As each RX receives the /A/ or /R/ character, it starts to buffer it’s data 4. Once all RXs receive their /A/ or /R/ character and the “buffer release point” has been reached (e. g. LMFC rising edge), all RXs release their data from the buffer which is now time aligned

Initial Lane Alignment Sequence • From step 2 on previous slide, each transmitter sends the /A/ or /R/ character at the same point in time by using the initial lane alignment sequence • All transmitters transmit the ILA starting at an LMFC rising edge • The /A/ or /R/ characters are the trigger signal for deterministic latency

Deterministic Latency Visual 1. is used 4. All lanes are 2. SYSREF After SYNC is to align LMFCs released on received, all the lanes 3. Each lane starts next LMFC edge send ILAS on next buffering it’s data and areedge now LMFC when it receives aligned the trigger (R or A char) x

Guaranteeing Deterministic Latency

How Do We Guarantee Deterministic Latency? • There are two requirements needed to guarantee deterministic latency from startup to startup: 1. Guarantee the LMFCs in each device are aligned (or have constant phase difference) every time the system starts 2. Set release point to occur after the latest arriving lane by: • • Having total link delay less than the LMFC period Setting a buffer release point that occurs after all lanes have arrived This is what the standard expects, but it’s not always possible based on hardware tradeoffs (smaller buffers)!

1. Aligning the LMFCs in Each Device • The LMFCs need to be aligned in each device to create a known timing reference point – All devices should receive SYSREF in unison every time the system starts – SYSREF must meet setup and hold times relative to device clocks • Technically the LMFCs can be misaligned, but the phase difference needs to be the same every time the system starts

2. Set Release Point to Occur After the Latest Arriving Lane • The standard says that the total link delay should be less than the LMFC period, so the easy choice is to set the release point at the LMFC edge • The LMFC period is decided by the linerate, F (octets per frame), and K (frames per multi-frame) • LMFC period = 10 * F * K / Linerate • 10 = number of bits per octet • For example, if F = 8, K = 16, and the linerate is 7372. 8 Mbps, then the LMFC period is 173. 6 ns. – In the example, if the total link delay is less than 173. 6 ns then the LMFC period can be chosen as the release point What happens if the total link delay is greater than 173. 6 ns?

Buffer Release Point • The elastic buffer release point can be shifted from the LMFC rising edge by using the RBD parameter • RBD is defined as a shift in the elastic buffer release point from the LMFC rising edge by “RBD” frame periods – So an RBD setting of 4 shifts the release point 4 frame cycles from the LMFC rising edge – Frame period = 10 * F / Linerate – RBD must be between 1 and K. – RBD of K corresponds to the LMFC edge • RBD can be used for: – RBD is used to release the buffer earlier to achieve minimum latency – RBD is used to shift the release point away from the area of uncertainty near the total link delay 24

What Happens with Incorrect Release Point? • Consider the following scenario: • The LMFC period is 50 ns Why is there variation on the data delay? • The delay for each lane is 100 ns +/- 10 ns (90 ns to 110 ns) • Assume the buffer release point is set to the LMFC clock rising edge

The latest arriving lane arrives at 90 ns Buffer releases on LMFC rising edge 50 ns Data delay is 90 ns Total latency is 100 ns

The latest arriving lane arrives at 110 ns Buffer releases on next LMFC rising edge 50 ns Data delay is 110 ns Total latency is 150 ns

Incorrect Release Point • In this example, the latency IS NOT deterministic because it is not consistent from startup to startup • How do we avoid this scenario? 1. Calculate the expected maximum lane delay 2. Choose a release point that occurs after the maximum lane delay

Using RBD to shift the release point Release Point occurs “RBD” frame cycles after the LMFC boundary Set RBD = K/2 50 ns RBD Data delay is 110 ns Total latency is 125 ns for all cases

Achieving Deterministic Latency

3 -Step Process for Deterministic Latency • Deterministic latency can be achieved using this three step process 1. Determine Alignment of LMFCs 2. Calculate expected link delay (accounting for variation) 3. Choose release point that provides margin against error

Step 1: Determine Alignment of LMFCs • Things to consider: – Propagation delay of SYSREF and device clocks across board – Purposefully added delays to SYSREF, such as analog delays or dynamic digital delays (like in LMK 04828) – Delays from SYSREF input pins to resetting of LMFC (internal to devices)

LMFC Alignment Between Devices • The “SYSREF pin to LMFC reset” timing may be different for each device • Some datasheets will specify SYSREF to LMFC timing for various modes (e. g. DAC 38 J 84) and this delay needs to be added to the LMFC delay.

DAC 38 J 84 SYSREF to LMFC Latency

Step 2: Calculate Link Delay Variations occur due to device variances (specified in datasheet) or due to PVT effects These delays will come from the datasheet or need to be measured Total Link Delayor= calculated t +t +t TX_SER LMFC alignment between TX and RX has already been determined LANE RX_DESER +/- t. VARIANCE

Step 3: Choose Buffer Release Point Choose release point that guarantees enough margin around the expected lane arrival

Summary of Calculation • We have done the following: 1. Determined LMFC alignment between TX and RX 2. Calculated total link delay 3. Determined the appropriate buffer release point 2 1 3

Total Link Latency • The total link latency can be calculated using the following formula • “n” is the number of whole RX multi-frames traversed for the case where the total link delay exceeds one LMFC cycle Link Latency = (n * K + RBD) * TFRAME + (t. RX_LMFC – t. TX_LMFC) n=2 1 2

Add ADC or DAC latency for Total Latency

Verifying Deterministic Latency

Example System Diagram

Deterministic Latency Test Setup 1. Generate a pulse with FPGA 2. Capture FPGA pulse with ADC 3. Output ADC’s MSB from FPGA 4. Observe relative timing on scope 5. Power up the system many times to confirm the relative timing stays constant

Device Setup for Test • ADC = ADC 16 DX 370 (16 -bit, dual channel, 370 MSPS) – Device clock = 370 MHz (2. 7 ns) • JESD 204 B Parameters – L = 4, M = 2, F = 1, S = 1, K = 32 – Frame cycle = 10 * F / Linerate = 10 * 1 / 3700 Mbps = 2. 7 ns – LMFC cycle = Frame cycle * K = 2. 7 ns * 32 = 86. 4 ns • FPGA – Device clock = 92. 5 MHz (10. 8 ns)

Additional Delays in System Account for non-device related delays in calculations

Latency Calculation • Per the previous slides, the latency is calculated as Link Latency = (n * K + RBD) * TFRAME + (t. RX_LMFC – t. TX_LMFC) • For this experiment the following parameters apply – All units in Frame clock cycles – t. RX_LMFC = 28, t. TX_LMFC = 3. 5, n = 2, K = 32, RBD=28 – Link Latency = 116. 5 frame clock cycles • The following also extend the latency – ADC 16 DX 370 core latency = +12. 5 – DEVCLK routing skew and MSB output routing delay = +1. 4 – SYSREF/DEVCLK sampling skew = +1. 5 – Additional receiver processing delays • +3 rx_tdata release delay (for earliest sample in the 4 -sample set output each period of 92. 5 MHz clock. This delay may instead be +4) • +4 latching MSBs before output (at Fs/4 rate) • Total Calculated Latency = 116. 5 + 12. 5 + 1. 4 + 1. 5 + 7 = 138. 9 cycles

Result • Calculated latency for this setup is 138. 9 frame cycles • Measured latency is 379. 6 ns / 2. 7 ns = 140. 4 cycles • Note: Would have been better to use a short pulse as test signal to avoid confusion over correct output pulse Latency=379. 6 ns

Bonus!

What’s the easiest way to achieve deterministic latency? 1. Setup a test similar to the one shown to observe relative delays 2. Vary RBD until a 1 LMFC period latency jump is observed 3. Choose release point by taking the last RBD value before the latency jump was observed and add the expected latency variation to that value (plus extra margin) Release Release Point Total Latency Optimal Release Point

End! 49