DDR Signaling and Measurement 1242002 2 Memory Bus

DDR Signaling and Measurement 12/4/2002

2 Memory Bus ‘DDR’ Review § Memory – MCH connects to Memory Mainboard: MCH chip is soldered down to the board DIMM connectors are soldered into holes through the board Mainboard has copper traces which connect the MCH and the connector DIMM boards: Plugs into the DIMM connector ‘Gold fingers’ on one end of this board make contact with springy connector pins Many DRAM chips are soldered to the other side of the board DIMM board has copper traces which connect gold fingers to DRAM chips Memory die’s are inside wire-bond packages called ‘DRAMs’ Dynamic Random Access Memory Anywhere from 9 to 36 DRAMs on one memory board DRAM http: //www. micron. com/products/modules/ddrsdram/index. html 2 DIMMs Introduction 12/4/2002

3 DDR Signaling Overview § Reads & Writes Write: Data is being sent from the mch to be stored inside a dram’s memory Read: Data is being sent from the dram to mch to be directed to some other part of the system (usually CPU) Note: When probing a data or strobe signal you will see that the dram drives (a read) and other times the mch drives (a write). § DDR Transaction Step 1: MCH sends command over command address lines DRAMs reads commands from these lines on next command clock DRAM determines if this command is directed at it DRAM prepares to store or fetch data depending on nature of command Step 2: Data is retrived/sent If Read, every DRAM on one DIMM begin driving their DQ and DQS lines. If Write, all DQ & DQS lines in the mch begin driving First DQS line drives low – the preamble DQ & DQS lines start toggling Ø Ø 64 DQ lines and 9 (or 18) DQS lines. DQ lines hold the data , DQS transitions clock the data Introduction 12/4/2002

4 DDR Signal Groups § CMDCLKs, Chip Selects, Command, Address (Register repeats to each DRAM) § DQ/DQS Introduction 12/4/2002

DDR Signaling – Memory Read § Read case example Step 1: MCH sends command over command address lines Note: Command Address lines are only ever driven by MCH Chip Select 1 is asserted during command, other chip selects unasserted MCH is sending Address and Command bits to the register chip on each dimm. The register chip latches these signals on the next CMDCLK and then forwards the signals to all the DRAMs. Each chip select signal goes to one DIMM. This signal chooses which DIMM the currently sent command is targetting. That way only one DIMM will respond. In this case, it’s DIMM 1. Introduction 12/4/2002 5

DDR Signaling – Memory Read (cont’d) § Data is sent from DRAMs to MCH Step 2: Data is retrieved from the DRAMs of one DIMM Note: DQ&DQS lines are involved in this transfer of actual data There is a several clock cycle delay between when the command was issued (previous slide) and when the data starts being transmitted. This is called the read latency, also called ‘CAS Latency’, and is given in units of CMDCLK cycles (2. 5, 3, …). Note that only one agent is ever using the DQ & DQS lines at one time, in this case it is DIMM A 0. Introduction 12/4/2002 6

7 DDR Command Clock Topology § CMDCLK One CMDCLK differential pair is sent to each DIMM and is driven by pins on the mch itself Introduction 12/4/2002

8 DDR CMDCLK Timings § Cycle to Cycle Jitter § Clock Skew between DIMMs This is not specified except in the routing guidelines for the MCH, which are not publicly available Introduction 12/4/2002

9 Getting Ready for Clock Measurements § Finding Probe points A clock pair is named something like: DDRA_CMDCLK 0 and DDRA_CMDCLK 0_N (P and N side of diff pair) In Allegro Viewer, you can use the search expression ‘*CMDCLK*’ to find all of them The MCH is the driver, so we want to look at the DIMM connectors to find the probe points Remember that board is ‘flipped’ when looking at it from the back DIMM B 0 A 0 B 1 A 1 B 2 A 2 MCH § Scope Considerations: We need to probe both the P and N sides of the differential pair, and then use the MATH SUBTRACT function on scope to see the differential signal Introduction 12/4/2002

10 Getting Ready (cont’d) § Measuring Clock Skew Scope Tips: Deskew all your probes (make sure when measuring the same signal, you get the same result from each probe – the waveforms overlay almost perfectly) CMDCLK is a differential signal: Put one probe on P side of clock Put one probe on N side of clock Use MATH to subtract the two waveforms and get the ‘differential’ signal Measure CMDCLK signal at first DIMM on your bus (say DIMM A 0) Simultaneously measure CMDCLK at another DIMM (say DIMM A 1) Measure the delta in time between the two signals at Vref=1. 25 V Introduction 12/4/2002

11 Getting Ready (cont’d) DT § Measuring Clock Jitter Measure one CMDCLK (using 2 channels of scope and MATH subtraction) Use infinite persistance mode of scope Look at the edge AFTER the one you triggered on Measure the ‘smearing’ of that edge at Vref=1. 25 V NOTE: there is smearing on the edge you triggered on also… Ideally, there would be none Trigger Scope Here Measure Jitter Here Introduction 12/4/2002

12 DDR Data/Strobe overview § DDR Data and Strobe lines These lines communicate the actual raw data 64 data lines per bus, and 2 busses on this board § Source Synchronous transmission of data Before burst, DQ & DQS signals are tri-state (mid-level = 1. 25 V) Burst of data begins with DQS preamble Gives time for bus to settle and DQ’s to goto valid levels DQS begins toggling, each switch latches in all the DQ’s go up and down depending on data to be transferred HIGH = 1, LOW = 0 Burst lasts for multiples of 4 strobes, i. e. 4, 8, 12, … transitions of DQS Reads: DQS is aligned with DQ (Simplifies DRAM design) Writes: DQS is centered between DQ bits (this is more typical and straightforward implementation) Burst ends on DQS preamble Gives time for DQ signals to go back to tri-state (mid-level) Preamble Introduction 12/4/2002

13 DIMM & DRAM types § x 4 vs x 8 DRAMs DIMMs always have 64 DQ lines coming from the board x 4 DRAMs have 4 DQ lines, and 1 DQS line A DIMM with x 4 DRAMs needs 16 DRAMs to get 64 total DQ lines Therefore, also need 16 DQS lines X 8 DRAMs have 8 DQ lines, and 1 DQS line A DIMM with x 8 DRAMs needs 8 DRAMs to get 64 total DQ lines Therefore, also need 8 DQS lines § Single Rank vs Dual Rank Remember that Chip Select chooses which DIMM a command is meant for Some DIMMs accept two separate Chip Selects They behave as if they are 2 separate DIMMs but on one physical board They require double the number of DRAMs, and only have are active at a time Ø Ø X 8 16 DRAMs X 4 32 DRAMs, requires DRAMs to be physically stacked on each other, 2 high Single Rank – one chip select goes to the DIMM Dual Rank – two chip selects go to the DIMM § ECC DRAM ECC Supporting DRAMs have an extra DRAM devoted to holding the extra parity data ECC allows the system to survive if a small number of bits gets corrupted Introduction 12/4/2002

14 Strobe & Data decoder chart § How to determine which DQS goes with a DQ bit: DQS DQ (x 4 DRAMs) DQ (x 8 DRAMs) 0 0 -3 0 -7 1 8 -15 2 16 -19 16 -23 3 24 -27 24 -31 4 32 -35 32 -39 5 40 -43 40 -47 6 48 -51 48 -55 7 56 -59 56 -63 8 Lower 4 ECC Bits 9 4 -7 Not used 10 12 -15 Not used 11 20 -23 Not used 12 28 -31 Not used 13 36 -39 Not used 14 44 -47 Not used 15 52 -55 Not used 16 60 -63 Not used 17 Upper 4 ECC Bits Not used Introduction For example, if you have x 4 DRAMs, and you want to find the setup time for DQ 27, you would need to also probe DQS 3 actually latches DQ 27, not any other DQS or clock. 12/4/2002

15 DDR Data / Strobe topology § DQ and DQS lines have the same topology One DQ line routs to every DIMM On DIMM it routs to either 1 DRAM (single rank) or 2 DRAMs (dual rank) With 4 DIMM Slots, potential for 8 loads + MCH on one DQ line DQ & DQS lines within a group are length matched (see previous slide for groupings) to each DIMM Any mismatch eats into timings Series resistor on each DIMM 0 -Ohm series resistor on mainboard between MCH and first DIMM Parallel termination at end of bus Terminated to Vtt=1/2*Vdd=1. 25 V Ø Vdd is voltage rail for DDR. For DDR-1, Vdd=2. 5 Volts On actual board terminators are ‘RPACKs’ Ø Ø RPACK is 4 (or more) resistors all in one device Saves space on routing § See last slide (from second assignment) for drawing of DQ/DQS Topology Introduction 12/4/2002

DDR Data/Strobe Timings § From Micron Data Sheet For WRITE cycles, the DQS waveform would be shifted a half pulse to the right, as shown Introduction by green arrow and dotted edge 12/4/2002 16

17 Measuring DQ/DQS Signals § How to measure DQ/DQS DQ & DQS are single ended signals Only need one probe per DQ/DQS line Tricky part is getting correct dimm to drive Try tweaking MARS software which runs on target system § Locate probe points on MCH and on DIMM connector Nets are named ‘DDRA_DQ 27’ for the portion before the 0 -ohm series resistor Named ‘DDRA_DQ 27_R’ for the portion after the 0 -ohm resistor DQ 0 is highlighted in yellow. Note that it Highlight both ends touches every other DIMM slot. Why? Introduction 12/4/2002

Lab Assignments 12/4/2002

19 Lab Assignment: Clock skew and jitter measurements § Hand PPT lab report. § Activities: Locate at least four CMDCLK receiver clock probe points on layout. There will be as many CMDCLK signals to measure as DIMMs plugged into the platform Measure skew between the different CMDCLK’s Pick one CMDCLK and measure Jitter on that signal For both measurements think about how large a sample measurement size you need to get accurate and meaningful results § PPT minimum contents Hypothesis: What you are measuring and why it is important. Results Summary: Jitter and Skew Supporting data: Illustrate where probes are placed Describe complete scope setup including trigger settings What voltages did you use for the timing points and why? Skew: Show all clock waveform super imposed on one graph. Ø Determine skew from this Ø Describe issues with this simple jitter method. Jitter: Show simple jitter (infinite persistence) include picture of jitter. Introduction 12/4/2002

20 Project Assignment: Simulation to Lab Correlation § Goal: Tune simulations to measurements § Activities: Pick a DQ Data bit to correlate Locate DDR signal being probed. Determine all segments and component value. You may need to get component values from examining the board. Measure necessary DC voltages Modify the HSPICE DDR template file with actual values. Probe points One probe is on a data signal on the DIMM before the series resistor (End closest to DRAM chip) The other probe is on a the same data signal at the MCH. Capture a read signal at the MCH. Compare measured signal to simulated signal. You may need to bring into a spreadsheet and time shift data to get to over lap. Adjust elements in the simulation model achieve reasonable agreement with measurements. You need to define what reasonable is and justify that. § PPT minimum contents Hypothesis: Simulation model accurately predict real circuits. Results Summary: Overlay of simulation and measurement waveforms. Overview of component changes that had the most effect on achieving the best results. Supporting data: Illustrate where probes are placed Describe complete scope setup including trigger settings Show you circuit diagram and slides to illustrate changes. For example you may need W elements for transmission lines or you may not. Inductor and cap values may need to be altered. Introduction 12/4/2002

Starter Schematic For DDR Simulation Introduction 12/4/2002 21