NTP Architecture Protocol and Algorithms David L Mills

NTP Architecture, Protocol and Algorithms David L. Mills University of Delaware http: //www. eecis. udel. edu/~mills mailto: mills@udel. edu Sir John Tenniel; Alice’s Adventures in Wonderland, Lewis Carroll 10 -Nov-20 1

Process decomposition Server 1 Peer/Poll 1 Server 2 Peer/Poll 2 Server 3 Peer/Poll 3 Remote Servers Peer/Poll Processes System Process Selection and Clustering Algorithms Combining Algorithm Clock Discipline Process Loop Filter VFO Clock Adjust Process o Peer process runs when a packet is received. o Poll process sends packets at intervals determined by the clock discipline process and remote server. o System process runs when a new peer process update is received. o Clock discipline process runs at intervals determined by the measured network phase jitter and clock oscillator (VFO) frequency wander. o Clock adjust process runs at intervals of one second. 10 -Nov-20 2

NTP protocol header and timestamp formats NTP Protocol Header Format (32 bits) LI VN Mode Strat Poll Root Delay Prec Root Dispersion Reference Identifier Reference Timestamp (64) Cryptosum LI VN Strat Poll Prec NTP Timestamp Format (64 bits) Originate Timestamp (64) Seconds (32) Receive Timestamp (64) Value is in seconds and fraction since 0 h 1 January 1900 Transmit Timestamp (64) NTPv 4 Extension Field Length Extension Field 1 (optional) Extension Field 2… (optional) Key/Algorithm Identifier Authenticator (Optional) leap warning indicator version number (4) stratum (0 -15) poll interval (log 2) precision (log 2) Message Hash (64 or 128) Fraction (32) Field Type Extension Field (padded to 32 -bit boundary) Last field padded to 64 -bit boundary NTP v 3 and v 4 NTP v 4 only authentication only Authenticator uses DES-CBC or MD 5 cryptosum of NTP header plus extension fields (NTPv 4) 10 -Nov-20 3

NTP packet header format Packet header Variables Description leap version mode stratum t r D E refid reftime T 1 T 2 T 3 T 4 MAC leap indicator (LI) version number (VN) protocol mode stratum poll interval (log 2 s) clock reading precision (log 2 s) root delay root dispersion reference ID reference timestamp originate timestamp receive timestamp transmit timestamp destination timestamp* MD 5 message hash (optional) LI VN Mode Strat Poll Root Delay Root Dispersion Reference Identifier Prec Reference Timestamp (64) Originate Timestamp (64) Receive Timestamp (64) Transmit Timestamp (64) MAC (optional 160) * Strictly speaking, T 4 is not a packet variable; it is the value of the system clock upon arrival. 10 -Nov-20 4

NTP date and timestamp formats and important dates NTP Date (signed, twos-complement, 128 -bit integer) Seconds (64) Era Number Era (32) 10 -Nov-20 Fraction (32 or 64) NTP Timestamp (unsigned 64 -bit integer) Seconds (32) Fraction (32) 5

Process decomposition Server 1 Peer/Poll 1 Server 2 Peer/Poll 2 Server 3 Peer/Poll 3 Remote Servers Peer/Poll Processes System Process Selection and Clustering Algorithms Mitigation Algorithms Clock Discipline Process Loop Filter VFO Clock Adjust Process o Peer process runs when a packet is received. o Poll process sends packets at intervals determined by the clock discipline process and remote server. o System process runs when a new peer process update is received. o Clock discipline process runs at intervals determined by the measured network phase jitter and clock oscillator (VFO) frequency wander. o Clock adjust process runs at intervals of one second. 10 -Nov-20 6

NTP on-wire protocol org rec xmt t 2 0 0 t 3 t 1 = org t 2 = rec t 6 t 3 t 4 t 7 t 5 = org t 6 = rec t 1 t 2 = clock t 3 = xmit t 5 t 6 = clock t 7 = xmit t 1<>0? T 2 = t 2 0 T 1 = t 1 T 2 T 3 = clock t 5<>T 1? T 6 = t 6 t 3==T 3? T 5 = t 5 T 6 T 7 = clock t 2 t 1 0 0 t 1 = xmit org rec xmt 0 0 T 1 = clock 10 -Nov-20 t 3 t 6 t 4 t 1 t 2 t 3 t 4 = clock t 3 <> 0? T 4 = t 4 t 1 == T 1? t 5 t 3 = org t 4 = rec t 5 = xmit T 3 = t 3 T 4 T 5 = clock Packet Variables Peer B State Variables Name Description org rec xmt originate timestamp receive timestamp transmit timestamp Packet Header Variables Name Description t 7 tn tn+1 tn+2 tn+3 t 8 t 5 t 6 t 7 t 8 = clock Packet Variables t 7 <> T 3? T 8 = t 8 t 5 == T 5? State Variables originate timestamp receive timestamp transmit timestamp destination timestamp t 7 <> T 3? org Duplicate Test t 5 == T 5? xmt Bogus Test Peer A 7

Transition matrix Association Mode Packet Mode ACTIVE NO_PEER NEWPS ACTIVE PROC PASSIVE PROC ERROR CLIENT PASSIVE CLIENT SERVER BCAST FXMIT NEWMC NEWBC PROC SERVER BCAST BCLIENT ERROR PROC The default (empty box) behavior is to discard the packet without comment. 10 -Nov-20 8

Packet sanity tests 10 -Nov-20 9

Clock filter algorithm T 2 Server T 3 x q 0 T 1 Client T 4 o The most accurate offset q 0 is measured at the lowest delay d 0 (apex of the wedge scattergram). o The correct time q must lie within the wedge q 0 ± (d - d 0)/2. o The d 0 is estimated as the minimum of the last eight delay measurements and (q 0 , d 0) becomes the peer update. o Each peer update can be used only once and must be more recent than the previous update. 10 -Nov-20 10

Clock filter performance o Left figure shows raw time offsets measured for a typical path over a 24 -hour period (mean error 724 ms, median error 192 ms) o Right graph shows filtered time offsets over the same period (mean error 192 ms, median error 112 ms). o The mean error has been reduced by 11. 5 d. B; the median error by 18. 3 d. B. This is impressive performance. 10 -Nov-20 11

Clock select principles B A D C Correct DTSS correctness interval = q - l £ q 0 £ q + l m = number of clocks f = number of presumed falsetickers A, B, C are truechimers D is falseticker o The correctness interval for any candidate is the set of points in the interval of length twice the synchronization distance centered at the computed offset. o The DTSS interval contains points from the largest number of correctness intervals, i. e. , the intersection of correctness intervals. o A truechimer has a correctness interval that includes points in the intersection interval. . o Formal correctness assertions require at least half the candidates be be truechimers. If not, no candidate can be considered a truechimer. 10 -Nov-20 12

system process: select algorithm For each of m associations construct a correctness interval [q – l, q + l] Consider the lowpoint, midpoint and highpoint of these intervals. Sort these values in a list from lowest to highest. Set the number of falsetickers f = 0. Set n = 0. Scan from lowest endpoint to highest. Add one to n for every lowpoint, subtract one for every highpoint. If n ≥ m - f, stop; set l = current lowpoint Set n = 0. Scan from highest endpoint to lowest. Add one to n for every highpoint, subtract one for every lowpoint. If n ≥ m - f, stop; set u = current highpoint. If l < u? yes 10 -Nov-20 no Add one to f. Is f < m / 2? no Failure; a majority clique could not be found. . yes Success; the intersection interval is [l, u]. 13

Cluster principles peer jitter select jitter j. R(3) j. R(2) j. R(4) j. S(1) j. S(3) j. R(4) j. R(2) j. R(1) a b o Candidate 1 is further from the others, so its select jitter j. S 1 is highest. o (a) jmax = j. S 1 and jmin = j. R 2. Since jmax > jmin, the algorithm prunes candidate 1 to reduce select jitter and continues. o (b) jmax = j. S 3 and jmin = j. R 2. Since jmax < jmin, pruning additional candidates will not reduce select jitter. So, the algorithm ends with j. R 2, j. R 3 and j. R 4 as survivors. 10 -Nov-20 14

system process: cluster algorithm Let (q, j. R, L) represent a candidate with peer offset q, jitter j and a weight factor L equal to stratum as the high order field and root distance as the low order field. Sort the candidates by increasing L. Let n be the number of candidates and nmin ≤ n the minimum number of survivors. For each candidate compute the selection jitter j. S (RMS peer offset differences between this and all other candidates). Select jmax as the candidate with maximum Lj. S. Select jmin as the candidate with minimum j. jmax < jmin or n ≤ nmin or jmax is prefer peer? yes no Delete the outlyer candidate with jmax; reduce n by one. Done. The remaining cluster survivors are the pick of the litter. 10 -Nov-20 15

NTP dataflow analysis Server 1 D, E Peer 1 q, d, e, j Server 2 D, E Peer 2 q, d, e , j Server 3 D, E Peer 3 q, d, e , j Selection and Combining Algorithms System Q, D, E, J o Each server provides delay D and dispersion E relative to the root of the synchronization subtree. o As each NTP message arrives, the peer process updates peer offset q, delay d, dispersion e and jitter j. o At system poll intervals, the clock selection and combining algorithms updates system offset Q, delay D, dispersion E and jitter J. o Dispersions e and E increase with time at a rate depending on specified frequency tolerance f. 10 -Nov-20 16

Error budget - notation o Constants (peers A and B) r maximum reading error f w Packet variables DB peer root delay EB peer root dispersion o Sample variables T 1, T 2, T 3, T 4 protocol timestamps x clock offset y roundtrip delay z dispersion t interval since last update System variables Q clock offset D E js j t m maximum frequency error dispersion normalize: 0. 5 o 10 -Nov-20 o o root delay root dispersion selection jitter interval since last update number of peers Peer variables q clock offset d roundtrip delay e dispersion jr filter jitter n number of filter stages t interval since last update 17

Definitions o Precision: elapsed time to read the system clock from userland. o Resolution: significant bits of the timestamp fraction. o Maximum error: maximum error due all causes (see error budget). o Offset: estimated time offset relative to the server time. o Jitter: exponential average of first-order time differences o Frequency: estimated frequency offset relative to UTC. o Wander: exponential average of first-order frequency differences. o Dispersion: maximum error due oscillator frequency tolerance. o Root delay: accumulated roundtrip delay via primary server. o Root dispersion: accumulated total dispersion from primary server. o Estimated error: RMS accumulation from all causes (see error budget). 10 -Nov-20 18

Time values and computations Packet Variables Peer Variables System Variables S S Client S Server o Packet variables are computed directly from the packet header. o Peer variables are groomed by the clock filter. o System variables are groomed from the available peers. 10 -Nov-20 19

Clock discipline algorithm NTP qr+ qc - Phase Detector Clock Filter Vs Loop Filter VFO Vc Vd Clock Adjust x y Phase/Freq Prediction o Vd is a function of the phase difference between NTP and the VFO. o Vs depends on the stage chosen on the clock filter shift register. o x and y are the phase update and frequency update, respectively, computed by the prediction functions. o Clock adjust process runs once per second to compute Vc, which controls the frequency of the local clock oscillator. o VFO phase is compared to NTP phase to close the feedback loop. 10 -Nov-20 20

NTP clock discipline with PPS steering NTP qr+ qo- Phase Detector VFO Vd Vc y Clock Filter Vs Loop Filter Frequency Estimator PPS o NTP daemon disciplines variable frequency oscillator (VFO) phase Vc relative to accurate and reliable network sources. o Kernel disciplines VFO frequency y to pulse-per-second (PPS) signal. o Clock accuracy continues to be disciplined even if NTP daemon or sources fail. o In general, the accuracy is only slightly degraded relative to a local reference source. 10 -Nov-20 21

Measured PPS time error for Alpha 433 Standard error 51. 3 ns 10 -Nov-20 22

Further information o o NTP home page http: //www. ntp. org • Current NTP Version 3 and 4 software and documentation • FAQ and links to other sources and interesting places David L. Mills home page http: //www. eecis. udel. edu/~mills • Papers, reports and memoranda in Post. Script and PDF formats • Briefings in HTML, Post. Script, Power. Point and PDF formats • Collaboration resources hardware, software and documentation • Songs, photo galleries and after-dinner speech scripts Udel FTP server: ftp: //ftp. udel. edu/pub/ntp • Current NTP Version software, documentation and support • Collaboration resources and junkbox Related projects http: //www. eecis. udel. edu/~mills/status. htm • Current research project descriptions and briefings 10 -Nov-20 23