Backbone Performance Comparison Jeff Boote Internet 2 Warren

Backbone Performance Comparison Jeff Boote, Internet 2 Warren Matthews, Georgia Tech John Moore, MCNC 1

Overview • We (in NC) were asked to compare the relative performance of various IP service providers w Interest from both local CIOs and Internet 2 • We decided to measure relative end-to-end latency and jitter • Recruited a few other ITECs (Ohio and Texas) and GA Tech to help • Jeff Boote got interested since we were using owamp 2

Method • Setup owamp machine at each site with multiple interfaces per NIC • Use host routes to force traffic to a specific destination via a specific provider • Create a mesh of these running continuously and dump results to a database • Add traceroute information to verify paths and look for routing changes 3

Path types • Path will vary depending on whether source and destination sites share provider, or not. • Doesn’t take “natural” or policy routing into consideration, but useful for comparative purposes. 4

As we progressed… • New paths became available… w VPLS (Layer 2 VLAN) between three of the ITECs (NC, OH and TX) § Described in sidebar w NLR Packet. Net between NC and GT § Not all that interesting, since both sites attach to the same NLR router in Atlanta • Added NLR to new interface on same NIC, added VPLS to a separate NIC on the same machines • TAMU site setup and running, but o good data available yet w Had to remove host routes due to other routing changes going on locally 5

Available Data from OWAMP • • • Latency variation (jitter ~ 95%-min) TTL (num hops) Duplicates Loss Reordering (not likely at 1 pps) 6

OWAMP “sender” configuration Only last router before “backbone” shown • Each host has multiple virtual addresses configured (one per “network”) • Continuous stream of packets (1 pps - exp dist. ) per network address “pair” • Traffic is directed onto specific network based on dest address 7

LATAB (Traceroute when source is routed through Abilene) OH nms 4 -ipls IPLS CHIN NYCM KSCY NC WASH nms 4 -hstn HSTN ATLA TAMU GT nms 4 -wash 8

LATQW (Traceroute when source is routed through Qwest) CHI-EDGE CHI-CORE DCA-EDGE NC ATLA-CORE ATLA-EDGE OH GT 9

LATL 3 (Traceroute when source is routed through Level 3) Qwest Washington Asymmetric routing: Northbound via Charlotte Southbound via Raleigh Unknown NC Charlotte Atlanta OH Unknown GT ATLAL 3 10

LATO 3 (Traceroute when source is routed through another provider - GT/Cogent) OH Qwest NC CORE ATLA GT 11

LATNLR ATLA NC GT 12

LATVPLS OH NC TAMU 13

Preliminary Results • Small amount of data collected so far • Working on how best to visualize combination of pieces (latency, loss, routing changes, etc. ) w Looking for “stability” metric (but stability is application dependent) • More analysis needed 14

Loss overview 15

NC to GT • NLR is lower latency. w This is expected as GT and NC are connected to the same router. w NC connection is backhauled via NLR L 2 service. Qwest NLR • Qwest and Abilene go via Washington. w The long way… Level 3 Abilene • For the Level 3 path, there is an unidentified hop just before the GT campus. w Rate limiter? w Expected NLR and Level 3 paths to be closer 16

GT to NC NLR Qwest Cogent Level 3 Abilene • NLR and Level 3 paths similar • Cogent hands off to Qwest to get to NC 17

Latency Range NC to GT Level 3 via Raleigh Level 3 via Charlotte Input to GT is always longer? 18

NC to OH Qwest Level 3 • Marginally quicker across Qwest (via Washington and Chicago). • Abilene via New York, Chicago and Indianapolis. Abilene 19

OH to NC • Qwest Abilene OH doesn’t use Level 3, so no return path to NC via Level 3 20

Latency Range NC to OH No return path for L 3_NC_OH 21

GT to OH Abilene Cogent • Abilene more direct via Indianapolis • Qwest via Chicago • Cogent, Level 3 hand off to Qwest Level 3 22

OH to GT • Qwest OH doesn’t use Level 3, so no return path to GT via Level 3 Abilene 23

Latency Range GT to OH No return path for L 3_GT_OH. 24

Summary • From a latency perspective, topology is the overriding parameter w So far we’re not seeing huge latency deltas between R&E and commodity between two endpoints • Loss in commodity networks is pretty good w They’ve improved in the last 10 years • Looking for a quality metric (stability? ) to combine things we can measure 25

VPLS Sidebar • Virtual Private LAN Service - multipoint Ethernet service over IP/MPLS backbone • Created between ITECs as overlay on Abilene • PE routers sit in Giga. Po. P address space, interconnected via interdomain LSPs • Abilene T 640 s are P routers 26

VPLS Overview • • • Full Mesh of LSPs BGP for inter-PE communication Ethernet encapsulation at PE-CE 27

View from Ohio To NC To TX No routers! 28

View from NC PE Local NC MAC address TX MAC address OH MAC address 29