Network Operator Perspective MPLS 12 Years After Tom

Network Operator Perspective MPLS: 12 Years After Tom Bechly IETF 74, San Francisco IAB Plenary March 18, 2009

MPLS: A Successful Protocol • MPLS has been and is a successful protocol – From perspective of RFC 5218 (What Makes for a Successful Protocol? ), MPLS was used for its intended purpose and at intended scale • Goal was to switch packets to support rapidly expanding global networks – MPLS is “wildly successful” (RFC 5218) in that its use has exceeded its original design goal thru development of numerous extensions – From service provider perspective MPLS was successful in supporting growth, reducing cost, and providing basis for new services • Original goal of bringing Layer 2 switching speed to Layer 3 was accomplished, but somewhat discounted over time due to hardware evolution – L 2 was hardware switched and L 3 was process switched • MPLS was easily leveraged for traffic engineering, VPNs, and layer 2 transport. • For the service provider, MPLS has become one the most reached for and extended tools in the tool chest (150+ RFCs) 2

MPLS CE Router MPLS CORE PE Router P Router CE Router Customer Edge (CE) Router CE Router • Enables network edge routers to apply simple MPLS labels to packets or frames • Forwards packets by swapping labels with minimal lookup • Integrates Layer 2 switching and Layer 3 routing Provider Edge (PE) Router/Switch Provider Core Router/Switch 3

MPLS/RSVP-TE Benefits • MPLS with RSVP-TE provides overall path control in network – Use with constraint based routing – Control over latency and delay variation – Bridges gap between ability to deploy capacity versus current demand in existing network • Use of MPLS allowed gathering measurement statistics on LSPs – Probably more important than actual path control – Provides ability to accurately measure traffic between router pairs • Traffic volumes, latency, and delay variation – Measure traffic between hubs, metros, and regions – Measure asymmetry of flows, over time – A time series depiction can be built to trend traffic for efficient investment and to provide required service • MPLS became an enabler for the development of additional services – L 2 VPNs and L 3 VPNs 4

Verizon Public IP • AS 701 was initially implemented as an overlay over a dedicated frame relay network – Path control was effected thru manipulating path of frame relay PVCs • As capacity requirements increased, the network was migrated to an overlay over ATM – The cost of this became untenable, as capacity requirements continued to increase • MPLS with RSVP-TE deployed in EMEA (AS 702) in 1999 – First deployment of RSVP-TE in production network – Deployed in US (AS 701) in 2000 • Deployed for traffic engineering to provide control over path selection that was not available thru L 3 protocols – Shortest path algorithm did not always provide optimal route • MPLS technology has enabled the Verizon Public IP network to grow to be one of the largest in the world 5

Verizon IP Network – 410 unique switch/router hubs (Po. Ps) – Six continents, 150+ countries 6

Verizon Layer 3 VPN Services: VBNS+ and Private IP • v. BNS (very-high-performance Backbone Network Service) was established in 1995 – Cooperative research and development agreement between Verizon (formerly MCI) and National Science Foundation (follow on to NSFnet) – Evolved to a commercial product: v. BNS+ for gov and edu market • MPLS routing/switching implemented in network in 1999 – Initially MPLS was implemented for traffic engineering • L 3 VPN (RFC 2547) was implemented in 2001 – There approximately 40 nodes in 19 US cities, full mesh of TE LSPs • Verizon PIP (Private IP) was established in 1999 – Layer 3 VPN (RFC 4364), wide area network for business customers – Quality of Service, strong SLAs, etc. • Large global network – There approximately 625 nodes across 162 cities in 59 countries • Uses LDP for label distribution, with partial mesh of LSPs 7

Private IP Global Reach MP 10163 v 5. 03 8

Verizon Layer 2 Services: MAE® Services and Converged Packet Architecture (CPA) • MAE® Services established 1992 as metro Internet Exchange point • Evolved into MPLS based national service for extended peering and L 2 VPNs (VPWS), implemented in 2002 – Service interworking (ATM, Frame Relay, and Ethernet), based on draft Martini pseudowires and draft Shah ARP Mediation • Implemented across public internet within full mesh of GRE tunnels – ISIS, RSVP-TE signaled LSPs, and LDP signaled pseudowires • CPA supports Ethernet access and Ethernet services – L 2 VPNs: both EVPL (PWE 3) and VPLS (RFC 4762) – Quality of Service, strong SLAs, etc. • Large global network – There approximately 115 nodes across 27 countries • RSVP-TE used to signal LSPs – Full mesh for EVPL and VPLS – Currently 10, 000+ LSPs 9

Lessons Learned • Implementation defects significantly impact early perception of technology – For AS 701, there was internal resistance to moving from ATM underlay network to MPLS – When defects in the MPLS implementation on vendor equipment were encountered these initially viewed by some as defects in the technology 10

Lessons not Learned (VPLS) • RFC 4762: Virtual Private LAN Service (VPLS) Using Label Distribution Protocol (LDP) Signaling – Hierarchy is managed thru HVPLS, specified within RFC • RFC 4761: Virtual Private LAN Service (VPLS) Using BGP for Auto. Discovery and Signaling – Hierarchy is managed thru route reflectors and multi-segment pseudowires • Both approaches are currently in production in different service provider networks • Some vendors have implemented both standards • This increases to overall cost and complexity of technology and network development across the community – Resolution and mitigation of differences is far more economic during protocol development than once into implementation – Gateway function has high development and operational cost – The added costs and complexity are continuously accretive 11

Lessons not Learned (RFC 5085 – PW VCCV) • Pseudowire Virtual Circuit Connectivity Verification (VCCV) – RFC 5085 – Three modes of operation: (Type 1: PWE 3 Control Word Bit, Type 2: MPLS Router Alert Label, Type 3: MPLS PW Label with TTL == 1 – Mode is negotiated, so all three are optional • Vendors, to this point, have not implemented all modes nor the same modes • This leads to interoperability issues in mixed vendor networks – Delays significantly availability of feature – Adds to development and integration costs VCCV Mode Vendors Y Vendors X Control Word* Yes No Router Alert Label Yes No TTL Expiry* No Yes 12

Continuing Challenges • Latency sensitive customers – These are typically financial customers that are sensitive to a 2 ms increase or change in latency • Require traffic to be on path with deterministic low latency – Due to network event traffic may be rerouted, via Fast Reroute and the re -signaled LSP – Paths are recalculated periodically to ensure low latency path – Once optimal path is available, traffic is re-routed (make before break) to this path – As this path could be significantly shorter (2 – 10 ms), there will be out of order packets that may impact some hosts • Nodes in network within the core, may carry a high number of LSPs – Latency sensitive customers are requesting notification on any maintenance that will impact LSPs carrying their traffic 13

MPLS Going Forward • MPLS has been an extremely successful protocol – It has been widely deployed and extended • MPLS based networks and facilities to continue to grow and expand – This growth is continuing and will continue for some time 14