Evolution Network NETWORK EVOLUTION AT COLT Amit SEAMLESS

Evolution Network NETWORK EVOLUTION AT COLT Amit SEAMLESS Dhamija APRICOT 2018 Kathmandu, Nepal Amit Dhamija Network Engineering amit. dhamija@colt. net | @Amit. Dhamijain

Agenda • Colt’s Legacy Network • Colt’s VISION for One global converged packet network • Network design evolution towards Next. Gen technologies • Experiences and Key Takeaways

COLT’S LEGACY NETWORK IGP - ISIS MPLS-LDP BGP – AS 8220 IP Network Services: • Internet access • MPLS L 3 VPN’s (unicast and multicast) IGP- OSPF MPLS – RSVP/TE T-LDP – PW Ethernet Network Services: • E P 2 P • Etree • ELAN

Colt’s VISION: One global converged packet network The integration of the network from the service, technology, system and process point of view to offer innovative services, reduce provisioning time, automate end-to-end orchestration and offer truly combined network services. Converged Network Service flexibility Simplified Design & Architecture Resiliency Scalability virtualisation

Network design evolution towards Next. Gen technologies

New Network At a Glance E 2 E Converged Network (Ethernet / IP / Multicast Services) Infrastructure Protocols • • ISIS – IGP MPLS-SR RSVP BFD etc. Services • EVPN – P 2 P/E-LAN • IP Services – L 3 VPN/Internet access • M-VPN – NG multicast VPN’s

Next. Gen MPLS Evolution

Legacy Network – MPLS Design MPLS-LDP IP Network PE-X PE-Y MPLS-LDP is used as the control plane protocol for IP Network. PE-X MPLS-RSVP Ethernet Network PE-Y MPLS-RSVP is used as control plane protocol for Ethernet Network. MPLS applications – L 3 VPN’s/6 PE/6 VPE. • MPLS application – VPWS/VPLS • • Traffic engineering with explicit paths. Shortest IGP Path cost. • • Disjoint paths. Fast convergence. • • Fast convergence. ECMP Paths. •

Challenges with existing MPLS Design MPLS LDP MPLS RSVP • LDP-FRR coverage issue. • • LDP generates lots of needless labels and is inefficient. RSVP traffic engineering LSP’s scalability issues. • RSVP traffic engineering ECMP issues.

New Network – MPLS Design (Segment Routing) https: //datatracker. ietf. org/doc/draft-ietf-spring-segment-routing/ Why We Chose Segment Routing • Simplicity – Single MPLS design as standard. • MPLS SR supports both modes under one network – (SR-SPT and SR-TE). • Solves the ECMP for traffic engineering by using (Node SID & Adj-SID) algorithm. • Simplifies the control plane for COLT’s new network – solves the RSVP scaling issues/no dedicated MPLS protocols. • Convergence – 100% protection with any topology. • Interoperability with existing MPLS Protocols. • Supports all the existing services. .

Segment Routing Control Plane PE-N PE-W COLT MPLS Segment Routed Network PE-E PE-S • Deployed using IPv 4 transport with MPLS data plane. • Label Allocation – Only loopbacks with Index ID. • SRGB – 100 K. • Range – 65 K label space per node. router isis COLT net 49. 0001. 1120. 6409. 4142. 00 interface Loopback 0 passive circuit-type level-2 -only address-family ipv 4 unicast prefix-sid index 4142 explicit-null

Segment Routing – TI LFA Coverage Prefixes reachable in L 2 All paths protected Some paths protected Unprotected Protection coverage Critical Priority 0 0. 00% High Priority 0 0. 00% Medium Priority 735 0 0 100. 00% Low Priority 1295 0 0 100. 00% Total 2030 0 0 100. 00%

Segment Routing Integration with IP/LDP Network Mapping Servers Advertise. SR labels for legacy LDP PE Nodes PE-N LDP ISIS / MPLS-LDP Mapping Servers PE IP Network PE-W ISIS / MPLS-SR PE-E Mapping Servers PE-S New-Network • Mapping Severs – In path for our deployment. • Features working with this deployment: L 3 VPN’s/Internet access/Unicast-v 4/v 6/QOS etc. • Our use case for Integration: LDP over SR / LDP-SR & SR-LDP.

Segment Routing Traffic Engineering – SR-TE https: //tools. ietf. org/html/draft-fils-spring-segment-routing-policy-04 PCEP LSPDB Synch PCEP for tunnel req & label imposition PE-1 NSO Centralized statefull PCE SR-TED BGP-LS for topology info MPLS SR TED Synch PE-2 LSP Setup/ Tear down(On-Demand) • Ongoing evaluation in our engineering lab. • SR-TE features - Disjoint paths with Node/Link and SRLG with static and dynamic paths.

Next. Gen Services Evolution

Legacy Network – Services Design PE-Z IRR VRR T-LDP BGP/MP-BGP IP Network PE-Y PE-X BGP is used as the control plane protocol for IP-Services. • • • Internet access – IPv 4/IPv 6(6 PE) Layer 3 VPN’s – VPNv 4 and VPNv 6(6 VPE) Deployed using RR’s. PE-X PE-Y L 2 -PE G. 8032/ERP L 2 -PE ERP in the access rings and T-LDP sessions in the core ring for Ethernet Services. • E-P 2 P – MPLS-VPWS (Psuedowires) • ELAN – MPLS-VPLS (Mesh of Pseudowires) • E-TREE (Psuedowires based)

Challenges with existing Service Design • Different implementation methods – Layer 3 & Layer 2 VPN’s. • Complicated provisioning and troubleshooting. • Layer 2 VPN’s control and data plane are mixed/flooding issues. • Full mesh of pesudowires required in core network/scalability issues. • Multihomed customers no support of active-active redundancy.

New Network – Services Design (BGP) Why We Chose EVPN Technology • • • IRR Simplicity - Single BGP protocol as the standard for all the services in network. IP and MAC distribution by using “BGP”. No use of pseudowires for L 2 VPN’s. Faster convergence for all BGP based services. Ease of configuration, operations and manageability. PR’s BGP Control Plane ERR PE-2 BGP- Control Plane AR-1 AR-2 PE-3 ERR BGP- Control Plane AR-1 AR-2

Ethernet Services (EVPN) P 2 P Services – MP-BGP https: //tools. ietf. org/html/rfc 8214 Single BGP Ethernet AD Route containing [RD, RT, ESI, Label(X), Eth-Tag ID(AC)] AR-2 EVPN NLRI AFI=25/SAFI=70 COLT EVPN VPWS AR-1 Show bgp l 2 vpn evpn Route Distinguisher: 1111 (default for vrf VPWS: 1111) *>i[1][0000. 0000][111]/120 10. 91. 126. 111 100 *> [1][0000. 0000][222]/120 0. 0 0 i • Next Gen solution for Ethernet VPN’s. • EVPN VPWS – Data plane uses MPLS-SR & Control plane – BGP based. • Simple to deploy and scalable solution. 0 I

Ethernet Services (EVPN) E-LAN Services – MP-BGP https: //tools. ietf. org/html/rfc 7432 Four Routes - ELAN 1. Ethernet AD 2. Ethernet MAC 3. ESI Route 4. Inclusive multicast route EVPN NLRI AFI=25/SAFI=70 AR-3 COLT EVPN MPLS AR-1 AR-2 Show bgp l 2 vpn evpn Route Distinguisher: 10. 91. 126. 115: 1 *>i[1][10. 91. 126. 115: 1][0000. 0100. acce. 5500. 1401][4294967295]/18 4 10. 91. 126. 115 100 0 i *>i[1][000 a. 5 b 7 e. 7300. 0000][0]/120 10. 91. 126. 115 100 0 i *>i[3][0][32][10. 91. 126. 115]/80 10. 91. 126. 115 100 0 I *>i[2][0][48][0000. 0200. 0001][0]/104 10. 91. 126. 115 100 0 i • EVPN-MPLS is similar to L 3 VPN MPLS. • Known Unicast – MPLS-SR as transport & BUM – Ingress Replication with MPLS-SR. • Rapid convergence – non zero ESI for single homed customers.

IP Services using PW-HE Feature – MP-BGP PW-HE binds the L 2 EVPN VPWS to the GRT or L 3 -VPN’s MP-BGP L 3 -VPN/GRT PR’s PW-HE VRF EV EVPN VPWS MP-BGP AR-1 • • PW-HE using MPLS-SR as transport – Unicast PW-HE using MPLS-RSVP as transport - Multicast WS EVPN VPWS MP-BGP VP PN VP W PE-3 PN EV S PE-2 AR-2 xconnect group xxxxx p 2 p xxxxx interface PW-Ether 1 neighbor evpn evi 787 target 200 source 100 Logical Interface (PW-HE)

NFV – Network function Virtualization

Colt’s Focus towards Network Function Virtualization • Virtualization various network infrastructure segments • RR / PCE Controller/ DDOS controllers etc. • We deploy on commodity x 86 hardware & dedicated VM’s for each network function. • We run each instance as an independent island. • Using KVM Hypervisor for our deployment. VNF KVM-Hypervisor x 86 HW VNF

Example - Route Reflector Virtualization (v. RR) • • Next Gen “BGP-RR” deployed for all the services by virtualization of control plane function. Virtualized all the RR’s on KVM Hypervisor. Server 1 Server 2 Server 3 Internet RR IRR-1 IRR-2 IRR-3 MPLS Layer 3 VPN’s RR VPNRR-1 VPNRR-2 VPNRR-3 Ethernet RR E-RR -1 E-RR -2 E-RR -3 Dedicated VM’s for each RR per Server. Centralized architecture design, each PE forms sessions with all three RR’s.

Experiences and Key takeaways

Operations: Experiences from our Deployment • All MPLS Services are fully supported by Segment Routing. • L 2 VPN’s / VPLS / EVPN / L 3 VPN’s / IPv 6. • LDP-SR Inter working full multi-vendor support (Cisco/Juniper & Nokia, in our case). • ISIS doesn’t advertise the merge flags when destination prefix looses the last backup path – Patch! • Inconsistency in forwarding entries in LC and RP due to TI-LFA bug – Patch! • No labels assignment for connected prefixes other then loopbacks on MPLS-SR ( SR feature Limitation). • Native multicast solution for SR not yet available (Tree-SID, BIER etc. ) • Max SID Depth is a problem with SR-TE due to nature of source routing – Binding SID! • No Interop issue in EVPN between Cisco/Juniper & Nokia. • EVPN Flow based load balancing no support at this stage. • Hierarchical EVPN for large networks is still a challenge.

Key Takeaways • Single Infrastructure for all services. . ü Simplified the transport and service architecture. ü Reduce the cost – opex and capex. ü Faster Service delivery. ü Better Orchestrator. • It’s good to be an early investor in any new technology. • Benefits Realized with Next Gen MPLS technologies – simplified design, deployment & operations. • Next Gen MPLS technologies proved easy to deploy, maintain and use. • Questions?

For your time Thank you amit. dhamija@colt. net