OTV Technology Introduction Natale Ruello Technical Marketing Engineer

OTV Technology Introduction Natale Ruello Technical Marketing Engineer – Nexus 7000

Addressing Business Goals with LAN Extensions Business Goals 99. 999% Global Availability Service Velocity and On-demand Capacity LAN Extensions Enable Distributed Clusters to improve Application Availability without compromising Network Resiliency Attributes Availability Unleash Compute Virtualization beyond a single physical data center for fast service and capacity additions Adaptability Maximize Asset Utilization Supports migration of workloads and consolidation of servers across locations Streamline Operations & Reduce OPEX Enables improved change management methods across multiple physical locations Non-disruptive model for minimal operational overhead BRKRST-2033 to avoid power/cooling hot spots or compute/network idleness © 2009 Cisco Systems, Inc. All rights reserved. Cisco Public Cost Optimization 3

Enabling IT Solutions with LAN extensions Vmotion Data Center Cluster Data Center A B MSCS Cluster LAN Extension Solaris Sun Cluster Enterprise RAC (Real Appl. Cluster) HACMP Legato Automated Availability Mgr Metro Cluster Metrocluster BACnet (building automation/control) BRKRST-2033 © 2009 Cisco Systems, Inc. All rights reserved. Cisco Public 4

Overlay Transport Virtualization (OTV) O T V Overlay - A solution that is independent of the infrastructure technology and services, flexible over various inter-connect facilities Transport - Transporting services for layer 2 and layer 3 Ethernet and IP traffic Virtualization - Provides virtual connections, connections that are in turn virtualized and partitioned into VPNs, VRFs, VLANs OTV LAN Extensions BRKRST-2033 OTV delivers a virtual L 2 transport © 2009 Cisco Systems, Inc. All rights reserved. Cisco Public 5

Challenges with LAN Extensions Real Problems Solved by OTV § Extensions over any transport (IP, MPLS) Fault Domain North Data Center Fault Domain § Failure Boundary Preservation § Site independence / Isolation § Optimal BW utilization (no head-end replication) § Resiliency/multi-homing LAN Extension § Built-in end-to-end Loop Prevention § Multi-site connectivity (inter and intra DC) § Scalability § VLANs, Sites, MACs Only 5 CLI commands § ARP, Broadcasts/Floods § Operations Simplicity Fault Domain § Topology Flexibility BRKRST-2033 © 2009 Cisco Systems, Inc. All rights reserved. Cisco Public South Data Center Fault Domain 6

Traditional Layer 2 VPNs Eo. MPLS Dark Fiber VPLS Presentation_ID © 2009 Cisco Systems, Inc. All rights reserved. Cisco Confidential 7

Flooding Behavior § Traditional Layer 2 VPN technologies rely on flooding to propagate MAC reachability. § The flooding behavior causes failures to propagate to every site in the L 2 -VPN. x 2 Site A Site C MAC 1 propagation Site B § A solution that provides layer 2 connectivity, yet restricts the reach of the flood domain is necessary in order to contain failures and preserve the resiliency. Presentation_ID © 2009 Cisco Systems, Inc. All rights reserved. Cisco Confidential 8

Pseudo-wires Maintenance § Before any learning can happen a full mesh of pseudo-wires/tunnels must be in place. § For N sites, there will be N*(N-1)/2 pseudo-wires. Complex to add/remove sites. § Head-end replication for multicast and broadcast. Sub-optimal BW utilization. § A simple overlay protocol with built-in functionality and point-to-cloud provisioning is key to reducing the cost of providing this connectivity Presentation_ID © 2009 Cisco Systems, Inc. All rights reserved. Cisco Confidential 9

Multi-Homing § Require additional protocols to support Multi-homing. § STP is often extended across the sites of the Layer 2 VPN. Very difficult to manage as the number of sites grows. § Malfunctions on one site will likely impact all sites on the VPN. Active L 2 VPN L 2 Site § A solution that natively provides automatic detection of multi-homing without the need of extending the STP domains is key. Presentation_ID © 2009 Cisco Systems, Inc. All rights reserved. Cisco Confidential 10

What can be improved § Data Plane Learning Control Plane Learning Moving to a Control Plane protocol that proactively advertises MAC addresses and their reachability instead of the current flooding mechanism. § Pseudo-wires and Tunnels Dynamic Encapsulation No static tunnel or pseudo-wire configuration required. Optimal replication of traffic done closer to the destination, which translates into much more efficient bandwidth utilization in the core. § Multi-Homing Native Built-in Multi-homing Ideally a multi-homed solution should allow load balancing of flows within a single VLAN across the active devices in the same site, while preserving the independence of the sites. STP confined within the site (each site with its own STP Root bridge) Presentation_ID © 2009 Cisco Systems, Inc. All rights reserved. Cisco Confidential 11

Overlay Transport Virtualization Technology Pillars OTV is a “MAC in IP” technique for supporting Layer 2 VPNs OVER ANY TRANSPORT. Dynamic Encapsulation Protocol Learning No Pseudo-Wire State Maintenance Built-in Loop Prevention OTV Optimal Multicast Replication Preserve Failure Boundary Multi-point Connectivity Seamless Site Addition/Removal Point-to-Cloud Model Automated Multi-homing BRKRST-2033 © 2009 Cisco Systems, Inc. All rights reserved. Cisco Public 12

OTV at a Glance § Ethernet traffic between sites is encapsulated in IP: “MAC in IP” § Dynamic encapsulation based on MAC routing table § No Pseudo-Wire or Tunnel state maintained IP packet Ethernet Frame Encap MAC IF MAC 1 Eth 1 MAC 2 IP B MAC 3 IP B Decap OTV IP A West Site BRKRST-2033 Ethernet Frame © 2009 Cisco Systems, Inc. All rights reserved. OTV Communication between MAC 1 (West) and MAC 2 (East) Cisco Public IP B MAC IF MAC 1 IP A MAC 2 Eth 1 MAC 3 Eth 2 East Site 13

OTV Data Plane: Unicast MAC Table contains MAC addresses reachable through IP addresses MAC TABLE OTV Inter-Site Traffic 1 Layer 2 Lookup MAC TABLE VLAN MAC IF 100 MAC 1 Eth 2 100 MAC 2 Eth 1 100 MAC 3 IP B 100 MAC 4 IP B VLAN MAC 100 MAC 1 IP A 5 100 MAC 2 IP A Layer 2 Lookup 100 MAC 3 Eth 3 100 MAC 4 Eth 4 OTV MAC 2 Eth 2 MAC 1 MAC 3 External IP B External IP A Eth 1 MAC 1 MAC 3 L 2 3 Encap MAC 1 Eth 4 Eth 3 Core 2 MAC 4 MAC 1 MAC 1 3 MACIP 3 A IP B IP A IP B L 3 IF 6 L 3 L 2 MAC 1 MAC 3 4 Decap MAC 3 East West No Pseudo-Wire state is maintained. The encapsulation is done based on a Layer 2 destination lookup. BRKRST-2033 The encapsulation is done in hardware by the Forwarding Engine. © 2009 Cisco Systems, Inc. All rights reserved. Cisco Public 14

Building the MAC tables The OTV Control Plane § The OTV control plane proactively advertises MAC reachability (controlplane learning). § The MAC addresses are advertised in the background once OTV has been configured. § No protocol specific configuration is required. MAC Addresses Reachability Core IP A IP B East West IP C South BRKRST-2033 © 2009 Cisco Systems, Inc. All rights reserved. Cisco Public 15

OTV Control Plane MAC address advertisements – Multicast Core § Every time an Edge Device learns a new MAC address, the OTV control plane will advertise it together with its associated VLAN IDs and IP next hop. § The IP next hops are the addresses of the Edge Devices through which these MACs are reachable in the core. § A single update reaches all neighbors. OTV update is replicated by the core 4 VLAN 3 New MACs are learned on VLAN 100 Vlan 100 MAC A Vlan 100 MAC B Vlan 100 MAC C 1 O Up TV da te V e OT dat Up 3 Core 2 IP B IP A 3 IP C OTV te Upda West VLAN MAC IF 100 MAC A IP A 100 MAC B IP A 100 MAC C IP A East MAC IF 100 MAC A IP A 100 MAC B IP A 100 MAC C IP A 4 South-East BRKRST-2033 © 2009 Cisco Systems, Inc. All rights reserved. Cisco Public 16

Multicast Groups in the Core Summary of the Multicast groups used by OTV § OTV will leverage the multicast capabilities of the core. § This is the summary of the Multicast groups used by OTV: § An ASM/Bidir group to exchange MAC reachability. § An SSM group range for the multicast data generated by the site. Presentation_ID © 2009 Cisco Systems, Inc. All rights reserved. Cisco Confidential 17

What if core multicast is not an option? OTV in Unicast Mode – The Adjacency Server Mode § The use of multicast in the core provides significant benefits: Reduce the amount of hellos and updates OTV must issue Streamline neighbor discovery, site adds and removes Optimize the handling of broadcast and multicast data traffic § However multicast may not always be available § The OTV Adjacency Server Mode of operation provides a unicast based solution. Presentation_ID © 2009 Cisco Systems, Inc. All rights reserved. Cisco Confidential 18

Adjacency Server Core with no support for multicast: Adjacency Server § Despite the naming this is NOT a physical server. It is just a mode of operation of the Edge Devices. § An OTV node which sends a multicast packet on a non-multicast capable network will “unicast replicate (head-end)” the packet. § One of the OTV Edge Devices will be configured as an Adjacency Server and it will be responsible for communicating the IP addresses where the other Edge Devices can be reached. § The group of IP addresses is called overlay Adjacency List (o. AL) § Two configuration steps: 1. Configure an OTV Edge Device to be the Adjacency Server 2. Configure the other Edge Devices to point to the Adjacency Server to retrieve each other IP address Presentation_ID © 2009 Cisco Systems, Inc. All rights reserved. Cisco Confidential 19

Adjacency Server § At first, the Adjacency Server knows about no other OTV Edge Devices because their o. AL is empty. § Once other OTV Edge Devices start sending to the Adjacency Servers their site-id and IP address, the Adjacency Server will build up its o. AL. § The contents of the o. AL is advertised and sent unicast to each member of the o. AL. Now the Edge Devices can communicate with each other. Site 2 Site 1 o. A IP A Adjacency Server Presentation_ID 3 Site , IP C L o. AL Core L A B C D E e 2 Sit IP B , I o. AL Site 1, IP Site 2, IP Site 3, IP Site 4, IP Site 5, IP PB Site 3 Site 4, I PD © 2009 Cisco Systems, Inc. All rights reserved. IP D Site. Cisco 4 Confidential Site 5, IP E Site 5 20

STP BPDU Handling § When STP is configured at a site, an Edge Device will send and receive BPDUs on the internal interfaces. § An OTV Edge Device will not originate or forward BPDUs on the overlay network. § An OTV Edge Device can become (but it is not required to) a root of one or more spanning trees within the site. § An OTV Edge Device will take the typical action when receiving Topology Change Notification (TCNs) messages. The BPDUs stop here OTV Core Presentation_ID © 2009 Cisco Systems, Inc. All rights reserved. Cisco Confidential 21

Unknown Unicast Packet Handling § Flooding of unknown unicast over the overlay is not required and is therefore suppressed. § Any unknown unicasts that reach the OTV edge device will not be forwarded onto the overlay. § The assumption here is that the end-points connected to the network are not silent or uni-directional. § MAC addresses for uni-directional host are learnt and advertised by snooping the host’s ARP reply No MAC 3 in the MAC Table MAC TABLE OTV VLAN MAC IF 100 MAC 1 Eth 1 100 MAC 2 IP B Core MAC 1 MAC 3 Presentation_ID © 2009 Cisco Systems, Inc. All rights reserved. OTV Cisco Confidential 22

Controlling ARP traffic Proxy ARP § OTV Edge Devices can proxy ARP replies on behalf of remote hosts § ARP traffic spanning multiple sites can thus be significantly reduced § An ARP cache is maintained by every OTV edge device § The ARP cache is populated by snooping ARP replies § Initial ARP requests are broadcasted to all sites § Subsequent ARP requests are suppressed answered locally § The ARP cache could also be populated at MAC learning time, this would allow the suppression of all ARP related broadcast 4 Subsequent ARP requests (IP A) 5 Proxy ARP reply (IP A) First ARP request (IP A) BRKRST-2033 ARP reply OTV 1 2 Core 3 ARP Cache MAC 1 IP 1 MAC 2 IP 2 © 2009 Cisco Systems, Inc. All rights reserved. Snoop & cache ARP reply OTV AED One time traffic Cisco Public 23

OTV solves Layer 2 fault propagation Summary § STP Isolation: BPDUs are not forwarded over the overlay § Unknown unicasts are not flooded across sites Selective flooding is optional § Cross site ARP traffic is reduced with Proxy ARP § Broadcast can be controlled based on a white list as well as a rate limiting profile BRKRST-2033 © 2009 Cisco Systems, Inc. All rights reserved. Cisco Public 24

Multi-Homing: Loop Condition Handling § OTV includes the logic necessary to avoid the creation of loops in multi-homed site scenarios. § Each site will have its own STP domain, which is separate and independent from the STP domains in other sites, even though all sites will be part of common Layer 2 domain. STP domain 1 OTV No STP Core OTV STP domain 2 OTV Presentation_ID © 2009 Cisco Systems, Inc. All rights reserved. Cisco Confidential 25

Authoritative Edge Device § OTV provides loop-free multi-homing by electing a designated forwarding device per site for each VLAN. § The designated forwarder is referred to as the Authoritative Edge Device (AED). § The Edge Devices at the site peer with each other on the internal interfaces to elect the AED § The AED is the only edge device that will forward multicast and broadcast traffic between a site and the overlay. OTV Core OTV OTV AED Presentation_ID © 2009 Cisco Systems, Inc. All rights reserved. Cisco Confidential 26

Multi-Homing: AED & Broadcast § A broadcast packet gets to all the Edge Devices within a site. § The AED for the VLAN is the only Edge Device that forwards broadcast packets on the overlay network. § All the Edge Devices at a remote site will receive the broadcast packet, but only the AED at the remote site will forward the packet into the site. § Once sent into the site, the packet gets to all switches on the site specific Spanning Tree. OTV Broadcast stops here OTV Core OTV Bcast pkt Broadcast stops here OTV AED Presentation_ID © 2009 Cisco Systems, Inc. All rights reserved. Cisco Confidential 27

Multi-Homing AED & Unicast Forwarding § One AED is elected for each VLAN on each site § Different AEDs can be elected for each VLAN to balance traffic load § Only the AED forwards unicast traffic to and from the overlay § Only the AED advertises MAC addresses for any given site/VLAN § Unicast routes will point to the AED on the corresponding remote site/VLAN MAC TABLE VLAN MAC 100 MAC 1 200 MAC 2 IF OTV IP A OTV IP B AED IP A OTV Core OTV AED IP B OTV AED BRKRST-2033 © 2009 Cisco Systems, Inc. All rights reserved. Cisco Public AED 28

Configuration OTV CLI configuration Connects to the core. Used to join the Overlay network. Its IP address is used as source IP for the OTV encap ASM/Bidir group in the core used for the OTV Control Plane. interface Overlay 0 description otv-demo SSM group range used to carry the site’s mcast traffic data. otv join-interface Ethernet 1/1 otv control-group 239. 1. 1. 1 otv data-group 232. 192. 1. 2/32 otv extend-vlan 100 -150 otv site-vlan 100 Site VLANs being extended by OTV VLAN used within the Site for communication between the site’s Edge Devices BRKRST-2033 © 2009 Cisco Systems, Inc. All rights reserved. Cisco Public 29

Configuration OTV CLI configuration with adjacency server Connect to the core. Used to join the core mcast groups. Their IP addresses are used as source IP for the OTV encap interface Overlay 0 Configures this Edge device as an Adjacency Server description otv-demo Use a remote Edge Device as the Adjacency Server (mutually exclusive with the previous line) otv join-interface Ethernet 1/1 otv adjacency-server or otv use-adjacency-server 10. 10. 10 otv extend-vlan 100 -150 otv site-vlan 100 Site VLANs being extended by OTV VLAN used within the Site for communication between the site’s Edge Devices BRKRST-2033 © 2009 Cisco Systems, Inc. All rights reserved. Cisco Public 30

Nexus 7000 Rollout plan § EFT Target start date: Mid January, 2010 § FCS Q 1 CY 2010 Presentation_ID © 2009 Cisco Systems, Inc. All rights reserved. Cisco Confidential 31