Quick Start Guide Fabric Path Architecture Solutions Group

Quick Start Guide Fabric. Path Architecture & Solutions Group US Public Sector Advanced Services Mark Stinnette, CCIE Data Center #39151 Date 9 September 2013 Version 1. 13. 2 © 2013 Cisco and/or its affiliates. All rights reserved. 1

This Quick Start Guide (QSG) is a Cookbook style guide to Deploying Data Center technologies with end-to-end configurations for several commonly deployed architectures. This presentation will provide end-to-end configurations mapped directly to commonly deployed data center architecture topologies. In this cookbook style; quick start guide; configurations are broken down in an animated step by step process to a complete end-toend good clean configuration based on Cisco best practices and strong recommendations. Each QSG will contain set the stage content, technology component definitions, recommended best practices, and more importantly different scenario data center topologies mapped directly to complete end-to-end configurations. This QSG is geared for network engineers, network operators, and data center architects to allow them to quickly and effectively deploy these technologies in their data center infrastructure based on proven commonly deployed designs. © 2013 Cisco and/or its affiliates. All rights reserved. 2

Fabric. Path Design : : 2 SPINE (Routing at Aggregation) • • Simplest design option : : traditional Aggregation / Access designs Simplified configuration Removal of STP Traffic distribution over all uplinks without v. PC port-channels • • Active / Active gateways (via v. PC+ or Anycast HSRP) VLAN anywhere (no trunk ports) Option for v. PC+ for legacy access switches and computer connectivity Easily deploy L 4 -7 services Natural Evolution of the v. PC Design © 2013 Cisco and/or its affiliates. All rights reserved. 3

Fabric. Path Design : : 4 SPINE (Routing at Aggregation w/ Anycast HSRP) • Scale out; n-way Active HSRP in Fabric. Path (up to 4 today) • No longer need v. PC+ at SPINE for active/active HSRP • No peer-link or peer-keepalive link required • Leaf software needs to understand Anycast HSRP in Fabric. Path © 2013 Cisco and/or its affiliates. All rights reserved. 4

Fabric. Path Design : : Dedicated SPINE (Centralized Routing) © 2013 Cisco and/or its affiliates. All rights reserved. 5

Alternative View Fabric. Path Design : : Dedicated SPINE (Centralized Routing) • • • Paradigm shift with respect to typical designs (CLOS Fabric topology) Simplifies SPINE design Traditional “Aggregation” layer becomes pure Fabric. Path SPINE Design helps ensure that any application node are at most only two hops away Fabric. Path LEAF switches provide server connectivity like traditional designs Fabric. Path LEAF switches also provide L 2/L 3 boundary, inter-VLAN routing, North South routing Fabric. Path Deployment in Preparation For Dynamic Fabric Automation (DFA) © 2013 Cisco and/or its affiliates. All rights reserved. 6

Fabric. Path Design : : Multi POD (w/ FP Multi-Topology) NX-OS 6. 2 • Provides DC wide vs. POD local VLAN segmentation / isolation Can support VLAN ID reuse in multiple PODs • Define Fabric. Path VLANs : : map VLANs to topology : : map topology to Fabric. Path core ports • Optional design for “disconnected” PODs Each POD can use same non-default FP topology; don’t need Fabric. Path Core since each POD is on its own island © 2013 Cisco and/or its affiliates. All rights reserved. • Where to place DC wide L 2/L 3 boundary (v. PC+ or Anycast HSRP) Fabric. Path Core Pick a any Aggregation POD Routed Sub-interfaces on Routed Core / WAN Edge via CE edge ports • Default topology always includes all Fabric. Path core ports Map DC Wide VLANs to default topology • POD local core ports also mapped to POD local topology Map POD local VLANs to POD local topology 7

Fabric. Path Terminology © 2013 Cisco and/or its affiliates. All rights reserved. 8

Fabric. Path Encapsulation © 2013 Cisco and/or its affiliates. All rights reserved. 9

Benefits Overview Fabric. Path is a next generation Layer 2 technology from Cisco that provides multi-path Ethernet capabilities in L 2 switching networks. Fabric. Path combines the benefits of L 2 switching such as easy configuration and workload flexibility with greater scalability and availability. Specifically, Fabric. Path adds to L 2 switching some routing type capabilities such as all active links, fast convergence, and loop avoidance mechanisms in the data plane. It allows Layer 2 networking without Spanning Tree Protocol. Fabric. Path provides the following benefits: • Eliminates Spanning Tree Protocol (STP) with built-in loop prevention and mitigation (TTL & RPF) • Single control plane for unknown unicast, broadcast, and multicast traffic • VLAN anywhere • FP is transparent to L 3 protocols • Easy to configure • Easy to manage • Flexibility • Create arbitrary any topology • Multiple designs to integrate L 2/L 3 boundaries • Start small and expand as needed (bandwidth growth) • Efficient and Scalable • Layer 3 availability similar features • Leverage parallel paths • Expanding available bandwidth at L 2/L 3 Default Gateway level • MAC address table scale (conversational learning) : : all Fabric. Path VLANs use conversation MAC address learning • Fast Convergence and low latency • Enhances mobility and virtualization in the Fabric. Path network • Capable of running v. PC (called v. PC+) to connect devices to the edge in a port channel • Multi-tenant support, traffic engineering, meet security separation requirements via Fabric. Path topologies © 2013 Cisco and/or its affiliates. All rights reserved. 10

Feature Configuration Feature Benefit Overview fabricpath VLAN mode & switchport mode Eliminate STP protocol from the infrastructure fabric The Fabric. Path ports carry traffic only for those VLANs configured as Fabric. Path VLANs. It is mandatory to enable the same FP mode VLAN EVERYWHERE on all switches in the FP fabric (otherwise, FP multidestination trees will be incorrectly built). VLAN pruning is performed automatically on FP core ports for FP traffic only. fabricpath forwarding tables Service Continuity Fabric. Path uses 3 HW forwarding tables to switch frames (1) MAC address table, (2) Switch-ID table, (3) Multidestination tree table fabricpath switch IDs Service Continuity Each switch in the FP fabric is allocated with a global switch ID value; this is allocated automatically or manually set (recommended). The switch ID information will be used in the MAC address-table for L 2 forwarding. The v. PC+ system also uses an emulated switch ID; which you assign on both peer devices. fabricpah IS-IS link metric Increase High. Availability FP will always take the path with the lowest metric. Its recommended to use the default reference bandwidth. fabricpath timers Improve Convergence Time On a case by case basis, if convergence time needs to be improved upon switch reload, modify lsp-gen-interval and spf-interval timers. fabricpath root priority Service Continuity FP uses two Multi destination Trees, Tree 1 (ftag 1) for broadcast, unknown unicast, multicast & Tree 2 (ftag 2) –multicast. Recommend to use on SPINE switches for primary and secondary root. STP for Classical Ethernet (CE) Service Continuity The FP fabric must be the root of the L 2 domain when connected to other legacy L 2 domains / switches. Make sure STP priority is the lowest for the entire FP fabric. v. PC+ Increase High. Availability Fabric. Path & v. PC+ combined provides two main purposes, (1) dual attach a host to the FP fabric, (2) Leverage Active/Active HSRP capability © 2013 Cisco and/or its affiliates. All rights reserved. 11

Feature Configuration Feature Benefit Overview Fabric. Path multicast load-balance Service Continuity Cisco NX-OS provides a way to control two peers to be partial designated forwarders when both v. PC paths are up. When this control is enabled, each peer can be the designated forwarder for multi destination southbound packets for a disjoint set of RBHs/FTAGs (depending on the hardware). The designated forwarder is negotiated on a per-v. PC basis. There are three designated forwarder states for a v. PC port: § All—If the local v. PC leg is up and the peer v. PC is not configured or down, the local switch is the designated forwarder for all RBHs/FTAGs for that v. PC. § Partial—If the v. PC path is up on both sides, each peer is the designated forwarder for half the RBHs or FTags. For the latter, the v. PC port allows only the active FTags on that peer. This mode is used in a FEX with v. PC+ topology. § None—If the local v. PC path is down or not configured, the local switch does not forward any multi destination packets from this v. PC path. The fabricpath multicast load-balance command is required for configuring v. PC+ with FEX ports. © 2013 Cisco and/or its affiliates. All rights reserved. 12

Feature Configuration Feature Benefit Overview Overload Bit Improve Convergence Time RFC 3277 based Overload bit is advertised in updates to prevent a corner case when a single switch restarts causing temporary loops or traffic black holing. This feature also prevents neighbors from using a switch as a transit during initial convergence as well as lowering impact insertion or removal of a switch to the FP domain. Multiple Topologies Design Separation With multiple topologies, we can create up to 16 topologies where a subset of VLANs are mapped to a particular topology; allowing more design possibilities. Anycast HSRP Increase High. Availability Provides up to 4 active Default Gateways for the network which lowers the risk of disruption for routed and Inter-VLAN traffic and provides bandwidth capacity at L 2/L 3 boundaries. The Anycast HSRP feature removes the reliance on v. PC+ to provide the Active/Active HSRP feature at the L 2/L 3 boundary. fabricpath static routes Traffic Engineering The static route feature gives users capabilities to enter routes directly in the forwarding tables, ensuring predictable operation of the network. Certain uses cases where users want to override the routes computed by IS-IS. Users might want to route traffic to a particular switch using a particular link, better load balancing or routing traffic through a firewall (policing) in the network. © 2013 Cisco and/or its affiliates. All rights reserved. 13

Install license bootflash: ///enchanced_layer 2_pkg. lic show license usage Initial Baseline (Only 4 Commands !!) feature lacp install feature-set fabricpath Default / Admin VDC Only vlan 1 – 200 mode fabricpath interface po 2 switchport mode fabricpath interface e 3/1, e 4/1 channel-group 2 mode active interface e 5/1, e 5/2 switchport mode fabricpath feature lacp install feature-set fabricpath vlan 1 – 200 mode fabricpath interface po 2 switchport mode fabricpath interface e 1/1, e 1/2 channel-group 2 mode active interface e 1/3, e 1/4 switchport mode fabricpath © 2013 Cisco and/or its affiliates. All rights reserved. Step 1 : : install | validate Enhanced L 2 License Step 2 : : install Fabric. Path Step 3 : : enable Fabric. Path Step 4 : : configure Fabric. Path VLANs Step 5 : : configure Fabric. Path core ports Beginning with the Cisco NX-OS Release 5. 1 and when you use an F Series modules and NX-OS Release 5. 1(3) N 1(1) with 5500 you can use the Fabric. Path feature Default / Admin VDC Only interface po 2 switchport mode fabricpath interface e 1/1, e 1/2 channel-group 2 mode active interface e 1/3, e 1/4 switchport mode fabricpath 14

Manually Set the Fabric. Path Switch-ID & Root fabricpath switch-id 10 fabricpath switch-id 11 Root for FTAG 1 fabricpath domain default root-priority 255 Root for FTAG 2 SW 10 SW 11 fabricpath switch-id 100 fabricpath switch-id 101 SW 100 Each peer devices will have a unique global switch ID value – make the FP network more deterministic Suggested switch ID scheme: SPINE : : 2 digit ID LEAF : : 3 digit ID Emulated Switch (v. PC+) : : 4 digit ID SW 101 Step 1 : : set the FP Switch-ID Step 2 : : set the FP Root Multi destination Tree 1 (ftag 1) – broadcast, unknown unicast, multicast Multi destination Tree 2 (ftag 2) –multicast Recommend to use on SPINE switches Higher Number the better !! F 2/F 2 E uses both trees for UU/Bcast/Mcast F 1 uses MDT 2 for Mcast only © 2013 Cisco and/or its affiliates. All rights reserved. fabricpath domain default root-priority 254 (start at 255 and go backwards) -or(start at 200 in case you need to introduce another MDT at a later time; ie expanded SPINE x 4) 15

Manually Set the Spanning-Tree : : Single Virtual Root Bridge) vlan 1 – 200 mode fabricpath spanning-tree pseudo-information vlan 1 – 200 root priority 0 optional vlan 1 – 200 mode fabricpath spanning-tree pseudo-information vlan 1 – 200 root priority 0 The entire Fabric. Path domain will look like one virtual bridge to the CE domain – set best (lowest) STP root priority on the v. PC+ peers (recommended at least at the access edge leaf switches); just make sure the priority is lower than anything else in the network (classical Ethernet) vlan 20, 40 spanning-tree vlan 20, 40 priority 8192 FP will use the same bridge ID c 84 c. 75 fa. 6000 The root and sender bridge MAC addresses of this pseudo -information are the same on every switch in the Cisco Fabric. Path domain All ports at the edge of a Cisco Fabric. Path network are configured with the equivalent of root guard (don’t need to configure this feature), a feature that would block a port should it receive superior Spanning Tree Protocol BPDUs © 2013 Cisco and/or its affiliates. All rights reserved. optional Step 1 : : set FP domain to be root bridge Note that the spanning-tree priority command would also work; however, it would change the priority for the spanning tree regardless of whether the switch were sending regular BPDUs (when Cisco Fabric. Path is not running) or sending BPDUs with the pseudo-information (when Cisco Fabric. Path is operational on the switch). In some scenarios, this change can have undesirable side effects. 16

Tune Timers for Fast Convergence fabricpath domain default spf-interval 50 50 50 lsp-gen-interval 50 50 50 fabricpath timers linkup-delay 60 Problem Set: The IS-IS adjacency is established and the access-edge started sending traffic to aggregationedge, but the control plane was not ready to forward the traffic to the next hop. The default spf and lsp-gen intervals are 8 sec (default) and it attributes to the long convergence. To address this issue, the default spf and lsp-gen intervals of {max-wait, initial-wait, second-wait} are brought down to 50 msec, with this configuration, the aggregation-edge restoration will yield sub-second convergence for Layer 2 traffic Note: Future enhancements such as Layer 2 IS-IS overload bit support in 6. 2 will help to improve unicast and multicast convergence during Fabric. Path node failure scenarios when default IS-IS timers are used. Step 1 : : tune the IS-IS timers in Fabric. Path Step 2 : : (optional) tune the Fabric. Path linkup-delay To achieve fast convergence during node failures and recovery scenarios, it is recommended to tune the IS-IS timers in Cisco Fabric. Path. This tuning is particularly important when a switch is inserted in the topology. This configuration is recommended for all switches in the network © 2013 Cisco and/or its affiliates. All rights reserved. Optional, to provide better network convergence upon a Cisco Fabric. Path switch restart, you should set a Cisco Fabric. Path linkup-delay timer to 60 17

Enable v. PC+ : : Dual Attachment & Active/Active HSRP feature vpc vpc domain 1 role priority 1 peer-keepalive destination […. ] source […. ] …. ip arp synchronize fabricpath multicast load-balance fabricpath switch-id 1000 dual-active exclude interface vlan 20 v. PC+ SW 1000 interface po 2 switchport mode fabricpath vpc peer-link With v. PC+, a Fabric. Path switch is emulated between the CE and Fabric. Path domain. All packets originating behind the Emulated Switch will be marked with the source Switch ID of the emulated switch v. PC+ is an extension of v. PC for Fabric. Path. It allows dual-homed connections from Classical Ethernet (CE) switches and hosts capable of port channels. It also provides for active-active HSRP. Assign the same emulated switch ID on both v. PC peers; but the emulated switch ID must be unique between different v. PC domains The configuration of peer-link and peer-keepalive links are required – as traditional v. PC Enable IP ARP Synchronization of ARP entries between v. PC Peers improves convergence for North. South and East-West Layer 3 traffic when one of the v. PC+ peers is brought back up Note: Since Fabric. Path does not rely on Spanning Tree Protocol, and the v. PC+ peer link is a Fabric. Path Core port, so the peer-switch command is not needed under the vpc domain [x] configuration © 2013 Cisco and/or its affiliates. All rights reserved. vpc domain 1 role priority 2 peer-keepalive destination […. ] source […. ] …. ip arp synchronize fabricpath multicast load-balance fabricpath switch-id 1000 dual-active exclude interface vlan 20 Step 1 : : enable v. PC+ Step 2 : : set the emulated switch-id Step 3 : : enable dual-active exclude for v. PC SVIs In a v. PC environment, the Secondary v. PC switch will bring down the SVIs by default when the peer-link is brought down. This behavior is fine in CE environment as the v. PC legs are also brought down on the secondary v. PC switch. However in the v. PC+ environment, the down links to the Access-Edge switches are Fabric. Path Core ports; in the absence of the v. PC+ peer-link, the SVIs can still communicate through the Fabric. Path core ports. The v. PC dual-active exclude vlan command helps to configure a VLAN list such that the SVI can continue to stay up on the secondary v. PC switch even if the v. PC+ peer-link is down. 18

Note: In a Fabric. Path v. PC+ environment both HSRP peers are actively forwarding, there is no need to configure preemption, different priorities, and fast hello timers. Enable v. PC+ : : Active/Active HSRP @ SPINE (Full Configuration) feature interface-vlan feature hsrp feature lacp feature vpc v. PC+ vlan 1 – 200 mode fabricpath SW 1000 feature interface-vlan feature hsrp feature lacp feature vpc vlan 1 – 200 mode fabricpath spanning-tree pseudo-information vlan 1 – 200 root priority 0 ------------------------ vpc domain 1 role priority 1 system-priority 4096 peer-keepalive destination […. ] source […. ] peer-gateway auto-recovery reload-delay restore 30 ip arp synchronize fabricpath multicast load-balance fabricpath switch-id 1000 dual-active exclude interface vlan 20 vpc domain 1 role priority 2 system-priority 4096 peer-keepalive destination […. ] source […. ] peer-gateway auto-recovery reload-delay restore 30 ip arp synchronize fabricpath multicast load-balance fabricpath switch-id 1000 dual-active exclude interface vlan 20 interface po 2 switchport mode fabricpath vpc peer-link interface e 3/1, e 4/1 channel-group 2 mode active © 2013 Cisco and/or its affiliates. All rights reserved. Step 1 : : enable v. PC+ Step 2 : : set the emulated switch-id Step 3 : : enable dual-active exclude for v. PC+ SVIs interface vlan 20 ip address 20. 20. 5/24 no ip redirect hsrp 20 ip 20. 20. 254 interface vlan 20 ip address 20. 20. 6/24 no ip redirect hsrp 20 ip 20. 20. 254 interface po 2 switchport mode fabricpath vpc peer-link interface e 3/1, e 4/1 channel-group 2 mode active 19

Enable v. PC+ : : Dual Attachment @ LEAF feature lacp feature vpc vlan 1 – 200 mode fabricpath v. PC+ SW 1000 feature lacp feature vpc vlan 1 – 200 mode fabricpath spanning-tree pseudo-information vlan 1 – 200 root priority 0 vpc domain 10 role priority 1 peer-keepalive destination […. ] source […. ] …. ip arp synchronize fabricpath multicast load-balance fabricpath switch-id 1001 vpc domain 10 role priority 2 peer-keepalive destination […. ] source […. ] …. ip arp synchronize fabricpath multicast load-balance fabricpath switch-id 1001 v. PC+ SW 1000 1001 interface po 2 switchport mode fabricpath vpc peer-link interface e 1/1, e 1/2 channel-group 2 mode active interface port-channel 20 switchport mode trunk switchport trunk allowed vlan 20 – 40 vpc 20 interface e 1/5 channel-group 20 force mode active © 2013 Cisco and/or its affiliates. All rights reserved. interface po 2 switchport mode fabricpath vpc peer-link v. PC 20 Step 1 : : enable v. PC+ Step 2 : : set the emulated switch-id Step 3 : : add devices redundantly with v. PC+ VLANs carried on v. PC+ member ports must be Fabric. Path mode VLANs interface e 1/1, e 1/2 channel-group 2 mode active interface port-channel 20 switchport mode trunk switchport trunk allowed vlan 20 – 40 vpc 20 interface e 1/5 channel-group 20 force mode active 20

Fabric. Path Authentication interface port-channel 2 switchport mode fabricpath isis authentication-type md 5 fabricpath isis authentication key-chain FP-KEYS fabricpath domain default authentication-type md 5 authentication key-chain FP-KEYS key 0 key-string Cisc 0! accept-lifetime 00: 00: 00 Sep 1 2012 infinite send-lifetime 00: 00: 00 Sep 1 2012 infinite global level authentication : : authenticates and controls the FP LSPs and PSNPs interfaces level authentication : : authenticates the HELLO; the FP ISIS adjacency Fabric. Path provides 2 levels of authentication 1. Authentication at the interfaces level 2. Authentication at the global level The Key chain is used in both forms of authentication Supported combinations: Step 1 : : configure the key chain Step 2 : : configure global Fabric. Path authentication Step 3 : : configure Fabric. Path core port authentication You can configure the accept lifetime and send lifetime for a key. By default, accept and send lifetimes for a key are infinite, which means that the key is always valid. accept-lifetime [local] start-time duration-value | infinite | end-time] send-lifetime [local] start-time duration-value | infinite | end-time] © 2013 Cisco and/or its affiliates. All rights reserved. 21

NX-OS 6. 2(2) HSRP CP : : Control Plane DP : : Data Plane Any. Cast HSRP feature-set fabricpath feature interface-vlan feature hsrp Step 1 : : Step 2 : : Step 3 : : Step 4 : : Step 5 : : Step 6 : : Step 7 : : vlan 1 – 200 mode fabricpath spanning-tree pseudo-information vlan 1 – 200 root priority 0 fabricpath switch-id 10 SW 10 fabricpath domain default root-priority 255 hsrp anycast 100 ipv 4 switch-id 1000 vlan 20 priority 110 feature-set fabricpath feature interface-vlan feature hsrp enable required features configure SVI configure hsrp configure anycast bundle associate anycast switch id associate a set vlans designate active HSRP router Active HSRP CP Active HSRP DP Virtual FP-ID 1000 vlan 1 – 200 mode fabricpath spanning-tree pseudo-information vlan 1 – 200 root priority 0 SW 11 fabricpath switch-id 11 fabricpath domain default root-priority 254 hsrp anycast 100 ipv 4 switch-id 1000 vlan 20 interface vlan 20 ip address 20. 20. 2/24 ip router ospf 1 area 0. 0 ip ospf passive-interface no ip redirect hsrp version 2 hsrp 100 ip 20. 20. 1 interface vlan 20 ip address 20. 20. 3/24 ip router ospf 1 area 0. 0 ip ospf passive-interface no ip redirect hsrp version 2 hsrp 100 ip 20. 20. 1 interface e 5/1, e 5/2 switchport mode fabricpath SW 100 § You don’t need to enable v. PC+ to achieve active/active HSRP § § No v. PC domain configuration required No peer-link required The Fabric. Path feature-set has to be enabled before configuring HSRP anycast 4 gateways are supported in an HSRP Anycast bundle for a common VLAN HSRPv 2 is required (IPv 4/IPv 6 address-families supported) An Anycast bundle can reference multiple VLANs © 2013 Cisco and/or its affiliates. All rights reserved. SW 101 § Downstream switches use the virtual FP-ID to equal cost route traffic destined to § § all HSRP anycast devices; the active HSRP router advertises the anycast switch ID as the source switch ID in Fabric. Path IS-IS All Leaf devices need to support and be aware of the Anycast functionality Nexus 7000 : : NX-OS 6. 2(2) and later releases Nexus 5500 & 6000 : : NX-OS 6. 0(2)N 2(1) and later releases Nexus 5500 & 6000 can support Anycast HSRP Gateway functionality in 6. 0(2)N 3(1) and later releases 22

NX-OS 6. 2(2) HSRP CP Any. Cast HSRP : : 4 SPINE Common Configuration feature-set fabricpath feature interface-vlan feature hsrp vlan 1 – 200 mode fabricpath spanning-tree pseudo-information vlan 1 – 200 root priority 0 HSRP DP SW 10 SW 11 SW 12 SW 13 Anycast HSRP Capable: : A FP switch can work as an Anycast HSRP Router / Gateway Anycast HSRP Aware: : Same as "Anycast HSRP Leaf". Can send traffic to multiple Anycast HSRP capable switches Can recognize Anycast TLV sent from Anycast HSRP capable switches N 7 K is Anycast HSRP Capable & Aware : : 6. 2(2) N 6 K/N 5 K is Anycast HSRP Aware : : 6. 0(2)N 2(1) --------N 6 K/N 5 K is Anycast HSRP Capable & Aware : : 6. 0(2)N 3(1) Virtual FP-ID 1000 interface e 5/1, e 5/2 switchport mode fabricpath SW 100 SPINE 7 k-1 SPINE 7 k-2 SPINE 7 k-3 SPINE 7 k-4 fabricpath switch-id 10 fabricpath switch-id 11 fabricpath switch-id 12 fabricpath switch-id 13 fabricpath domain default root-priority 255 fabricpath domain default root-priority 254 fabricpath domain default root-priority 253 fabricpath domain default root-priority 252 hsrp anycast 100 ipv 4 switch-id 1000 vlan 20, 100 -120 priority 110 hsrp anycast 100 ipv 4 switch-id 1000 vlan 20, 100 -120 interface vlan 20 ip address 20. 20. 1/24 ip router ospf 1 area 0. 0 ip ospf passive-interface no ip redirect hsrp version 2 hsrp 100 ip 20. 20. 254 interface vlan 20 ip address 20. 20. 2/24 ip router ospf 1 area 0. 0 ip ospf passive-interface no ip redirect hsrp version 2 hsrp 100 ip 20. 20. 254 interface vlan 20 ip address 20. 20. 3/24 ip router ospf 1 area 0. 0 ip ospf passive-interface no ip redirect hsrp version 2 hsrp 100 ip 20. 20. 254 interface vlan 20 ip address 20. 20. 4/24 ip router ospf 1 area 0. 0 ip ospf passive-interface no ip redirect hsrp version 2 hsrp 100 ip 20. 20. 254 © 2013 Cisco and/or its affiliates. All rights reserved. SW 101 HSRP CP : : Control Plane DP : : Data Plane 23

NX-OS 6. 2(2) Optional : : Overload Bit & VLAN Pruning Common Configuration feature-set fabricpath feature interface-vlan feature hsrp vlan 1 – 200 mode fabricpath interface e 5/1, e 5/2 switchport mode fabricpath domain default spf-interval 50 50 50 lsp-gen-interval 50 50 50 SW 10 Overload bit : : You can configure the overload bit for Fabric. Path IS-IS. You achieve consistent routing behavior in conditions where a node reboots or gets overloaded. always—The overload bit is always on; out of service. on-startup—The overload bit is set upon system startup and remains set for the specified number of seconds. SW 11 VLAN pruning : : The switch will only attract data traffic for the VLANs that have active Classic Ethernet (CE) ports on an F 1 Series module, F 2 Series module, or switch virtual interfaces (SVIs) for those VLANs. Optional command is only mentioned for informational purposes only; use appropriately. SPINE 7 k-2 SPINE 7 k-3 SW 101 SPINE 7 k-4 fabricpath switch-id 10 fabricpath switch-id 11 fabricpath switch-id 12 fabricpath switch-id 13 fabricpath domain default root-priority 255 fabricpath domain default root-priority 254 fabricpath domain default root-priority 253 fabricpath domain default root-priority 252 fabricpath domain default set-overload-bit on-startup [sec] vlan pruning enabled © 2013 Cisco and/or its affiliates. All rights reserved. SW 13 Virtual FP-ID 1000 SW 100 SPINE 7 k-1 SW 12 24

NX-OS 6. 2(2) Fabric. Path Static Routes : : Traffic Engineering Common Configuration feature-set fabricpath feature interface-vlan feature hsrp vlan 1 – 200 mode fabricpath Fabric. Path uses Layer 2 Integrated Intermediate System-to-System (IS-IS) as a link state protocol to compute unicast topologies. You can configure unicast static routes in the forwarding tables to ensure a predictable operation of the network or to override the routes computed by dynamic protocols such as IS-IS in Fabric. Path. For example, you might want to route traffic to a particular device using a specific link to ensure better load balancing or to route traffic through a firewall in the network. interface e 5/1, e 5/2 switchport mode fabricpath SW 10 SW 11 SW 12 SW 13 Virtual FP-ID 1000 SPINE 7 k-1 fabricpath switch-id 10 fabricpath domain default root-priority 255 interface e 5/3 switchport mode fabricpath route switch-id 100 e 5/3 ------------fabricpath topology 1 fabricpath route switch-id 100 e 5/4 © 2013 Cisco and/or its affiliates. All rights reserved. SW 100 SW 101 25

NX-OS 6. 2(2) Multiple Topologies SW 1 SW 2 Virtual FP-ID 1000 SPINE 7 k-1 SPINE 7 k-2 SPINE 7 k-5 feature-set fabricpath feature interface-vlan feature hsrp SW 10 vlan 10 – 20, 50 – 60, 100 – 200 mode fabricpath vlan 100 – 200 mode fabricpath Virtual FP-ID 1100 fabricpath switch-id 11 fabricpath switch-id 1 fabricpath domain default topology 1 root-priority 255 topology 2 root-primary 255 fabricpath domain default topology 1 root-priority 254 topology 2 root-primary 254 fabricpath domain default root-priority 255 fabricpath topology 1 member vlan 10 – 20 fabricpath topology 2 member vlan 50 – 60 interface e 5/1, e 5/2 switchport mode fabricpath interface e 6/1 fabricpath topology-member 1 switchport mode fabricpath interface e 6/2 fabricpath topology-member 2 switchport mode fabricpath SW 100 SW 13 Virtual FP-ID 1101 SW 102 interface e 5/1, e 5/2, e 5/3, e 5/4 switchport mode fabricpath SPINE 7 k-6 feature-set fabricpath feature interface-vlan feature hsrp vlan 100 – 200 mode fabricpath The Fabric. Path domain can now consist of multiple logical topologies. By default all VLANs belong to Topology 0, which consisted of a single pair of multi-destination trees. Now with NX-OS 6. 2(2) a total of 16 Fabric. Path topologies can be configured and assigned to a Fabric. Path domain; allowing VLANs to be assigned to different topologies. fabricpath switch-id 2 fabricpath domain default root-priority 255 interface e 5/1, e 5/2, e 5/3, e 5/4 switchport mode fabricpath © 2013 Cisco and/or its affiliates. All rights reserved. SW 12 SW 11 § All VLANs by default belong to topology 0 (VLANs anywhere) § A VLAN can only belong to a one topology § All interfaces by default belong to topology 0 § An interface can belong to multiple topologies 26

NX-OS 6. 2(2) Multiple Topologies SW 1 SW 2 Virtual FP-ID 1000 SPINE 7 k-3 SPINE 7 k-4 feature-set fabricpath feature interface-vlan feature hsrp vlan 80 – 90, 100 – 200 mode fabricpath switch-id 12 fabricpath switch-id 13 fabricpath domain default topology 1 root-priority 255 fabricpath domain default topology 1 root-priority 254 fabricpath topology 1 member vlan 80 – 90 interface e 5/1, e 5/2 switchport mode fabricpath interface e 6/1 fabricpath topology-member 1 switchport mode fabricpath LEAF 5 k-3 feature-set fabricpath SW 10 vlan 80 – 90, 100 – 200 mode fabricpath SW 12 SW 11 SW 13 Virtual FP-ID 1101 Virtual FP-ID 1100 fabricpath switch-id 102 fabricpath topology 1 member vlan 80 – 90 interface e 1/3, e 1/4 fabricpath topology 1 switchport mode fabricpath SW 100 SW 101 SW 102 spanning-tree pseudo-information vlan 80 – 90, 100 – 200 root priority 0 Be aware of the Fabric. Path topology scaling limits across the different NX-OS versions on the Nexus 5000 & 6000 switching platforms. The Fabric. Path domain can now consist of multiple logical topologies. By default all VLANs belong to Topology 0, which consisted of a single pair of multi-destination trees. Now with NX-OS 6. 2(2) a total of 16 Fabric. Path topologies can be configured and assigned to a Fabric. Path domain; allowing VLANs to be assigned to different topologies. § All VLANs by default belong to topology 0 (VLANs anywhere) § A VLAN can only belong to a one topology § All interfaces by default belong to topology 0 § An interface can belong to multiple topologies © 2013 Cisco and/or its affiliates. All rights reserved. 27

Fabric. Path is Easy & Simple !! interface e 1/5 ip address 192. 168. 1. 1/24 vrf membership vpc-keepalive vpc domain 1 peer-keepalive destination 192. 168. 1. 2 source 192. 168. 1. 1 vrf vpc-keepalive interface port-channel 1000 switchport mode trunk vpc peer-link interface e 1/1 -2 switchport mode trunk channel-group 1000 mode active interface e 1/3 switchport mode trunk channel-group 1 mode active interface port-channel 1 vpc 1 interface e 2/5 ip address 192. 168. 1. 2/24 vrf membership vpc-keepalive vpc domain 1 peer-keepalive destination 192. 168. 1. 1 source 192. 168. 1. 2 vrf vpc-keepalive interface port-channel 1000 switchport mode trunk vpc peer-link interface e 2/1 -2 switchport mode trunk channel-group 1000 mode active interface e 2/3 switchport mode trunk channel-group 1 mode active interface port-channel 1 vpc 1 e 1/5 v. PC Fabric. Path + v. PC+ Advantages Active/active path at L 2 Active/active for HSRP Works with all LC Advantages Active/active path at L 2 Active/active for HSRP Ease of configuration No more STP Extensibility Drawbacks Need dedicated infrastructure (PL, PKL) Configuration on both peer devices Consistency check to care about STP still here (but runs as fail safe mechanism) Drawbacks Need dedicated infrastructure (PL, PKL) Need F 1 (+M 1) or F 2 interface e 1/1 -3 switchport mode fabricpath interface e 2/1 -3 switchport mode fabricpath e 2/5 e 1/1 -2 e 1/3 e 2/1 -2 VPC e 2/3 e 3/1 -2 interface e 3/1 -2 switchport mode trunk channel-group 1 mode passive VPC Configuration © 2013 Cisco and/or its affiliates. All rights reserved. e 1/1 -2 e 1/3 e 2/1 -2 Fabric. Path e 2/3 e 3/1 -2 interface e 3/1 -2 switchport mode fabricpath Fabric. Path Configuration 28

Common Design Migration Starting Point 7 k – Aggregation 5 k/2 k – Access Pods Dual Layer v. PC Mix F 1 / M 1 line cards After Migration Completion 7 k – SPINE role 5 k – LEAF role v. PC converted to Fabric. Path core ports Peer-Link also FP core port = v. PC+ (only F 1/F 2 support Fabric. Path) Additional Reading Here : : http: //www. cisco. com/en/US/prod/collateral/switches/ps 9441/ps 9402/white_paper_c 11 -709336. html © 2013 Cisco and/or its affiliates. All rights reserved. 29

Strong Recommendations and Key Notes • Fabric. Path VLANs must be configured on all switches in the FP domain • It is recommended to configure the switch ID manually on all Fabric. Path switches • For Active-Active HSRP capability, it is recommended to configure v. PC+ on the Aggregation-Edge switches even if there are no v. PC legs. Note: subject to v. PC rules; so no dynamic routing over v. PC to firewalls, Core layer, WAN edge • The Nexus 6000 will supports a routing protocol over v. PC+ with the 6. 0(2)N 2(1) release • Implement Layer 3 routing backup path Separate L 3 port channel; point-to-point links Separate L 2 port channel; use dedicated VLAN in Classical Ethernet (CE) mode as transit VLAN inside this L 2 trunk • Disable IP redirects on SVIs and configure passive interface to avoid any routing adjacency over SVIs • ARP sync feature with v. PC+ is recommended for improved traffic convergence during Aggregation-Edge failure and restoration • It is recommended to configure highest and second highest MDT root priority on the Aggregation-Edge switches • Have option of choosingle links or port-channels between Aggregation-Edge and Access-Edge for ECMP. If port channels are used, configuring IS-IS metric is preferred. With path costing, member link failure is transparent to IS-IS protocol so that the traffic would continue to use the same path © 2013 Cisco and/or its affiliates. All rights reserved. 30

Strong Recommendations and Key Notes • It is recommended to have lowest path cost for the links between Aggregation devices so the multicast hello packets always take the peer-link which is direct link between the AGG devices • It is recommended to tune Layer 2 IS-IS SPF and LSP generation timers to achieve better convergence during failure and restoration scenarios. These timers should be tuned to 50 msec with 50 msec initial wait and second wait. This is a requirement until the overload bit support is available with Layer 2 IS-IS • Use default reference BW (its 400 Gbps default) fabricpath domain default reference-bandwidth ? • IS-IS metric cost (1 Gb = cost 400, 10 Gb = cost 40, 20 Gb = cost 20) • IS-IS link metric for port-channel depends on NX-OS version Up to NX-OS 6. 0: IS-IS metric for port-channel is calculated based on number of configured member ports; meaning you may need to use LACP min-link feature to tear down port-channel if number of active member ports goes below a specific limit Since NX-OS 6. 1: IS-IS metric for port-channel is calculated based on number of active ports • Dual-active exclude VLAN configuration is recommended so that the SVIs can continue to be active on the secondary v. PC+ peer in the event of peer-link failure. This also helps to stay with default HSRP timers there by reducing control plane load associated with aggressive HSRP timers • Do not use dual-active exclude command for VLANs if you have v. PC attached devices, for example at the access (leaf) © 2013 Cisco and/or its affiliates. All rights reserved. 31

Strong Recommendations and Key Notes • In typical v. PC deployments it is not necessary to tune the HSRP hello timers from the defaults (3/10 s). In a Classic Ethernet environment, in which in a single VLAN only one HSRP gateway can be active at a time, fast failover between the remaining peers is essential. Typically, fast hello timers and preemption is configured to enforce the required behavior. But in a Cisco Fabric. Path v. PC+ environment, both HSRP peers are actively forwarding, fast hello timers and preemption is no longer required, and configurations can be left at their defaults. • In CE-Fabric. Path hybrid networks, it is recommended to configure the lowest Spanning-tree root priority on all Fabric. Path Edge switches • The MAC timer should be consistent on all devices in the Layer 2 topology. The MAC and ARP aging timers can be left at defaults, 1800 sec & 1500 sec respectively • The M 1/F 1 mixed VDC currently supports up to 16 K MAC/ARP entries. This limitation will be lifted with the Layer 2 proxy learning feature in the upcoming NX-OS release • The M 1, M 1 -XL, M 2 & F 2 E in a mixed VDC topology; meaning when F 2 E is placed in a chassis with M-series it will operate in Layer 2 mode only leveraging the M for Layer 3 (proxy L 3 forwarding); this will enable 128 K MAC/ARP scale. • If an ASA cluster is attached to the Nexus 7000 series Aggregation-edge switches, source-dest-ip or src-dst ip-l 4 port is the recommended load balance algorithm if the ASA cluster is in single context mode or if the VLANs are fewer in multi-context mode. This is to prevent traffic polarization on links towards ASA cluster member © 2013 Cisco and/or its affiliates. All rights reserved. 32

Strong Recommendations and Key Notes • Better use port-channel instead of individual links for the 2 following reasons Decrease the number of direct IS-IS adjacency (1 for the whole port-channel instead of X IS-IS adjacencies if X individual links are used between the 2 switches) Allows to use the whole port-channel capacity for multidestination tree #1 or #2 (if multiple parallel individual links exist between 2 switches, only 1 link will be selected for tree #1 and potentially 1 another link for tree #2) • ECMP vs. Port Channel Can use ECMP, port-channel, or both simultaneously Port-channels have one main advantage over ECMP – treated as single logical link in Fabric. Path IS-IS. Individual link failure invisible to upper layer protocols. Also allows more bandwidth for branches of Multidestination trees With 4 member port channel, whole interface becomes single branch of tree with 40 G BW With 4 parallel ECMP paths, only one of the 4 interfaces becomes part of the tree ECMP with port-channel : 2 levels of load-balancing decision : First level : FP Core Link selection (based on L 3/L 4 fields by default) Second level : Port-Channel member selection (based on src-dst ip by default) © 2013 Cisco and/or its affiliates. All rights reserved. 33

Strong Recommendations and Key Notes • Do not use UDLD with Fabric. Path UDLD (normal or aggressive) does not bring any benefits on single physical link and port channels with FP enabled (for port channel, activate LACP instead of relying on UDLD to detect member port issues) Physical link level protection and the bi-directional IS-IS hellos should take care of all (or near all) potentially link level issue • HSRP preemption does not add any value but may hurt at large VLAN scale, when you need to maintain HSRP adjacency for each of the VLANs. Control plane will just be burning cycles with no efficient and positive impact on data path. Consider not using HSRP preemption in the Fabric. Path design. • Fabric. Path and Jumbo MTU Interoperability with N 5 k/N 6 k and N 7 k; its recommended to disable ISIS hello padding on N 7 k with the “no fabricpath isis hello-padding always” command when jumbo MTU is enabled © 2013 Cisco and/or its affiliates. All rights reserved. 34

Building Fabric. Path Routing Tables : : Control Plane Operation Fabric. Path Routing Table on S 10 Fabric. Path Routing Table on S 11 Switch IF S 10 - S 10 L 2, L 4, L 6 S 11 L 1, L 3, L 5 S 11 - S 100 L 1 S 100 L 2 S 101 L 3 S 101 L 4 S 140 L 5 S 140 L 6 Fabric. Path Routing Table on S 100 Fabric. Path Routing Table on S 140 Switch IF S 10 L 1 S 10 L 5 S 11 L 2 S 11 L 6 S 100 - S 100 L 5, L 6 S 101 L 1, L 2 S 101 L 5, L 6 S 140 L 1, L 2 S 140 - Step 1 : : Enable Fabric. Path on desired interfaces Step 2 : : L 2 IS-IS hello are sent out on all Fabric. Path Ports Step 3 : : Establish L 2 IS-IS Adjacency Step 4 : : Send L 2 IS-IS updates to exchange local link-states Step 5 : : All Fabric. Path switches calculate unicast paths to all other switches in the L 2 fabric and create the ‘Fabric. Path Routing Table’ based on the results © 2013 Cisco and/or its affiliates. All rights reserved. ü Forwarding path selection based on destination Switch-ID ü Switch Table basically contains (Switch-ID, Output Interface) ü Up to 16 ‘Next-Hop’ Interfaces (ECMP) per Switch-ID 35

FTAG/tree 1 handles unknown unicast, broadcast and some multicast Fabric. Path Forwarding : : Broadcast (ARP Request) Multidestination Trees on S 10 Tree IF 1 L 1, L 3, L 5 2 Root for Tree 1 Root for Tree 2 decap encap DSID→FF Ftag→ 1 L 5 SSID→ 100 ftag DMAC→FF SMAC→A Multidestination Trees on S 100 Tree IF 1 L 1, L 2 2 L 2 Payload DMAC→FF Tree IF SMAC→A 1 L 5 DMAC→FF 2 L 5, L 6 SMAC→A Payload Fabric. Path MAC Table on S 100 Switch IF A e 1/1 (local) © 2013 Cisco and/or its affiliates. All rights reserved. ftag Multidestination Trees on S 140 Payload Broadcast FTAG/tree 2 handles multicast only Step 1 : : Host A communicates to Host B for the first time – Sends ARP request to B Step 2 : : S 100 adds A into MAC table as the result of new source learning on CE port Step 3 : : Since destination MAC is all ‘F’; S 100 floods this frame out all CE ports [Learn MACs of directly-connected devices unconditionally] Step 4 : : Meanwhile, S 100 selects ‘Tree 1’, marks this in the Fabric. Path header and floods this frame out all Fabric. Path ports (L 1, L 2) that are part of Tree 1 Step 5 : : S 10 floods this frame further, out (L 3, L 5) based on local info about Tree 1 Step 6 : : S 101 and S 140 remove the Fabric. Path header and flood the frame out all local CE ports. Fabric. Path MAC Table on S 140 Switch IF Don’t Learn Remote MAC since DMAC is unknown / is a Flooded Frame 36

FTAG/tree 1 handles unknown unicast, broadcast and some multicast Fabric. Path Forwarding : : Unknown Unicast (ARP Reply) Multidestination Trees on S 10 Tree IF 1 L 1, L 3, L 5 2 Root for Tree 1 Root for Tree 2 encap decap DSID→MC 1 Ftag→ 1 L 5 SSID→ 140 ftag DMAC→A SMAC→B Multidestination Trees on S 100 Tree IF 1 L 1, L 2 2 L 2 Payload ftag Fabric. Path MAC Table on S 100 Switch IF A e 1/1 (local) B S 140 (remote) If DMAC is Known then Learn Remote MAC © 2013 Cisco and/or its affiliates. All rights reserved. FTAG/tree 2 handles multicast only Unknown Multidestination Trees on S 140 Payload DMAC→A SMAC→B DMAC→A Payload Step 1 : : Host B sends ARP Reply back to Host A Step 2 : : S 140 adds B into the MAC Table from source learning on CE port A Step 3 : : Since A is unknown, S 140 floods the frame out all CE ports Step 4 : : Meanwhile, S 140 selects Tree 1, marks this in the Fabric. Path header and floods this frame out all Fabric. Path ports (L 5) that are part of Tree 1 Step 5 : : S 10 floods this frame further (L 1, L 3) along Tree 1 Step 6 : : S 100 floods this frame further (L 2) along Tree 1. Also, upon removing the Fabric. Path header, S 100 finds host A was learned locally. Therefore adds B to the MAC Table as remote, associated with S 140 Tree IF 1 L 5 2 L 5, L 6 Fabric. Path MAC Table on S 140 Switch IF B e 2/2 (local) MAC A is Unknown 37

Fabric. Path Forwarding : : Known Unicast (Data) encap DSID→ 140 Ftag→ 1 SSID→ 100 DMAC→B SMAC→A Destination Switch ID is used to make routing decisions through the Fabric. Path core & no MAC learning or lookups required inside the FP core Fabric. Path Routing Table on S 11 Switch IF S 10 L 2, L 4, L 6 S 11 - DSID→ 140 Ftag→ 1 S 100 L 2 SSID→ 100 S 101 L 4 DMAC→B S 140 L 6 SMAC→A decap Payload Fabric. Path Routing Table on S 100 Switch Fabric. Path Routing Table on S 140 IF S 10 L 1 S 11 L 2 S 100 - S 101 L 1, L 2 S 140 L 1, L 2 Hash L 1, L 2 Switch IF Payload S 10 L 5 SMAC→A S 11 L 6 DMAC→B S 100 L 5, L 6 S 101 L 5, L 6 S 140 - DMAC→B SMAC→A Payload Fabric. Path MAC Table on S 100 Switch IF A e 1/1 (local) B S 140 (remote) © 2013 Cisco and/or its affiliates. All rights reserved. Step 1 : : Host A starts sending traffic to Host B after ARP resolution Step 2 : : S 100 finds B was learned as remote; associated with S 140, encap all subsequent frames to B with S 140 as destination in FP header Step 3 : : S 100 Routing Table indicates multiple paths to S 140; runs ECMP hash and this time S 100 selects L 2 as next-hop Step 4 : : Routing Table lookup at S 11 indicates L 6 as next hop for S 140 Step 5 : : S 140 finds itself as destination in Fabric. Path header and B is also known locally; decaps FP header, adds A as remote & associates with S 100 Fabric. Path MAC Table on S 140 Switch IF A S 100 (remote) B e 2/2 (local) 38

Fabric. Path Loop Mitigation TTL=1 TTL=2 When the frame is originally encapsulated, the system sets the TTL to 32; on each hop through the Fabric. Path network, each switch decrements the TTL by 1. If the TTL reaches 0, that frame is discarded. This feature prevents the continuation of any loops that may form in the network. TTL=3 TTL=0 Loop prevention and mitigation is available in the data plane, helping ensure safe forwarding unmatched by any transparent bridging technology. Cisco Fabric. Path frames include a time-to-live (TTL) field similar to the one used in IP, and an applied reverse-path forwarding (RPF) check for multicast based on ‘Tree’ information © 2013 Cisco and/or its affiliates. All rights reserved. 39

Mixed Chassis Mode : : Supported Topologies Interop F 2 & F 2 E VDC With NX-OS 6. 1 and. Prior Releases : : • Always use identical line cards on either side of the v. PC+ Peer Link, v. PC member ports, and Fabric. Path core member ports (legs to downstream device) • The F 1 -series line cards can mix with M-series line cards • The F 2 -series line cards have to be in their own VDC; VDC type [F 2] meaning they can’t mix with F 1 or the M-series in the same VDC © 2013 Cisco and/or its affiliates. All rights reserved. 40

Mixed Chassis Mode : : Supported Topologies Starting in NX-OS 6. 2 and Later Releases : : • VDC type [F 2, F 2 E, F 2 E] must match between the 2 v. PC+ peer devices when F 2 & F 2 E are used in same VDC; meaning its ok to have F 2 on v. PC peer device 1 and F 2 E on v. PC peer device 2 for the v. PC Peer Link, v. PC member ports, or Fabric. Path core member ports • Note: in a F 2 & F 2 E type of design; only features related to F 2 apply (lowest common denominator) • Always use identical line cards on either side of the v. PC Peer Link, v. PC member ports, and Fabric. Path core member ports when M 1, M 1 -XL, M 2 & F 2 E in same VDC [M-F 2 E] or system • When F 2 E is placed in a chassis with M-series it will operate in Layer 2 mode only leveraging the M for Layer 3 (proxy L 3 forwarding); this will provide 128 K MAC scale © 2013 Cisco and/or its affiliates. All rights reserved. 41

© 2013 Cisco and/or its affiliates. All rights reserved. 42

Additional Resources & Further Reading External (public) Cisco Fabric. Path Best Practices http: //www. cisco. com/en/US/prod/collateral/switches/ps 9441/ps 9402/white_paper_c 07 -728188. pdf Scale Data Centers with Cisco Fabric. Path http: //www. cisco. com/en/US/prod/collateral/switches/ps 9441/ps 9402/white_paper_c 11 -605488. html Cisco Fabric. Path for Cisco Nexus 7000 Series Switches http: //www. cisco. com/en/US/prod/collateral/switches/ps 9441/ps 9402/white_paper_c 11 -687554. html Great External Resources Nexus 7000/6000/5000 Configuration Guides http: //www. cisco. com/en/US/products/ps 9402/products_installation_and_configuration_guides_list. html http: //www. cisco. com/en/US/products/ps 9670/products_installation_and_configuration_guides_list. html http: //www. cisco. com/en/US/partner/products/ps 12806/products_installation_and_configuration_guides_list. html Fabric. Path Scaling limits http: //www. cisco. com/en/US/docs/switches/datacenter/sw/verified_scalability/b_Cisco_Nexus_7000_Series_NXOS_Verified_Scalability_Guide. html#reference_3 AD 0536 C 32 FF 4 B 499 A 0936409729951 D http: //www. cisco. com/en/US/docs/switches/datacenter/nexus 5500/sw/configuration_limits/b_N 5500_Config_Limits_602 N 11_ chapter_01. html © 2013 Cisco and/or its affiliates. All rights reserved. 43

Additional Resources & Further Reading Quick Start Guide : : Virtual Port Channel (v. PC) https: //communities. cisco. com/docs/DOC-35728 © 2013 Cisco and/or its affiliates. All rights reserved. 44

© 2013 Cisco and/or its affiliates. All rights reserved. 45