Port Aggregation Ether Channel CIS 187 Multilayer Switched

Port Aggregation (Ether. Channel) CIS 187 Multilayer Switched Networks CCNP SWITCH SUDHIR KUMAR CCSI #33688

Spanning Tree and Ether Channel Etherchannel Bundle �Spanning Tree only allows a single link between switches to prevent bridging loops. �Cisco’s Ether. Channel technology allows for the scaling of link bandwidth by aggregating or bundling parallel links. �Treated as a single, logical link. �Access or Trunk link �Allows you to expand the link’s capacity without having to purchase new hardware (modules, devices). 2

Ether Channel �Ether. Channel allows for two to eight links. �Fast Ethernet (FE) Fast Ether. Channel Up to 1600 Mbps �Gigabit Ethernet (GE) Gigabit Ether. Channel Up to 16 Gbps � 10 -Gigabit Ethernet (10 GE) 10 Gigabit Ether. Channel Up to 160 Gbps 3

Ether Channel �The Cisco Catalyst family of switches supports two types of link aggregation: �Port Aggregation Protocol (PAg. P) - Cisco proprietary � Default when port channel is created. �Link Aggregation Control Protocol (LACP) - Industry standard 802. 3 ad-based protocol �Ether. Channel provides redundancy. �If one link fails traffic is automatically moved to an active link. �Transparent to end user. �LACP also allows for standby links. 4

Fast Ethernet Full duplex Dot 1 q auto Native = VLAN 2 VLANs 1 thru 100 � The key is consistency for all links in the bundle: � Media � Same media type and speed � Same duplex � VLANs – All ports within the bundle must be configure with: � Same VLAN (if access) � Same trunking encapsulation and mode (if trunk) � Mode on opposite switches do not have to be the same as long as it still forms a trunk. � Same Native VLAN � Pass the same set of VLANs 5

Distribution of Traffic and Load Balancing �Load is not balanced equally across links. �Ether. Channel uses a hashing algorithm. �Single input is used (such as Source IP address), the hash will only look at the bits associated with this input. (coming) �Two inputs are used (such as Source IP address and Destination IP address), the hash will perform an exclusive OR (XOR) operation on both inputs. (coming!) �Both of these will compute a binary number that selects a link number in the bundle to carry the frame. (coming!!!) 6

Load Balancing � Let’s take a brief look at how this works. � We will focus on the 2, 4 and 8 link possibilities as this is easier to understand the only options that provide more ideal load balancing. � A 2 link Ether. Channel bundle requires a 1 -bit index using an XOR. � If the index is 0, link 0 is selected � If the index is 1, link 1 is selected � A 4 link Ether. Channel bundle requires a 2 -bit index using an XOR. � 4 possible links: 00, 01, 10, 11 � An 8 link Ether. Channel bundle requires a 3 -bit index using an XOR. � 8 possible links: 000, 001, 010, 011, 100, 101, 110, 111 7

Boolean Operations - XOR TRUE XOR False = TRUE � XOR (Exclusive OR) operation � 0 = FALSE and 1 = TRUE � If both bits have the same value (both 0, both 1), the XOR will result in a 0 � Otherwise, if they differ (one is a 0 and the other a 1) the result will be 1. � One and ONLY one input value can be TRUE for output to be TRUE 8

Boolean Operations – XOR Gate 0 0 0 1 1 9 0 1 Truth Table Inputs Output 0 0 1 1 1 0 1 0 0 = FALSE 1 = TRUE XOR operation � Only one input value is TRUE for output to be TRUE

Load Balancing 0 1 �Example: 2 Link Ether Channel. � Packet sent from 172. 16. 1. 1 to 10. 10. 46 � The chosen hash uses Source IP and Destination IP address �At most there can only be 8 links in bundle, so only the last 3 rightmost bits (least-significant) of the addresses will ever need to be indexed or examined. � 3 bits will give us 8 choices (8 links max in a bundle) � 172. 16. 1. 11 => 000000010 10. 10. 46 => 00101110 �In our example we have 2 links in the Ether Channel (1 bit index): 10 � The XOR is performed only on the rightmost bit 1 XOR 0 � 1 XOR 0 = 1 � Link 1 is used

�If XOR of the two bits result in 0, then link 0 is used. �If XOR of the two bits result in 1, then link 1 is used. 11

Load Balancing 0 3 �Example: 4 Link Ether Channel �Packet sent from 172. 16. 1. 1 to 10. 10. 46 �Our hash used the Source ip and Destination IP address � 172. 16. 1. 1=>00000001 10. 10. 46 => 00101110 �If there are 4 links in the Ether Channel (2 bit index): �The XOR is performed only on 2 rightmost bits 01 XOR 10 �Each bit is computed separately � 01 XOR 10 = 11 1 XOR 0 = 1 0 XOR 1 = 1 �Link 3 (112) is used 12

Load Balancing 0 7 �Example: 8 Link Ether Channel �Packet sent from 172. 16. 1. 1 to 10. 10. 46 �Our hash used the Source IP and Destination IP address � 172. 16. 1. 1 => 00000001 10. 10. 46 => 00101110 �If there are 8 links in the Ether Channel (3 bit index): �The XOR is performed only on the 3 rightmost bits 001 XOR 110 �Each bit is computed separately � 001 XOR 110 = 111 1 XOR 0 = 1 0 XOR 1 = 1 13 �Link 7 (1112) is used

For more information �For information about load balancing the number of links other than 2, 4 or 8: �Understanding Ether Channel Load Balancing and Redundancy on Catalyst Switches �http: //www. cisco. com/en/US/tech/tk 389/tk 213/technol ogies_tech_note 09186 a 0080094714. shtml 14

Configuring Ether. Channel

Configuring Ether Channel Load Balancing Switch(config)# port-channel load-balance method l The load balancing method is configured in global configuration mode. 16

Load Balancing Switch(config)# port-channel load-balance ? dst-ip Dst IP Addr bits dst-mac Dst Mac Addr bits src-dst-ip Src XOR Dst IP Addr XOR src-dst-mac Src XOR Dst Mac Addr XOR src-ip Src IP Addr bits src-mac Src Mac Addr bits Hash Operation default l 6500 and 4500 switches also allow hash input to be based on: dst-port (destination port) src-dst-port (source and destination ports) l Dafaults for 29 xx and 35 xx (this may vary so check documentation) Layer 2 switching (switched port) is src-mac (coming) Layer 3 switching (routed port) is src-dst-ip (coming) l For non-IP traffic the switch will distribute frames based on MAC addresses. l Multicasts and broadcasts sent over one link in the Ether. Channel are not sent back over other links in the Ether. Channel. 17

Load Balancing Switch(config)# port-channel load-balance ? dst-ip Dst IP Addr bits dst-mac Dst Mac Addr bits src-dst-ip Src XOR Dst IP Addr XOR src-dst-mac Src XOR Dst Mac Addr XOR src-ip Src IP Addr bits src-mac Src Mac Addr bits Switch(config)# port-channel load-balance src-dst-ip l Normally, the default Source IP and Destination IP addresses will result in a fair statistical distribution of frames. l This is because of the random nature of multiple Source and Destination IP addresses. l However, if a single server’s destination IP address is receiving most of the traffic this may cause one link to be overused in a two link Ether. Channel. Two links in a four link Ether. Channel Four links in an eight link Ether. Channel. l Use only Source IP address or include MAC addresses to create a more balanced load across the bundle. 18

Ether Channel Protocols PAg. P LACP �The Cisco Catalyst family of switches supports both: �Port Aggregation Protocol (PAg. P) - Cisco proprietary � Default when port channel is created (coming) �Link Aggregation Control Protocol (LACP) - Industry standard 802. 3 ad-based protocol �Not many differences. �When a Cisco switch is connected to a non-Cisco switch use LACP. �Must be the same on both ends! 19

Ether Channel Protocols Fa 0/1 Fa 0/4 DLS 1(config)# interface range fa 0/1 - 4 DLS 1(config-if-range)# channel-protocol ? lacp Prepare interface for LACP protocol pagp Prepare interface for PAg. P protocol DLS 1(config-if-range)# channel-protocol pagp l PAg. P requires identical static VLANs or trunking encapsulation with same allowed VLANs. l If the VLAN, speed or duplex on a port in the bundle is changed PAg. P automatically reconfigures the rest of the ports in that bundle. 20

Ether Channel Protocols Fa 0/1 Channel Group Fa 0/4 DLS 1(config)# interface range fa 0/1 - 4 DLS 1(config-if-range)# channel-protocol ? lacp Prepare interface for LACP protocol pagp Prepare interface for PAg. P protocol DLS 1(config-if-range)# channel-group number mode {active | on | {auto [non-silent]} | {desirable [non-silent]} | passive} l Channel-group number: 1 – 64 l Does not need to be the same on both switches but its recommended that it usually is. 21

No PAg. P or LACP negotiation Ether Channel on on DLS 1(config)# interface range fa 0/1 - 4 DLS 1(config-if-range)# channel-protocol pagp DLS 1(config-if-range)# channel-group 1 mode ? active Enable LACP unconditionally auto Enable PAg. P only if a PAg. P device is detected desirable Enable PAg. P unconditionally on Enable Ether channel only passive Enable LACP only if a LACP device is detected l on – Forces port to channel without PAg. P negotiation. Both ends must be on. All ports channeling l You can use channel-group # mode on when the connecting device does not support PAg. P and you need to set up the channel unconditionally. 22

PAg. P modes Ether. Channel desirable auto DLS 1(config)# interface range fa 0/1 - 4 DLS 1(config-if-range)# channel-protocol pagp DLS 1(config-if-range)# channel-group 1 mode ? active Enable LACP unconditionally auto Enable PAg. P only if a PAg. P device is detected desirable Enable PAg. P unconditionally on Enable Etherchannel only passive Enable LACP only if a LACP device is detected l An interface in desirable mode can form an Ether. Channel with another interface that is in desirable or auto mode. Desirable (Active) - Actively asks to form a channel 23

PAg. P modes Ether. Channel auto desirable DLS 1(config)# interface range fa 0/1 - 4 DLS 1(config-if-range)# channel-protocol pagp DLS 1(config-if-range)# channel-group 1 mode ? active Enable LACP unconditionally auto Enable PAg. P only if a PAg. P device is detected desirable Enable PAg. P unconditionally on Enable Ether. Channel only passive Enable LACP only if a LACP device is detected l An interface in auto mode can form an Ether. Channel with another interface in desirable mode. Auto (default, passive) - Waits to be asked to form a channel. l An interface in auto mode cannot form an Ether. Channel with another interface that is also in auto mode because neither interface starts PAg. P 24 negotiation.

PAg. P modes Ether. Channel DLS 1(config)# interface range fa 0/1 - 4 DLS 1(config-if-range)# channel-protocol pagp DLS 1(config-if-range)# channel-group 1 mode ? active Enable LACP unconditionally auto Enable PAg. P only if a PAg. P device is detected desirable Enable PAg. P unconditionally on Enable Ether. Channel only passive Enable LACP only if a LACP device is detected DLS 1(config-if-range)# channel-group 1 mode auto ? non-silent Start negotiation only after data packets received l l l l 25 By default PAg. P uses the silent submode for desirable and auto. If you expect a switch to be on the other end you should use non-silent. “Use the non-silent keyword when you connect to a device that transmits bridge protocol data units (BPDUs) or other traffic. ” “Use the silent keyword when you connect to a silent partner (which is a device that does not generate BPDUs or other traffic). ” Either will work between switches. For more information on when to use silent or non-silent: http: //www. cisco. com/en/US/tech/tk 389/tk 213/technologies_configuration_example 09186 a 0080094953. sh tml

LACP modes Ether. Channel active passive DLS 1(config)# interface range fa 0/1 - 4 DLS 1(config-if-range)# channel-protocol lacp DLS 1(config-if-range)# channel-group 1 mode ? active Enable LACP unconditionally auto Enable PAg. P only if a PAg. P device is detected desirable Enable PAg. P unconditionally on Enable Ether. Channel only passive Enable LACP only if a LACP device is detected l An interface in the active mode can form an Ether. Channel with another interface that is in the active or passive mode. 26

LACP modes Ether. Channel passive active DLS 1(config)# interface range fa 0/1 - 4 DLS 1(config-if-range)# channel-protocol lacp DLS 1(config-if-range)# channel-group 1 mode ? active Enable LACP unconditionally auto Enable PAg. P only if a PAg. P device is detected desirable Enable PAg. P unconditionally on Enable Etherchannel only passive Enable LACP only if a LACP device is detected l An interface in the passive mode can form an Ether. Channel with another interface that is in the active mode. l An interface in the passive mode cannot form an Ether. Channel with another interface that is also in the passive mode because neither interface starts LACP negotiation. 27

Forming Ether. Channel on on PAg. P Negotiated Ether. Channel desirable auto LACP Negotiated Ether. Channel active passive 28

Configuring PAg. P DLS 1(config)# port-channel load-balance dst-ip DLS 1(config)# interface range fa 0/11 - 12 DLS 1(config-if-range)# switchport trunk encapsulation dot 1 q DLS 1(config-if-range)# switchport mode trunk DLS 1(config-if-range)# channel-protocol pagp DLS 1(config-if-range)# channel-group 1 mode desirable DLS 2(config)# port-channel load-balance src-dst-ip DLS 2(config)# interface range fa 0/11 - 12 DLS 2(config-if-range)# switchport trunk encapsulation dot 1 q DLS 2(config-if-range)# channel-protocol pagp DLS 2(config-if-range)# channel-group 1 mode auto l Notice: Load balancing does not have to match but usually it does. DTP on DLS 2 is dyanmic auto (result is trunk with DLS 1) PAg. P configured on both ends 29

Verifying We will discuss the significance of the Port-channel interface with MLS. DLS 1#show run ! port-channel load-balance dst-ip ! interface Port-channel 1 switchport trunk encapsulation dot 1 q switchport mode trunk ! interface Fast. Ethernet 0/1 !. . . interface Fast. Ethernet 0/11 switchport trunk encapsulation dot 1 q switchport mode trunk channel-group 1 mode desirable ! interface Fast. Ethernet 0/12 switchport trunk encapsulation dot 1 q switchport mode trunk channel-group 1 mode desirable 30 DLS 2#show run ! port-channel load-balance src-dst-ip ! interface Port-channel 1 switchport trunk encapsulation dot 1 q ! ! interface Fast. Ethernet 0/1 !. . . interface Fast. Ethernet 0/11 switchport trunk encapsulation dot 1 q channel-group 1 mode auto ! ! interface Fast. Ethernet 0/12 switchport trunk encapsulation dot 1 q channel-group 1 mode auto

Verifying DLS 1# show Ether. Channel protocol Group: 1 -----Protocol: PAg. P DLS 1# show Ether. Channel load-balance Ether. Channel Load-Balancing Operational State (dst-ip): Non-IP: Destination MAC address IPv 4: Destination IP address IPv 6: Destination IP address DLS 1# 31 DLS 1(config)# port-channel load-balance dst-ip DLS 1(config)# interface range fa 0/11 - 12 DLS 1(config-if-range)# channel-protocol pagp DLS 1(config-if-range)# channel-group 1 mode desirable

DLS 1# show etherchannel summary Flags: D - down P - in port-channel I - stand-alone s - suspended H - Hot-standby (LACP only) R - Layer 3 S - Layer 2 U - in use f - failed to allocate aggregator u - unsuitable for bundling w - waiting to be aggregated d - default port Number of channel-groups in use: 1 Number of aggregators: 1 Group Port-channel Protocol Ports ------+-----------+---------------1 Po 1(SU) PAg. P Fa 0/11(P) Fa 0/12(P) DLS 1# 32 DLS 1(config)# port-channel load-balance dst-ip DLS 1(config)# interface range fa 0/11 - 12 DLS 1(config-if-range)# channel-protocol pagp DLS 1(config-if-range)# channel-group 1 mode desirable

DLS 1# show etherchannel port Group: 1 -----Port: Fa 0/11 ------Port state Channel group Port-channel Port index = = DLS 1(config)# port-channel load-balance dst-ip DLS 1(config)# interface range fa 0/11 - 12 DLS 1(config-if-range)# channel-protocol pagp DLS 1(config-if-range)# channel-group 1 mode desirable Up Mstr In-Bndl 1 Mode = Desirable-Sl Po 1 GC = 0 x 0001 0 Load = 0 x 00 Flags: S - Device is sending Slow hello. <output omitted> Timers: H - Hello timer is running. <output omitted> Flags State SC C - Device is in Consistent state. Q - Quit timer is running. Can help determine if the load balancing is being distributed equally across the links Local information: Port Fa 0/11 Gcchange = 0 Pseudo port-channel = Po 1 Protocol = PAg. P Timers Hello Partner PAg. P Interval Count Priority Learning Group Method Ifindex Partner's information: Port Fa 0/11 Partner Name DLS 2 Partner Device ID 001 b. 8 fc 8. 0080 Partner Port Age of the port in the current state: 00 d: 00 h: 35 m: 29 s Port: Fa 0/12 ------. . . 33 Age Partner Group Flags Cap.

Configuring LACP DLS 1(config)# port-channel load-balance dst-ip DLS 1(config)# lacp system-priority 11111 DLS 1(config)# interface range fa 0/11 - 12 DLS 1(config-if-range)# switchport trunk encapsulation dot 1 q DLS 1(config-if-range)# switchport mode trunk DLS 1(config-if-range)# channel-protocol lacp DLS 1(config-if-range)# channel-group 1 mode active DLS 1(config-if-range)# lacp port-priority 99 Fa 0/13 -14 has a higher port priority so these will become the standby links should something happen to any of the active links. DLS 1(config)# interface range fa 0/13 - 14 DLS 1(config-if-range)# switchport trunk encapsulation dot 1 q DLS 1(config-if-range)# switchport mode trunk Default port-priority = 32, 768 DLS 1(config-if-range)# channel-protocol lacp DLS 1(config-if-range)# channel-group 1 mode active l Port Priority - (Optional for LACP) LACP uses the port priority to decide which ports should be put in standby mode. Not typically used (more with hardware limitation). Ports with lower priority are active, rest are standby. (Default is 32, 768) l System Priority - (Optional for LACP) Valid values are 1 through 65535. Higher numbers have lower priority. (Default is 32768, switch MAC is tiebreaker) 34 Recommended only when some ports are in standby.

Configuring LACP: DLS 1 and DLS 2 DLS 1(config)# port-channel load-balance dst-ip DLS 1(config)# lacp system-priority 11111 DLS 1(config)# interface range fa 0/11 - 12 DLS 1(config-if-range)# switchport trunk encapsulation dot 1 q DLS 1(config-if-range)# switchport mode trunk DLS 1(config-if-range)# channel-protocol lacp DLS 1(config-if-range)# channel-group 1 mode active DLS 1(config-if-range)# lacp port-priority 99 DLS 1(config)# interface range fa 0/13 - 14 DLS 1(config-if-range)# switchport trunk encapsulation dot 1 q DLS 1(config-if-range)# switchport mode trunk DLS 1(config-if-range)# channel-protocol lacp DLS 1(config-if-range)# channel-group 1 mode active DLS 2(config)# port-channel load-balance src-dst-ip DLS 2(config)# interface range fa 0/11 - 12 DLS 2(config-if-range)# switchport trunk encapsulation dot 1 q DLS 2(config-if-range)# channel-protocol lacp DLS 2(config-if-range)# channel-group 1 mode passive DLS 1(config)# interface range fa 0/13 - 14 DLS 2(config-if-range)# switchport trunk encapsulation dot 1 q DLS 2(config-if-range)# switchport mode trunk DLS 2(config-if-range)# channel-protocol lacp DLS 2(config-if-range)# channel-group 1 mode active 35

Verifying (only showing DLS 1) DLS 1#show run ! port-channel load-balance dst-ip ! interface Port-channel 1 switchport trunk encapsulation dot 1 q switchport mode trunk ! interface Fast. Ethernet 0/11 switchport trunk encapsulation dot 1 q switchport mode trunk lacp port-priority 99 channel-group 1 mode active ! interface Fast. Ethernet 0/12 switchport trunk encapsulation dot 1 q switchport mode trunk lacp port-priority 99 channel-group 1 mode active ! 36 interface Fast. Ethernet 0/13 switchport trunk encapsulation dot 1 q switchport mode trunk channel-group 1 mode active ! interface Fast. Ethernet 0/14 switchport trunk encapsulation dot 1 q switchport mode trunk channel-group 1 mode active

Verifying DLS 1# show etherchannel protocol Group: 1 -----Protocol: LACP DLS 1# show etherchannel load-balance Ether. Channel Load-Balancing Operational State (dst-ip): Non-IP: Destination MAC address IPv 4: Destination IP address IPv 6: Destination IP address DLS 1# 37 DLS 1(config)# port-channel load-balance dst-ip DLS 1(config)# interface range fa 0/11 - 12 DLS 1(config-if-range)# switchport trunk encapsulation dot 1 q DLS 1(config-if-range)# switchport mode trunk DLS 1(config-if-range)# channel-protocol lacp DLS 1(config-if-range)# channel-group 1 mode active DLS 1(config-if-range)# lacp port-priority 99 <output imitted>

Verifying DLS 1# show etherchannel summary Flags: D - down P - in port-channel I - stand-alone s - suspended H - Hot-standby (LACP only) R - Layer 3 S - Layer 2 U - in use f - failed to allocate aggregator u - unsuitable for bundling w - waiting to be aggregated d - default port Number of channel-groups in use: 1 Number of aggregators: 1 Group Port-channel Protocol Ports ------+-----------+--------------------1 Po 1(SU) LACP Fa 0/11(P) Fa 0/12(P) Fa 0/13(HH) Fa 0/14(H) DLS 1# 38

Odds and Ends (FYI) �Trunk ports send and receive PAg. P and LACP protocol data units (PDUs) on the lowest numbered VLAN. �Spanning tree sends packets over the first interface in the Ether. Channel. �For more information on Configuring Ether. Channel �http: //www. cisco. com/en/US/docs/switches/lan/cataly st 3550/software/release/12. 1_13_ea 1/configuration/guide /swethchl. html 39