Label scalability of Carrier Ethernet Benchmarking Carrier Ethernet

Label scalability of Carrier Ethernet Benchmarking Carrier Ethernet Technologies Workshop Session AI. 2 - Scientific and Technical Results Euro. NGI 2008, Krakow, Poland April 30, 2008 Wouter Tavernier, Koen Casier (Ghent University – IBBT) Luis Caro (University of Girona) Dimitri Papadimitriou (Alcatel-Lucent Bell NV) Vakgroep Informatietechnologie – Onderzoeksgroep IBCN

Agenda n PROBLEM STATEMENT n Labels in Carrier Ethernet l l Label scalability Label optimization n Results n Short word on label lookups n Conclusion Label scalability & efficiency of Carrier Ethernet – Wouter Tavernier Vakgroep Informatietechnologie – Onderzoeksgroep IBCN

Problem statement n Evolution of Ethernet being a LAN technology towards a Carrier Technology l l l Ethernet = low cost Ethernet = ubiquitous Ethernet = plug & play n LAN environments: traffic streams between tens of end-users Metro/Access environments: traffic streams between thousands of end-users n PROBLEM STATEMENT: n l Is Carrier Ethernet able to cope with increasing number of traffic streams LAN Label scalability & efficiency of Carrier Ethernet – Wouter Tavernier Vakgroep Informatietechnologie – Onderzoeksgroep IBCN Metro network L

No labels in bridged Ethernet n Forwarding in native Ethernet bridging : stateless (CL) 48 bit MAC-address based Sync 7 SD 1 DA 6 SA 6 Eth. Type 2 Payload 46 - 1500 A 2 5 Z C 1 2 4 Z C Dest B X Z C Out Port 2 5 5 3 Dest X Z A Label scalability & efficiency of Carrier Ethernet – Wouter Tavernier Vakgroep Informatietechnologie – Onderzoeksgroep IBCN Out Port 1 3 4 Z C X Z C 3 3 B C 1 Z C Y Z

Labels in Carrier Ethernet n The concept of a connection (LSP) allows for (CO): l l l n Traffic Engineering Advanced recovery/protection techniques BW guarantees Forwarding is based on a label linked to the connection state Two connections (LSPs) from A A to to E: E maintains state for the green and red connection and forwards based on a label D E A Labell B C Label scalability & efficiency of Carrier Ethernet – Wouter Tavernier Vakgroep Informatietechnologie – Onderzoeksgroep IBCN

Domain-wide PBB-TE label Ingress A has 2 LSPs to E, for e. g. : n Protection switching n Load balancing Labelremains constantalong connection(no SWAP)! D 00 00 E, 2 E 0, 0 <1 A SWAP)! E, 2 00 A> <2 E, 2 00 Distinct routes for dest E, by diff B-VID. , E > 0, <10 B C B-VID 100 reused on same link Label scalability & efficiency of Carrier Ethernet – Wouter Tavernier Vakgroep Informatietechnologie – Onderzoeksgroep IBCN E>

Link local ELS label Ingress A has 2 LSPs to E, for e. g. : n Protection switching n Load balancing D 100 E 20 0 A S-VID 100 reused B-VID on other same link 10 0 Distinct routes for dest E, by diff S-VID. Label can be Label remains swapped inalong constant intermediate hops connection (no along connection! SWAP)! B Label scalability & efficiency of Carrier Ethernet – Wouter Tavernier Vakgroep Informatietechnologie – Onderzoeksgroep IBCN C

Label usage optimization n Label balancing (online routing optimized ELS) l n Merging (ELS) l l n Shortest path routing takes into account labels used on a link as cost 1 incoming label per interface + 1 outgoing label GAIN: x-1 labels on outgoing local interface Shared forwarding (PBB-TE) l l 1 label needed GAIN: x-1 global labels . . x. incoming Label scalability & efficiency of Carrier Ethernet – Wouter Tavernier Vakgroep Informatietechnologie – Onderzoeksgroep IBCN

Label scalability n Given the label: ELS PBB-TE n How do different label schemes react under changing conditions? l l l What is the influence of the topology? What is the influence the traffic matrix? What is the influence of the traffic matrix on typical BW usage? Label scalability & efficiency of Carrier Ethernet – Wouter Tavernier Vakgroep Informatietechnologie – Onderzoeksgroep IBCN

Study assumptions n Study the impact on label usage of the following dimensions: l l l n Topology (CONNECTEDNESS) Traffic matrix (SIZE, UNIFORMITY) BW usage NORMALIZATION: l l Tests have run 100 times Only one dimension has been changed in a time n Shortest Path Routing (Dijkstra hop count) is taken as base routing algorithm n Base topology of 100 nodes (connectedness +-2) & 1000 demands ii x B i a b A 1 2 i 10 3 4 -9 ii L Label scalability & efficiency of Carrier Ethernet – Wouter Tavernier Vakgroep Informatietechnologie – Onderzoeksgroep IBCN C D E L

Connectedness of a network n Connectedness of a topology: what is the average node degree of a node in a topology: ring vs. full mesh n Example: l l l Single (un-)connected 4 -connected Full mesh Label scalability & efficiency of Carrier Ethernet – Wouter Tavernier Vakgroep Informatietechnologie – Onderzoeksgroep IBCN

Connectedness of the topology S Label scalability & efficiency of Carrier Ethernet – Wouter Tavernier Vakgroep Informatietechnologie – Onderzoeksgroep IBCN

Connectedness vs BW (Luis) 1. The topology generator considered for generating the topologies in this section is IGEN 2. For all the topologies the link capacity is set to 10 Gb/s and bandwidth request of 100 Mb, 200 Mb and 300 Mb are generated until the network is overloaded. 3. The implemented algorithm is the SPF 4. Overload network with links of 10 G capacity Label scalability & efficiency of Carrier Ethernet – Wouter Tavernier Vakgroep Informatietechnologie – Onderzoeksgroep IBCN A

Connectedness vs. BW (Luis) Label scalability & efficiency of Carrier Ethernet – Wouter Tavernier Vakgroep Informatietechnologie – Onderzoeksgroep IBCN

Connectedness vs. BW (Luis) Label scalability & efficiency of Carrier Ethernet – Wouter Tavernier Vakgroep Informatietechnologie – Onderzoeksgroep IBCN

Topology size vs. BW (Luis) Label scalability & efficiency of Carrier Ethernet – Wouter Tavernier Vakgroep Informatietechnologie – Onderzoeksgroep IBCN

Topology size vs. BW (Luis) Label scalability & efficiency of Carrier Ethernet – Wouter Tavernier Vakgroep Informatietechnologie – Onderzoeksgroep IBCN

Uniformity of the traffic matrix n UNIFORMITY: X demands directed to 1 vs. x destinations? n Example: l l 4 demands 2 destinations vs 4 destinations 4 paths / 1 dest 4 paths / 2 dest Label scalability & efficiency of Carrier Ethernet – Wouter Tavernier Vakgroep Informatietechnologie – Onderzoeksgroep IBCN 4 paths / 4 dest

Uniformity of the traffic matrix S Label scalability & efficiency of Carrier Ethernet – Wouter Tavernier Vakgroep Informatietechnologie – Onderzoeksgroep IBCN

Size of the traffic matrix n The size of a traffic matrix affects the number of demands that are routed over a network: l l l 10 demands 10000 demands … Label scalability & efficiency of Carrier Ethernet – Wouter Tavernier Vakgroep Informatietechnologie – Onderzoeksgroep IBCN

Scaling the order of the traffic S Label scalability & efficiency of Carrier Ethernet – Wouter Tavernier Vakgroep Informatietechnologie – Onderzoeksgroep IBCN

Lookups Must be Fast Year Aggregate Line Arriving rate of 40 B -rate POS packets (Million pkts/sec) 1997 622 Mb/s 1. 56 1999 2. 5 Gb/s 6. 25 2001 10 Gb/s 25 2003 40 Gb/s 100 2006 80 Gb/s 200 1. 2. Lookup mechanism must be simple and easy to implement Memory access time is the bottleneck 200 Mpps × 2 lookups/pkt = 400 Mlookups/sec → 2. 5 ns per lookup Label scalability & efficiency of Carrier Ethernet – Wouter Tavernier Vakgroep Informatietechnologie – Onderzoeksgroep IBCN

Direct lookup in ELS vs PBB-TE Direct Memory Lookup Memory Data • ELS: Address SVID/MPLS-label (Outgoing Port, new SVID label) • PBB-TE: • Label space is 12 bits • With 64 b data, this is 256 K of memory. • Label space is private to one link • Therefore, table size can be “negotiated” Label scalability & efficiency of Carrier Ethernet – Wouter Tavernier Vakgroep Informatietechnologie – Onderzoeksgroep IBCN • Label space is 48+12 bits • With 64 b data, this is 64 Ei. B of memory (2^60). • Label space is global • 2^60 > 2^12, therefore cannot hold all addresses in table and use direct lookup, less efficient alternatives: • Hashing • Binary/Multi-way Search Trie/Tree

Conclusion n The specific traffic matrix & topology used, clearly affect label usage l l The uniformity of the traffic matrix affects PBB-TE-type domain-wide labelling: more uniform is better The topology connectedness affects ELS-type link-local labelling: more connected is better n In typical metro-network with low node-degree, PBB-TE & ELS LL have similar performance n PBB-TE with SF and ELS with merging consistently score better than alternatives n Node local labelling techniques consistently score worse than alternatives. n The label length affects the memory space needed and accordingly affects cost and lookup speed Label scalability & efficiency of Carrier Ethernet – Wouter Tavernier Vakgroep Informatietechnologie – Onderzoeksgroep IBCN

Appendices Vakgroep Informatietechnologie – Onderzoeksgroep IBCN

Carrier-Grade Ethernet challenges n Flat address space scalability of number of MAC addresses to be learned l n Scalability in terms of number of VLANs l n n n Beyond 100 k learned addresses per node seems challenging 12 -bit VLAN ID (VID): 4 k VLANs possible network wide STP cannot converge faster than worst case 20 s sec (root failure) Traffic Engineering (routing) constraint by tree based structure. Limited OA&M Label scalability & efficiency of Carrier Ethernet – Wouter Tavernier Vakgroep Informatietechnologie – Onderzoeksgroep IBCN

Reference network x B ii i a b A 1 2 i 10 3 4 -9 ii L Label scalability & efficiency of Carrier Ethernet – Wouter Tavernier Vakgroep Informatietechnologie – Onderzoeksgroep IBCN C D E

Assumptions (Luis) § Results are evaluated in terms of the maximum and average number of: l l B For ELS (average calculated based on the number of links): § Labels per link with agg (meaning only paths are count) § Labels per link with agg and label merging For PBB-TE § Labels per destination with agg (meaning only paths are count) § Labels per destination with VLAN-reut (meaning link disjoint paths can use same label) § Labels per destination with INV-trees (paths that intersect only on a common segment ending at the destination can also use the same label) Label scalability & efficiency of Carrier Ethernet – Wouter Tavernier Vakgroep Informatietechnologie – Onderzoeksgroep IBCN

Tests on smaller networks not normalized Vakgroep Informatietechnologie – Onderzoeksgroep IBCN

Uniformity in small 3 -connected network S Label scalability & efficiency of Carrier Ethernet – Wouter Tavernier Vakgroep Informatietechnologie – Onderzoeksgroep IBCN

Connectedness in 28 n network S Label scalability & efficiency of Carrier Ethernet – Wouter Tavernier Vakgroep Informatietechnologie – Onderzoeksgroep IBCN

Traffic scaling in small network S Label scalability & efficiency of Carrier Ethernet – Wouter Tavernier Vakgroep Informatietechnologie – Onderzoeksgroep IBCN