A novel TCAM management for IPv 6 routing

A novel TCAM management for IPv 6 routing table update 一個新的為TCAM設計的 IPv 6路由表更新演算法 Design of High Performance Internet Routers (高效能網際網路路由器設計)

Outline l Introduction l Related Works Ø PLO_OPT Ø CAO_OPT l Proposed Scheme l Performance Evaluation l Conclusion 2

Introduction(1/5) l Router Ø The primary function of routers is to forward packets toward their final destinations. Router packet l What is IP address lookup? routing table action Ø In order to find the destination of each packet, the packet’s destination address lookuped in routing table. Ø IP lookup speed is the major bottlenecks faced by internet routers. 3

Introduction(2/5) Longest prefix match problem. 1 2 0 Router 208. 12. 21. 20 3 4 Routing table 208. 12. 21. 20 Longest match A prefix is an aggregation of IP addresses: 208. 12. 21/24 = {208. 12. 21. 0, … , 208. 12. 21. 255} 4

Introduction(3/5) l What is TCAM? Ø Ternary content-addressable memory (TCAM) is widely used as lookup engine for commercial router or network device Ø Two major properties leading TCAM to find the longest match quickly : 1. TCAM allows a fully parallel search of the routing table. 2. Each cell in a TCAM can take three logic states: “ 0”, “ 1”, or don’t-care “X”. 5

Three TCAM organizations 6

Introduction(4/5) l Longest prefix match using TCAM Case : Search key is an a prefix IP address TCAM Global mask Memory location 0 1 1 0 X 1 1 1 01 01 0: disable 1: enable Prefix SRAM 0 x 000 1 0 0 1 1 1 0 0 x 000 Port A len 0 0 x 001 0 1 1 0 0 X 01 0 x 001 Port B len 1 0 x 010 0 1 1 0 X X 1 0 x 011 0 1 1 X X X 0 x 100 1 0 0 X X X 0 x 101 1 0 1 X X X 1 Priority Encoder 0 x 010 0 x 001 0 x 010 Port C len 2 0 x 011 Port D len 3 0 0 x 100 Port C len 4 0 0 x 101 Port B len 5 len 1= 5 Port C Decreasing Length 7

Introduction(5/5) l In order to ensure the LPM is correct, the prefix length ordering in TCAM should always be maintained when update is taking place. TCAM 1 32 -bit prefixes 31 -bit prefixes 30 -bit prefixes . . . 9 -bit prefixes M-1 8 -bit prefixes Free space Prefix length ordering in TCAM 8

Outline l Introduction l Related Works Ø PLO_OPT Ø CAO_OPT l Proposed Scheme l Performance Evaluation l Conclusion 9

Related Work (1/7) l Update Scheme • Prefix-length ordering (PLO) constraint: Two prefixes of the same length don’t need to be in any specific order. • • PLO_OPT Chain-ancestor ordering (CAO) constraint There’s an ordering constraint between two prefixes if and only if one is a prefix of the other (enclosure). • CAO_OPT 10

Related Work (2/7) l PLO_OPT • Divide all prefixes into different groups by length. • Two prefixes in the same group can be in any order. • • 1 31 -bit prefixes. . . 21 -bit prefixes free space keep all the unused entries in the center of the TCAM. 20 -bit prefixes. . . The worst-case number of memory operations per update is L/2. L is the number of distinct lengths 32 -bit prefixes 24 9 -bit prefixes 25 8 -bit prefixes 11

Related Work (3/7) PLO_OPT Deletion Insertion 1 32 -bit prefixes 31 -bit prefixes. . . Prefix: 208. 12. 82/20 Prefix: 208. 12. 63. 82/32 21 -bit prefixes Free space end startboundary+1 +1 free space 20 -bit prefixes. . . 24 9 -bit prefixes 25 8 -bit prefixes 12

Related Work (4/9) l CAO_OPT • PLO constraint is more restrictive, the constraint can be relaxed to only overlapping prefixes. Q 3 Q 2 Q 1 free space Q 2 Q 4 Q 3 Q 4 Q 1 Prefixes that lie on the same chain of the trie need to be ordered. 13

Related Work (5/9) l CAO_OPT • • • A logical inverted trie can be superimposed on the prefixes stored in the TCAM. Only prefixes in the same path have ordering constraint. The CAO_OPT algorithm also keeps the empty space in the center of the TCAM. • For every prefix, the longest chain that this prefix belongs to should be split around the empty space as equally as possible. • The worst-case number of memory operations per update is D/2. where D is the max length of any chain in the trie. Q 3 Maximal chain Q 2 free space Q 4 Q 1 14

Related Work (6/9) l Notation statement • • LC(p) len(LC(p)) rootpath(p) ancestors of p children of p hcld(p) HCN(p) Children of p hcld(p) free space LC(p) p Ancestors of p 15

Related Work (7/9) CAO_OPT Insertion Case 2: Case 1: Insert q here When the prefix to be inserted is below above the free space LC(q) Moving prefixes on LC(q) HCN(q) Insert q here 16

Related Work (8/9) CAO_OPT Deletion It works on the chain that has prefix p adjacent to the free space q delete 17

Related Work (9/9) l Shortcoming to shift above schemes for IPv 6 Ø PLO: • There are 128 different length of prefix in IPv 6. Therefore, the worst case and average case of cost to shift the prefix stored in TCAM growth extremely. Ø CAO: • CAO need to maintain the additional trie structure using SRAM. Each update cost O(L) time to modify the data store in the trie. • In order to reorder the chain sequence, the router needs to hold-on for prefix update. It will cause packet drop rate increase when needing more memory access times. 18

Outline l Introduction l Related Works Ø PLO_OPT Ø CAO_OPT l Proposed Scheme l Performance Evaluation l Conclusion 19

Proposed Scheme (1/13) l Our method using level to group prefixes l Define level TCAM B 1 C H A H F G D A Level 1 D E E G H A B Level 4 C Level 3 G F Level 2 D E Level 1 F 8 Level 2 C Level 3 B Level 4 20

Proposed Scheme (2/13) l Three important properties of the prefix enclosure relationship • All the prefixes in the same level are disjoint. • The prefix containing any one of the level i prefixes must be stored in level (i+1) or higher level. • The highest level in IPv 4 and IPv 6 routing table is 7 and 11. 21

Proposed Scheme (3/13) l TCAM Other details about our method 1. 1 We put free space between level 1 and level 2 • Level 1 Free space More than 98% prefixes stored in level 1 and level 2. Level 2 AS 2. 0 AS 1221 AS 6447 V 6 Gene 1 V 6 Gene 2 Level 1 2808 867 2870 115709 123279 Level 2 184 62 205 2875 2770 Level 3 8 3 13 1048 794 Level 4 1 1 2 563 428 Level 5 0 0 0 269 183 Level 6 0 0 0 169 87 Level 7 0 0 0 130 71 Level 8 0 0 0 57 39 Level 9 0 0 0 23 27 Level 10 0 12 11 Level 11 0 0 0 6 8 . . M+1 Level M 22

Proposed Scheme (4/13) 2. We adds additional bits named “Level index” in TCAM. To avoid spending cost to maintain the trie structure. • 3. The length of level index is log 2 L bits 1 TCAM Level index Level 1 1 Free space 0 Level 2 2 Level M M . . M+1 Every level is associated with a link list called free_list to record its empty entries of this level. The free list of level 3 pos pos NULL 23

Proposed Scheme (5/13) l Initial 3 -bits level index+1 index A : 11** B C D 1***001 1***010 11**001 11**010 1***011 1100001 1101 H A B Level 4 C Level 3 G F Level 2 D E Level 1 TCAM Level index 0 x 0000 0 x 0001 (A)1100 free 000 001 (D)1101 free 000 001 0 x 0010 (E)1110 free 000 001 0 x 0011 (H)1001 free 000 001 0 x 0100 0 x 0101 free 000 0 x 0110 (G)100 X free 000 010 0 x 0111 0 x 1000 (F)111 X free 000 010 (C)11 XX free 000 011 Level 23 0 x 1001 (B)1 XXX (C)11 XX free 000 011 100 Level 342 Level 1 B cover A Disjoint C C Level 2 24

Proposed Scheme (6/13) l Insertion • 128 -bits IPv 6 prefixes Case 1: no ancestor, n children No action Ex 1: 1 child Layer-1 prefixes C 1 Log 2 N-bits index 1 Free space Insert q If free_list[2] != == NULL Insert q Layer-2 prefixes 2 Layer-3 prefixes 3 Layer-4 prefixes 4 q C 1 Layer-N-1 prefixes Layer-N prefixes N-1 N 25

Proposed Scheme (7/13) • 128 -bits IPv 6 prefixes Case 1: no ancestor, n children No action Ex 2: 3 children Layer-1 prefixes C 3 Log 2 N-bits index 1 Free space No action q If free_list[4] == != NULL C 1 No action Insert q Layer-2 prefixes Layer-3 prefixes Layer-4 prefixes C 2 C 1 2 3 4 C 2 C 3 Layer-N-1 prefixes Layer-N prefixes N-1 N 26

Proposed Scheme (8/13) • Case 2: m ancestors, no children Ex 1: 1 ancestors 128 -bits IPv 6 prefixes Layer-1 prefixes Replace A 1 by q Log 2 N-bits index 1 Find A 1 Free space Insert A 1 If free_list[2] == != NULL Insert A 1 q Layer-2 prefixes 2 Layer-3 prefixes 3 Layer-4 prefixes 4 Layer-N-1 prefixes N-1 Layer-N prefixes 27

Proposed Scheme (9/13) • Case 2: m ancestors, no children Ex 2: 3 ancestors 128 -bits IPv 6 prefixes Replace A 3 by q Layer-1 prefixes Log 2 N-bits index 1 Find A 3 Free space A 3 A 2 A 1 Insert A 1 A 2 If free_list[4] == != NULL Insert A 1 Layer-2 prefixes Find A 2 Find A 1 Layer-3 prefixes Layer-4 prefixes 2 3 4 A 3 q Layer-N-1 prefixes Layer-N prefixes N-1 N 28

Proposed Scheme (10/13) • 128 -bits IPv 6 prefixes Case 3: m ancestors, n children Ex 1: N-3 ancestors, 2 children No action Log 2 N-bits index Layer-1 prefixes C 2 1 Free space No action AN-3 An-2 A 1 C 1 q instead A 1 instead A 2 instead A 3 Layer-2 prefixes C 1 2 Layer-3 prefixes Find A 1 3 Layer-4 prefixes 4 Layer-5 prefixes 5 q C 2 AN-4 instead AN-3 Insert AN-3 If free_list[N] == != NULLInsert AN-3 Layer-N-1 prefixes Layer-N prefixes N-1 N 29

Proposed Scheme (11/13) l Deletion • The free list of level 4 Case 1: no parent Ex: Delete (B) 1 XXX TCAM Level index 0 x 0000 0 x 0001 (A)1100 001 (D)1101 0 x 0010 (E)1110 001 0 x 0011 (H)1001 0 x 0100 0 x 0101 free 000 0 x 0110 (G)110 X 010 0 x 0111 0 x 1000 (F)111 X 010 (C)11 XX 011 Level 3 0 x 1001 (B)1 XXX 100 Level 4 0 x 1001 Exactly Match Level 2 B Level 4 C Level 3 G F Level 2 D E Level 1 No parent, delete simply H NULL Level 1 A 30

Proposed Scheme (12/13) • Case 2: has parent, has sibling The free (C) list of level 2 (C)’s Difference set Finding sibling Ex: Delete (G) Finding parent level index-1 index+1 110 X 011 110 X 111 X 010 TCAM Level index 0 x 0000 0 x 0001 (A)1100 001 (D)1101 0 x 0010 (E)1110 001 0 x 0011 (H)100 X 001 0 x 0100 0 x 0101 free 000 0 x 0110 (G)110 X 010 0 x 0111 0 x 1000 (F)111 X (C)11 XX 0 x 1001 Empty entry S P 010 (G) (F) 0 x 0110 NULL Level 1 Exactly Has parent sibling Match Level 2 011 Level 3 100 Level 4 has sibling, delete simply H A C Level 3 G F Level 2 D E Level 1 31

Proposed Scheme (13/13) • Case 3: has parent, no sibling Ex: Delete (E) Finding grandparent sibling parent level index+1 index-1 1110010 1111001 111 X 010 111 X 011 111 X The free list of level 3 TCAM Level index 0 x 1000 0 x 0001 (A)1100 0001 (D)1101 0001 0 x 0010 (F)111 X (E)1110 0001 0 x 0011 (H)100 X 0001 0 x 0100 0 x 0101 free 0000 0 x 0110 Empty entry 0010 0 x 0111 0 x 1000 P (C)11 XX Empty entry GP 0010 0011 Level 3 0 x 1001 Empty entry 0100 Level 4 (F)111 X (C)11 XX NULL Level 1 Noinstead Exactly F Has C instead sibling parent grandparent match EF Level 2 Level 4 H A D C Level 3 F Level 2 E Level 1 32

Outline l Introduction l Related Works Ø PLO_OPT Ø CAO_OPT l Proposed Scheme l Performance Evaluation l Conclusion 33

Performance Evaluation (1/10) l We first collect IPv 6 routing table AS 2. 0, AS 1221, AS 6447 from http: //bgp. potaroo. net/. l We also use IPv 6 prefix generator from [1] to generate two routing tables named V 6 Gene 1, V 6 Gene 2. [1] Kai Zheng, Bin Liu, “V 6 Gene: AScalable IPv 6 Prefix Generaotr for Route Lookup Algorithm Benchmark” National Cheng Kung University CSIE Computer & Internet Architecture Lab 34

Performance Evaluation (2/10) l Prefixes’ distribution 35

Performance Evaluation (3/10) l Worst case analysis in theory • PLO_OPT : �L/2� In IPv 6, the min. length is 16 and the max. length is 128. =>�(128 -15)/2�= 57. • CAO_OPT : �D/2� In IPv 6, the D is less than 11. =>� 11/2�= 6. • Proposed scheme : 1+2*(D-2) =>1+2*(11 -2)=19. 36

Performance Evaluation (4/10) l Performance of insertion PLO_OPT CAO_OPT Proposed # of Lookup # of Write AS 2. 0 0 7. 204 0 1. 206 1. 068 1. 029 AS 1221 0 5. 426 0 1. 189 1. 076 1. 039 AS 6447 0 7. 498 0 1. 254 1. 077 1. 037 V 6 Gene 1 0 10. 837 0 1. 407 1. 084 1. 079 V 6 Gene 2 0 10. 560 0 1. 368 1. 062 1. 057 Worst case AS 2. 0 0 22 0 3 4 6 AS 1221 0 18 0 2 4 5 AS 6447 0 22 0 3 4 4 V 6 Gene 1 0 25 0 6 11 19 V 6 Gene 2 0 25 0 5 11 18 37

Performance Evaluation (5/10) l Performance of deletion PLO_OPT CAO_OPT Proposed # of Lookup # of Write AS 2. 0 1 9. 220 0 1. 164 3. 689 1. 064 AS 1221 1 7. 558 0 1. 144 3. 787 1. 044 AS 6447 1 9. 709 0 1. 236 3. 277 1. 058 V 6 Gene 1 1 11. 915 0 1. 371 2. 991 1. 064 V 6 Gene 2 1 11. 474 0 1. 363 2. 762 1. 053 Worst case AS 2. 0 1 17 0 3 20 3 AS 1221 1 13 0 3 36 4 AS 6447 1 11 0 3 22 3 V 6 Gene 1 1 25 0 5 50 9 V 6 Gene 2 1 26 0 5 86 11 National Cheng Kung University CSIE Computer & Internet Architecture Lab 38

Performance Evaluation (6/10) l To quantify the performance, We introduce the following TCAM hardware functions: 1. The TCAM search process is locked during the update process. 2. The TCAM memory width is 144 bits. 3. The TCAM has a sustained search rate of 360 million lookups per second. 4. Writing a prefix into TCAM takes three clock cycle. 5. Disable takes one clock cycle. 39

Performance Evaluation (7/10) Average clock cycle per insertion operation. 35 30 # of clock cycle 25 20 15 10 5 0 PLO_OPT CAO_OPT Proposed AS 2. 0 21. 612 3. 618 4. 114999999 AS 1221 16. 278 3. 567 4. 193 AS 6447 22. 467 3. 762 4. 188 V 6 Gene 1 32. 511 4. 221 4. 321 V 6 Gene 2 31. 68 4. 104 4. 233 40

Performance Evaluation (8/10) Average clock cycle per deletionion operation. 40 35 # of clok cycle 30 25 20 15 10 5 0 PLO_OPT CAO_OPT Proposed AS 2. 0 28. 66 3. 492 6. 881 AS 1221 23. 674 3. 432 6. 919 AS 6447 30. 127 3. 708 6. 451 V 6 Gene 1 36. 745 4. 113 6. 183 V 6 Gene 2 35. 422 4. 089 5. 921 41

Performance Evaluation (9/10) l The comparison of clock cycle per second between Search and update Operation Search Update # of operation per second 320, 000 2, 000 # of clock cycle per operation 1 Total clock cycle per second 320, 000 Lookup Write 1 3 4374 6312 42

Performance Evaluation (10/10) l Memory cost AS 2. 0 AS 1221 AS 6447 V 6 Gene 1 V 6 Gene 2 SRAM cost of CAO_OPT (Mbyte) 93. 813 29. 186 96. 594 3777. 027 3990. 375 SRAM cost of Proposed scheme (Mbyte) 0. 031 0. 059 43

Outline l Introduction l Related Works Ø PLO_OPT Ø CAO_OPT l Proposed Scheme l Performance Evaluation l Conclusion 44

Conclusion l We proposed an algorithm to manage the TCAM such that incremental update times remain small in average case. l This algorithm can update TCAM without maintain a tries structure. l This scheme can update the TCAM with less SRAM cost and less memory access time. 45

Thanks for your attention. 46