Pipelined van Emde Boas Tree Algorithms Analysis and

Pipelined van Emde Boas Tree: Algorithms, Analysis, and Applications Hao Wang and Bill Lin University of California, San Diego Spring 2005 044114

Introduction • Priority queues used many network applications • Per-flow weighted fair queueing • Management of per-flow packet buffers using DRAM • Maintenance of exact statistics counters for real-time network measurements • Items in priority queue sorted at all times (e. g. smallest key first) • Challenge: Need to operate at high speeds (e. g. 10+ Gb/s) IEEE Infocom 2007 2

Introduction • Binary heap common data structure for priority queues • O(lg n) time complexity, where n is # items • e. g. in fine-grained per-flow weighted fair queueing, n can be very large (e. g. 1 million) • O(lg n) may be too slow for high line rates • Pipeline heaps [Bhagwan, Lin 2000][Ioannou, Katevenis 2001] • Reduced time complexity to O(1) • At the expense of O(lg n) pipeline stages IEEE Infocom 2007 3

This Talk • Instead of pipelining binary heaps, we present new approach based on pipelining van Emde Boas trees • van Emde Boas (v. EB) trees introduced in 1975 • Instead of maintaining a priority queue of sorted items, maintain a sorted dictionary of keys • In many applications, since keys are represented by a w-bit integer, possible keys can only be from fixed universe of u = 2 w values • Only O(lg lg u) complexity vs. O(lg n) for heaps • Main result: pipelined v. EB with O(1) operation and O(lg lg u) pipeline stages IEEE Infocom 2007 4

van Emde Boas (v. EB) Trees • Goal: Maintain a sorted subset S from universe U = {0, 1, …, u – 1} of size u = 2 w, subject to INSERT, DELETE, EXTRACTMIN, SUCCESSOR, PREDECESSOR. e. g. • INSERT – Inserts new item into the queue • EXTRACTMIN – Removes item with smallest key IEEE Infocom 2007 5

v. EB Trees • Size u vs. n depends on application. e. g. Application w u = 2 w Per-flow 24 bits 4 × 106 Fair Queues Statistics Counters 8 bits n compare O(lg lg u) O(lg n) 106 flows u≈n 5 20 512 106 counters 512 groups u «n u≈n 3 20 8 • If u is only polynomial in n, i. e. u = O(nc), then O(lg lg u) = O(lg lg n), exponential speedup over O(lg n) IEEE Infocom 2007 6

v. EB Trees • Conceptually think of the universe of w-bits, U = {0, 1, …, 2 w – 1}, as a binary tree with height w • Think of top part H of w/2 bits, and bottom part of sub-trees of w/2 bits, L[0], L[1], …, L[ – 1] w/2 bits H H and L[0], …, L[ – 1] recursively defined as v. EB trees with w/2 bits L[0] L[1] . . . L[ -1] sub-trees IEEE Infocom 2007 7

v. EB Trees • Suppose w = 8 bits, consider e. g. a key x = 31 • Split x into high and low parts: 0 0 0 1 1 1 xh xl • Consider INSERT( x, S ) • If xh H, recursively call • INSERT( xl , L[xh] ) • Else, recursively call • INSERT( xh , H ) • INSERT( xl , L[xh] ) • Can avoid 2 nd recursion by storing min[S] w/2 bits H w/2 bits L[0] L[1] . . . L[ -1] sub-trees IEEE Infocom 2007 8

Representing v. EB Trees min[S] H L[0] n[S] L[1] L[ – 1] • min[S] : minimum key in S • n[S] : number of elements in S • H, L’s : just pointers to corresponding v. EB sub-trees IEEE Infocom 2007 9

INSERT Operation // if S empty set min to x // if x is smaller swap them // recursive call to either L[xh] or H // increment size of S Only One Recursive Call IEEE Infocom 2007 10

INSERT Operation Suppose w = 8 x= 0 0 0 1 1 1 INSERT( x, S ) • If xh H, recursively call xh xl • INSERT( xl , L[xh] ) • Else, recursively call 4 bits H • INSERT( xh , H ) • min( L[xh] ) ← xl 4 bits L[0] IEEE Infocom 2007 L[1] . . . L[255] 11

INSERT Operation Suppose w = 8 x= 0 0 0 1 1 1 INSERT( x, S ) • If xh H, recursively call xh xl • INSERT( xl , L[xh] ) • Else, recursively call 4 bits H • INSERT( xh , H ) • min( L[xh] ) ← xl 4 bits L[0] IEEE Infocom 2007 L[1] . . . L[255] 12

INSERT Operation Now w = 4 x= 1 1 INSERT( x, S ) • If xh H, recursively call xh xl • INSERT( xl , L[xh] ) • Else, recursively call • INSERT( xh , H ) L[1] • min( L[xh] ) ← xl IEEE Infocom 2007 4 bits 13

INSERT Operation Now w = 4 x= 1 1 INSERT( x, S ) • If xh H, recursively call xh xl • INSERT( xl , L[xh] ) • Else, recursively call H • INSERT( xh , H ) • min( L[xh] ) ← xl 2 bits L[0] IEEE Infocom 2007 L[1] 2 bits . . . L[255] 14

INSERT Operation Now w = 4 x= 1 1 INSERT( x, S ) • If xh H, recursively call xh xl • INSERT( xl , L[xh] ) • Else, recursively call H • INSERT( xh , H ) • min( L[xh] ) ← xl 2 bits L[0] IEEE Infocom 2007 L[1] 2 bits . . . L[255] 15

INSERT Operation Now w = 4 x= 1 1 INSERT( x, S ) • If xh H, recursively call xh xl • INSERT( xl , L[xh] ) • Else, recursively call H • INSERT( xh , H ) • min( L[xh] ) ← xl 2 bits L[0] L[1] 2 bits . . . L[255] Overall O(lg w) or O(lg lg u) time IEEE Infocom 2007 16

EXTRACTMIN Operation // if S empty, nothing to return // set return to current min // recursive call to either H or L[mh] IEEE Infocom 2007 // decrement size of S // set new minimum and return value 17

EXTRACTMIN Operation // if S empty, nothing to return // set return to current min // recursive call to either H or L[mh] IEEE Infocom 2007 // decrement size of S Again Only One //Recursive Call set new minimum and return value 18

Basic Idea of Pipelining • v. EB operations are recursively defined • Each operation only makes one recursive call at each level of recursion • Each call goes down to a sub-tree with universe defined by w/2 bits (then w/4 bits, w/8 bits, etc) • Therefore, intuitively can w/2 bits H unroll into lg lg u steps w/2 bits L[0] IEEE Infocom 2007 L[1] . . . sub-trees L[ -1] 19

Pipeline Structure input op 1 output arg 1 min stage 1 op 2 H n L 0 M 1 = pointers to all subtrees of width w/2 L 1 … L 1 n … M 2 = pointers to all subtrees of width w/4 L 1 n … M 3 = pointers to all subtrees of width w/8 L 1 n … M 4 = pointers to all subtrees of width w/16 arg 2 min stage 2 op 3 H L 0 arg 3 min stage 3 op 4 H L 0 arg 4 stage 4 : IEEE Infocom 2007 min H L 0 : 20

Main Issues • Want to initiate new operation at every pipeline cycle (e. g. initiate an INSERT or EXTRACTMIN) • Even if throughput is O(1), • Don’t want to wait lg lg u pipeline cycles to retrieve data (e. g. want data in same pipeline cycle for EXTRACTMIN) • But previous INSERT and EXTRACTMIN operation(s) may still be in pipeline • Two pipeline stages may need to access same data • Need to resolve memory access orders IEEE Infocom 2007 21

Basic Idea input op 1 • Operations proceed from top to bottom output arg 1 min stage 1 op 2 H n L 0 L 1 … L 1 n … arg 2 min stage 2 op 3 H L 0 • min[S] for the v. EB tree rooted at that level is updated immediately as operations flow thru pipeline arg 3 min stage 3 op 4 H L 0 arg 4 stage 4 : IEEE Infocom 2007 min H L 0 : • e. g. INSERT at Stage 1 will update min[S] immediately so next EXTRACTMIN at Stage 1 will retrieve correct min[S] 22

Other Operations • In addition to INSERT and EXTRACTMIN, paper describes • SUCCESSOR( x, S ) – Returns next element in S larger than x in the universe U • Can also define in a similar pipeline fashion • PREDECESSOR( x, S ) – Returns next element in S smaller than x in the universe U • EXTRACTMAX( S ) – Removes item with largest key • To facilitate these operations, we also • Store max[S] for corresponding S at each pipeline stage • Provide another pipeline memory structure for tracking “in -flight” SUCCESSOR and PREDECESSOR operations IEEE Infocom 2007 23

Conclusions • Pipelined van Emde Boas trees can achieve O(1) operations with O(lg lg u) pipelined stages • Same O(1) time complexity as pipelined heaps, but exponentially fewer pipeline stages than O(lg n) required for pipelined heaps when u is a polynomial of n • Can simultaneously support EXTRACTMIN and EXTRACTMAX, which is harder to do with heaps • Can support other operations like SUCCESSOR and PREDECESSOR, which have potential application to various network problem instances IEEE Infocom 2007 24

Thank You Spring 2005 044114