SPAA 2020 Tutorial Parallel Balanced Binary trees in

SPAA 2020 Tutorial Parallel Balanced Binary trees in Shared-Memory Yihan Sun University of California, Riverside

What’s in this tutorial • Algorithms and implementation details about parallel balanced binary trees (P-trees) • Simple with various functionalities • An open-source parallel library (PAM) and example code to use it • Applications that can be solved using the algorithms and the library in this tutorial 2

Trees [Sedgewick and Wayne] Sec. 3. 2 Binary Search Trees Sec. 3. 3 Balanced Search Trees • Trees are fundamental data structures for organizing data • Taught in entry-level undergraduate courses • In real-world applications, things are more complicated… Especially in parallel [TAOCP] 2. 3 Trees 6. 2. 2. Binary Tree Searching 6. 2. 3. Balanced Trees 6. 2. 4. Multiway Trees [CLRS] 12 Binary Search Trees 13 Red-Black Trees 14. 3 Interval trees 3

Applications Using Trees Document Search Engine Find information about “balanced” and “binary” Balanced AND Binary 1, 234, 567 results Doc 1: Balanced binary tree A binary tree is balanced if it keeps its height small … Doc 2: AVL tree is a balanced binary search tree structure … Struct Doc { int l; pair<Word, Weight>* w; }; class doc_tree { void build(Doc* d); Doc_set search(Word w); Doc_set and_search(Word* w); Doc_set or_search(Word* w); void add_doc(Doc d); …… }; Searching and updating may need to be done concurrently

Applications using trees Databases Find all young CS students with good grade select name from students where age < 25 and major = ‘CS’ and grade >= ‘A’ Struct Student { id name, grade, age, major, … }; class database { void build(Doc* d); Student* search(int id); Student* fitler(function f); void add_student(Student s); …… }; 5

Applications Using Trees Geometric queries (A 2 D range query) Find average temperature in Riverside struct Point { X x; Y y; Weight w; }; class range_tree { void build(Point* p); Double ave_weight_search(X x 1, X x 2, Y y 1, Y y 2); int count_search(X x 1, X x 2, Y y 1, Y y 2); int list_all_search(X x 1, X x 2, Y y 1, Y y 2); void insert(Point p); range_tree filter(func f); Point* output(); void update(X x, Y y, Weight w); };

Balanced Binary Trees • hyphaene compressa A wild balanced binary tree 7

Balanced Binary Trees • A wild balanced binary tree 8

Balanced Binary Trees • An abstract balanced binary tree 9

Balanced Binary Trees • Balancing schemes: invariants to keep the tree balanced and to bound the tree height • We discuss four standard balancing schemes Height balanced AVL Trees Randomized balanced Size balanced Redblack Trees Weightbalanced Trees Treaps 10

Applications Using Trees Databases Geometric queries Find information about “balanced” and “binary” Find all young CS students with good grade (A 2 D range query) Find average temperature in Riverside area: 62 F What we want Document Search Engine Balancedl. ANDl. Binary Elegance – Framework select 1, 234, 567 results name Genericlfor balancing schemes Doc 1: Balancedlbinary tree Genericiforfrom applications A binary tree is balanced if it students keeps its height small … Massivewhere Data - Performance Doc 2: AVL tree age < 25 Parallelism and concurrency AVL tree is a balanced binary and major = ‘CS’ theory and in practice search tree structure … Efficiency both andingrade >= ‘A’ Riverside Comprehensive Queries – Functionality Range queries, bulk updates, … Augmentation, … Dynamicity, multi-versioning, … 11

What does P-tree look like? Elegance • Genericlfor balancing schemes • Genericifor applications Massive Data • Parallelism and concurrency • Efficiency both in theory and in practice Framew ork Performanc e Functionality Comprehensive Queries • Range queries, bulk updates, … • Augmentation, … • Dynamicity, multi-versioning, … Join (A primitive for trees) 12

1 Genericlfor balancing schemes All algorithms except join are identical across balancing schemes One algorithm for multiple balancing schemes! 2 Genericifor applications Multiple applications based on the same tree structure One tree for different problems! A primitive for trees: 13

The Primitive Join • 4 2 10 (Rebalance if necessary) 14

The Primitive Join 4 2 10 6 5 12 8 3 1 9 11 13 14 7 15

The Primitive Join • Connect at a balancing point 10 4 2 1 6 3 12 8 5 9 7 11 13 14 Balanced! 16

The Primitive Join • Connect at a balancing point 10 8 12 9 4 1 3 5 13 14 6 2 11 7 17

The Primitive Join • Connect at a balancing point • Rebalance (specific to balancing schemes) • Join algorithms for four balancing schemes and the cost 10 bound [SPAA’ 16] 4 12 8 2 1 3 9 6 5 11 13 14 7 18

How does Join help? 2 Augmentation Using Join • A unified framework for augmentation • Model multiple applications Experiments • Generic across balancing schemes • Highly-Parallel • Theoretically efficient Applications 1 Algorithms Using Join 3 Persistence Using Join • Multi-versioning on trees 19

PART 1 Algorithms Using Join Generic algorithms across balancing schemes Parallel algorithms using divide-and-conquer paradigm Theoretically efficient 20

Join-based insertion 21

Join-based Algorithms: insertion Compare Insert 4 5 2 1 9 3

Join-based Algorithms: insertion Join results (rebalanced may be required) 5 2 1 9 3 4

Join-based Algorithms: insertion Join results (rebalanced may be required) 3 2 1 5 4 9 Join will rebalance for Insert!

Join-based split and Join 2 25

The Inverse of Join: Split 25 • 51 13 8 5 36 17 12 15 22 30 70 30 13 8 17 22 36 70 80 51 12 42 Split by 42 22 5 80 15 95

The Inverse of Join: Split ? 27

Another helper function: join 2 • u v k

Other Join-based algorithms 29

BST Algorithms • BST algorithms using divide-and-conquer scheme • Recursively deal with two subtrees (possibly in parallel) • Combine results of recursive calls and the root (e. g. , using join or join 2) • Usually gives polylogarithmic bounds on span func(T, …) { if (T is empty) return base_case; M = do_something(T. root); in parallel: L=func(T. left, …); R=func(T. right, …); return combine_results(L, R, M, …) }

Get the maximum value • In each node we store a key and a value. The nodes are sorted by the keys. Similar algorithm work on any map-reduce function

Map and reduce • Maps each entry on the tree to a certain value using function map, then reduce all the mapped values using reduce (with identity). • Assume map and reduce both have constant cost. map_reduce(Tree T, function map, function reduce, value_type identity) { if (T is empty) return identity; M=map(t. root); in parallel: L=map_reduce(T. left, map, reduce, identity); R=map_reduce(T. right, map, reduce, identity); return reduce(L, M), R);

Filter • filter(Tree T, function f) { if (T is empty) return an empty tree; in parallel: L=filter(T. left, f); R=filter(T. right, f); if (f(T. root)) return join(L, T. root, R); else return join 2(L, R); }

Construction

Output to array • to_array(Tree T, array A, int offset) { if (T is empty) return; A[offset+T. left. size] = get_entry(T. root); in parallel: to_array(T. left, A, offset); to_array(T. right, A, offset+T. left. size()+1); The size of the left subtree

Range query (1 D) • Call join Equivalent to using two split algorithms

Join-based Algorithms: Union • 37

Join-based Algorithms: Union Base case 7 4 0 5 8 2 1 9 3 6

Join-based Algorithms: Union 5 4 7 2 8 0 1 9 3 6

Join-based Algorithms: Union 5 0 7 2 4 1 Union 3 9 8 Union 6

Join-based Algorithms: Union 5 2 1 7 3 0 6 8 4 9

Join-based Algorithms: Union 5 2 1 0 7 3 6 4 8 9 Similarly we can implement intersection and difference.

Join-based Algorithms: Union • The bound also holds for intersection and difference 43

Join-based algorithms • A wide variety of algorithms using Join are proved to be work-optimal and have polylogarithmic span Functions Work Depth insert, delete, range, split, join 2 union, intersection, difference Join captures the intrinsic property of different map_reduce balanced binary trees in algorithms. filter build …… Optimal Achieves speedup 50 -90 x on 72 cores with hyperthreading Polylog 44

Outline 1 Algorithms Using Join • • • Generic across balancing schemes Parallel using divide-and-conquer Simple Theoretically efficient Fast in practice 2 Augmentation Using Join 3 Persistence Using Join 45

PART 2 Augmentation Using Join Augmented trees for fast range sum The augmented map framework for applications 46

Augmentation for Range Queries Geometric queries (A 2 D range query) Find average temperature in Riverside area: 62 F (A 2 D range query) Find all nearby Pokémon / Poke stops Riverside 47

Augmented Trees for 1 D Range Query • Each tree node store some extra information about the whole subtree rooted at it (E. g. , partial sum) 4, 11 sum of values in its subtree 54 1, 4 4 2, 7 6, 10 13 30 3, 2 2 5, 17 17 7, 3 key, value 3 Partial sum 48

Augmented Trees for 1 D Range Query • Different functionalities are achieved by different augmentations 4, 11 max of values in its subtree 17 2, 7 6, 10 7 17 1, 4 3, 2 5, 17 7, 3 key, value 4 2 17 3 Partial “sum” 49

Augmentation – Formalization 4, 11 54 • 2, 7 6, 10 13 30 1, 4 3, 2 5, 17 7, 3 key, value 4 2 17 3 Partial “sum” The augmented 50

Augmentation – Formalization 4, 11 key, value 54 Partial “sum” 2, 7 13 1, 4 4 11 = 4 + 7 + 2 + 11 + 17 + 10 + 3 6, 10 7 30 =4+7+2 3, 2 4 2 =2 5, 17 17 17 = 17 10 = 17 + 10 + 3 7, 3 3 3 =3 The augmented value • 51

Augmentation – Formalization 4, 11 key, value • 54 Partial “sum” 2, 7 13 1, 4 4 4 =4 11 = 4 + 7 + 2 + 11 + 17 + 10 + 3 6, 10 7 30 =4+7+2 3, 2 2 2 =2 5, 17 17 17 = 17 10 = 17 + 10 + 3 7, 3 3 3 =3 The augmented value 52

Augmentation – Formalization 4, 11 key, value • 54 Partial “sum” 2, 7 13 1, 4 4 4 =4 11 = (4 + 7 + 2) + g(4, 11) + (17 + 10 + 3) 6, 10 7 30 =4+7+2 3, 2 2 2 =2 5, 17 17 17 = 17 10 = 17 + 10 + 3 7, 3 3 3 =3 The augmented value 53

Augmentation – Formalization • 4, 11 key, value 54 Partial “sum” 2, 7 13 1, 4 4 4 =4 11 = 13 + 11 + 30 6, 10 7 30 =4+7+2 3, 2 2 2 =2 5, 17 17 17 = 17 10 = 17 + 10 + 3 7, 3 3 3 =3 The augmented value 54

Augmented trees • Define keys and values • Define what is the augmented value • Define what is the map function • Define what is the reduce function (and identity) The Wikipedia page of the augmented map defined by us [PPo. PP’ 18], which is an abstract data type (ADT) that extends the augmented trees 55

Augmentation: Applications • Can be applied to a variety of applications: q Range Sum: q 1 D stabbing query (can yield an interval tree): q 2 D range query (can yield a range tree): q Document searching: -, -, • Also, many tree-based geometric problems like segment queries, rectangle queries, … 56

Code for the Interval Tree struct interval_map { using interval = pair<int, int>; struct entry { using key_t = int; static bool comp(key_t a, key_t b) { return a < b; } using val_t = int; using aug_t = int; static aug_t base(key_t k, val_t v) { return v; } static aug_t combine(aug_t a, aug_t b) { return (a > b) ? a : b; } static aug_t I() { return 0; } }; using aug_map = aug_map<entry>; aug_map m; interval_map(interval* A, int n) { m = aug_map(A, A+n); } bool stabbing(int p) { return (m. aug_left(p) > p); }}; Construction Query Interval tree, 53 lines Range tree, 176 lines Document searching, 98 lines 57

Range Queries – Sequential and Parallel [PPo. PP’ 18, ALENEX’ 19] • All running times: Lower is better PAM Boost CGAL 600 150 1200 400 100 800 200 50 400 0 Construction (s) Small window query (us) Large window query (s) 58

Range Queries – Sequential and Parallel [PPo. PP’ 18, ALENEX’ 19] Self-speedup of PAM 72 cores, 144 threads Higher is better All running times: Lower is better PAM Boost CGAL 600 400 200 150 80 800 PAM is 100 fast even running 400 50 sequentially 0 0 Construction (s) Construction 0 Small window query (us) Query Large window query (s) 2 -9 x Faster than CGAL 2 x Faster than CGAL 1. 5 x Faster than Boost 1. 4 -26 x Faster than Boost PAM also gets ~60 x self 40 speedup in construction and 60 -80 x in queries 0 Self-speedup Construction Small window query Large window query 59

Applications using augmentation • 1 D stabbing query • 2 D range query, segment query, rectangle query • 2 D sweepline algorithms • Inverted index searching • Example code for all applications available on Github • https: //github. com/cmuparlay/PAM • Algorithms and more experiments available: • PAM: Parallel Augmented Maps, Yihan Sun, Daniel Ferizovic and Guy Blelloch, ACM Symposium on Principles and Practice of Parallel Programming (PPo. PP), 2018 • Parallel Range, Segment and Rectangle Queries with Augmented Maps, Yihan Sun and Guy E. Blelloch, Algorithm Engineering and Experiments (ALENEX), 2019 60

Outline 1 Algorithms Using Join 2 Augmentation Using Join • Formalize augmented trees: implemented by just join • Multiple applications: range-sum, stabbing queries, range queries, inverted index searching • Simple code • Parallel solution • Fast in practice 3 Persistence Using Join 61

PART 3 Persistence Using Join Persistent algorithms Multi-version concurrency control (MVCC) 62

What Are Persistence and MVCC? • Persistence [DSST’ 86]: for data structures • Preserves the previous version of itself • Always yields a new version when being updated • Multi-version Concurrency Control (MVCC): for databases • Let write transactions create new versions • Let ongoing queries work on old versions Why Persistence and MVCC? • To guarantee concurrent updates and queries to work correctly and efficiently • Queries work on a consistent version • Writers/readers do not block each other 63

Why Persistence and MVCC? An example of a document search engine. Riverside Whales OR dog Calories Document 1: Blue whales eat half a million calories in one mouthful. room AND magic Document 2: My dog probably thinks I’m magical because rooms light up when I enter them. A Document For end-user experience: Document 3: Banging your head against a wall for one hour burns 150 calories. Database Document 4: The University of California, Riverside is a public research university in Riverside, California. • Queries shouldn't be delayed by updates • Queries must be done on a consistent version of database Generally useful for any database systems with concurrent updates and queries. Hybrid Transactional and Analytical Processing (HTAP) Database System 64

Persistence Using Join • Path-copying: copy the affected path on trees • Copying occur only in Join! insert 4 5 3 1 3’ 5’ 8 4 9 65

Persistence Using Join • Path-copying: copy the affected path on trees • Copying occur only in Join! insert 5 4 3 1 8 9 66

Persistence Using Join • Path-copying: copy the affected path on trees • Copying occur only in Join! • Always copy the middle node • All the other parts in the algorithm remain unchanged • No extra cost in time asymptotically, small overhead in space • Safe for concurrency – multi-version concurrency control 5’ (MVCC) 5 5’ 3’ 3 1 3’ 8 4 9 67

Transactions using Multi-version Concurrency Control (MVCC) Bank: add the Bank: find all • Lock-free atomic updates with balance>10 • A series of operations • A bulk of operations (e. g. , union) • Easy roll-back • Do not affect other concurrent operations • Any operation works on as if a singleversioned tree with no extra (asymptotical) cost total balance Insurance agent: check Bob’s balance Frank: check my balance current Carol Wendy +$2 Carol 17 Bob 20 Alice 17 David 13 $2 Wendy Carol -$2 Carol” 15 Carol’ 17 Frank’ 7 Wendy 4 Wendy’ 6

Transactions using Multi-version Concurrency Control (MVCC) Bank: add the Bank: find all • Lock-free atomic updates with balance>10 total balance • A series of operations • A bulk of operations (e. g. , union) • Easy roll-back • Do not affect other concurrent operations • Any operation works on as if a singleversioned tree with no extra (asymptotical) cost • Concurrent writes? • Concurrent transactions work on snapshots • They don’t come into effect on the same tree? Insurance agent: check Bob’s balance current Carol Wendy +$2 Carol 17 Bob 20 Alice 17 David 13 $2 Wendy Carol -$2 Carol” 15 Carol’ 17 Frank’ 7 Wendy 4 Wendy’ 6

Batching • Collect all concurrent writes can commit using a single writer once a while delete 16 Insert 5 Insert 9 Insert 15 Insert 8 Delete 3 Delete 6 Database 71

Batching • Collect all concurrent writes can commit using a single writer once a while delete 16 Insert 5 Insert 9 Insert 15 Insert 8 Delete 3 Delete 6 Batching layer Union of [5, 8, 9, 15] Difference of [3, 6, 16] Database 72

Compare to Concurrent Data Structures [SPAA’ 19] • 73

Compare to Concurrent Data Structures [SPAA’ 19] M txns/s Throughput: Higher is better 140 120 100 80 60 40 20 0 1. 2 -1. 4 x A B C (50/50) (95/5) (100/0) Read/Write Faster than all the other concurrent data structures PAM Open. BW Masstree B+tree Chromatic 74

Compare to Concurrent Data Structures [SPAA’ 19] 50 40 30 20 10 0 1. 2 -1. 4 x 1 2 4 8 16 32 64 128 144 M txns/s Throughput: Higher is better # threads Workload A (50/50) PAM Open. BW Mass. Tree B+Tree Faster than all the other concurrent data structures (almost-linear) speedup scaling up to 144 Threads 75

Garbage Collection • Reference Counter Garbage Collector • Each tree node records the number of other tree nodes/pointers refers to it • Node 8 and 1 in the example have reference counter 2 5 3 1 3’ 5’ 8 4 9 Reference count: Node 1 3 5 8 9 5’ 3’ 4 Count 2 1 1 1 1

Garbage Collection [SPAA’ 19] • Each tree node records the number of other tree nodes/pointers refers to it • Node 8 and 1 in the example have reference counter 2 • Collect a node if and only if its reference count is 1 5 3 1 3’ collect(node* t) { if (!t) return; if (t->ref_cnt == 1) { node* lc = t->lc, *rc = t->rc; free(t); in parallel: collect(lc); collect(rc); } else dec(t->ref_cnt); } 5’ 8 4 9 Node 1 3 5 8 9 5’ 3’ 4 Count 12 1 1 21 1 1

Document Searching • Inverted index Document 1: Blue whales are the largest animals ever known to have lived on Earth. Document 2: Elephants are the largest land mammals. Document 3: Banging your head against a wall for one hour burns 150 calories. Document 4: Blue whales eat half a million calories in one mouthful. Searching queries: Word Document list … … blue 1, 4 whale 1, 4 earth 1 largest 1, 2 calories 3, 4 mammals 2 head 3 Eat 4 half 4 million 4 mouthful 4 Find corresponding document lists Add a new document: All updates need to be done atomically Add document 4 Add “ 4” to the lists of {Blue, whales, eat, half, million, calories, mouthful} 78

Inverted Indexes • Stores each word in the outer tree with its document list as an inner tree • Queries: concurrent analytical queries are done on the current version Orange objects: words Blue document objects: documents • OR/AND query: union/intersection on the two lists parallel • Nested trees: represent the affiliation relation tree join D 3 algorithm optimal D 2 union D 2 D 5 D 1 D 4 79

Inverted Indexes • Updates: take a Union with path-copying (update inner tree for duplicates) • Atomic: make multiple word-doc pairs visible at once new • Non-destructive: old versions for ongoing queries version current version parallel’ join tree’ tree D 3 algorithm Orange objects: words Blue objects: documents D 3’ optimal D 5’ D 2 D 5 D 2’ D 7 union D 7: parallel tree D 4’ D 1 D 4 D 7 new_version = current_version. add((parallel, D 7), (tree, D 7)) current_version = new_version 80

Experiments on TPC Benchmarks 15 12 9 6 3 0 13. 51 # Transactions / s # Queries / s • 100 GB data, 72 cores with hyperthreading • Compare to Hy. Per [Neumann et al. ] and Mem. SQL [Shamgnov’ 14] 12000 10280 Achieves good performance for both 8696 9000 updates and queries in HTAP 6000 workloads. 2. 84 1. 36 Query Throughput P-tree Hyper 4 -9 x faster than both systems in queries All throughput 3000 75 0 Update Throughput Mem. SQL Comparab le performance to the best of the previous in numbers: updates Higher is better 81

Applications using join-based MVCC • Inverted index searching • Indexes for HTAP database systems • Transactional systems with precise GC • Graph processing • Example code for all applications available on Github • Algorithms and more experiments available: • (library) PAM: Parallel Augmented Maps, Yihan Sun, Daniel Ferizovic and Guy Blelloch, PPo. PP 2018 • (version control, garbage collection) Multiversion Concurrency with Bounded Delay and Precise Garbage Collection, Naama Ben-David, Guy E. Blelloch, Yihan Sun and Yuanhao Wei, SPAA 2019. • (graph processing, compression) Low-Latency Graph Streaming Using Compressed Purely-Functional Trees, Laxman Dhulipala, Guy Blelloch, and Julian 82

Outline 1 Algorithms Using Join 2 Augmentation Using Join 3 Persistence Using Join • Multi-version concurrency control (MVCC) • Database systems with concurrent updates and queries • Fast in practice 83

Join Weightbalance AVL Red-black Treap 84

Join Red-black Treap ist en ce Weightbalance A ug e m rs AVL t a nt Pe n o i • Algorithms 85

Join • Algorithms • Augmentation • Persistence 86

In theory: Join-based algorithms Join • Algorithms • Augmentation • Persistence • Simple • Supporting a wide range of algorithms • Work-optimal and poly-log span In practice: The PAM library • Balancing scheme independent • Abstract augmentation • Persistent & multi-versioning P-trees Interval Trees High … 2 D range query Inverted Database Concise implementation 2 D segment query Index Management performance both sequentially and in Systems parallel 2 D rectangle query Searching 87