The complexity and correctness of algorithms with binary

The complexity and correctness of algorithms (with binary trees as an example)

An algorithm is an unambiguous sequence of simple, mechanically executable instructions. 3

Trees • A tree is a data structure used to represent different kinds of data and help solve a number of algorithmic problems • Game trees (i. e. , chess ), UNIX directory trees, sorting trees etc. • We will study at least two kinds of trees: Binary Search Trees and Red-Black Trees. 4

Trees • Trees have nodes. They also have edges that connect the nodes. Between two nodes there is always only one path. Tree nodes Tree edges 5

Trees • Trees are rooted. Once the root is defined (by the user) all nodes have a specific level. • Trees have internal nodes and leaves. Every node (except the root) has a parent and it also has zero or more children. root level 0 level 1 internal nodes level 2 level 3 leaves parent and child 6

Binary trees • A binary tree is a tree such that each node has at most 2 children. 7

Binary trees Recursive definition (important): 1. An empty tree is a binary tree 2. A node with two child sub-trees is a binary tree 3. [often implicit] Only what you get from 1 by a finite number of applications of 2 is a binary tree. Values stored at tree nodes are called keys. 56 26 200 28 18 12 24 190 213 27 8

Binary search trees • A binary search tree (BST) is a binary tree with the following properties: – The key of a node is always greater than the keys of the nodes in its left subtree. – The key of a node is always smaller than the keys of the nodes in its right subtree. 9

Binary search trees • Stored keys must satisfy the binary search tree property. 56 – y in left subtree of x, then key[y] key[x]. – y in right subtree of x, then key[y] key[x]. 26 200 28 18 12 24 190 213 27 10

Binary search trees: examples root 14 C A 10 D 16 root 8 11 15 18 14 10 8 16 11 15 11

Binary search trees • Data structures that can support dynamic set operations. – Search, minimum, maximum, predecessor, successor, insert, and delete. • Can be used to build – Dictionaries. – Priority Queues. 12

BST – Representation • Represented by a linked data structure of nodes. • root(T) points to the root of tree T. • Each node contains fields: – – key left – pointer to left child: root of left subtree. right – pointer to right child : root of right subtree. p – pointer to parent; p[root[T]] = NIL (optional). 13

Inorder traversal of a BST The binary search tree property allows the keys of a binary search tree to be printed, in (monotonically increasing) order, recursively. Inorder-Tree-Walk (p) if p NIL then 56 Inorder-Tree-Walk(left[p]) print key[x] Inorder-Tree-Walk(right[p]) 26 28 18 12 200 24 190 213 27 • Can you prove the correctness on in-order traversal? • How long does an in-order walk take? 14

Correctness of inorder traversal • Must prove that it prints all elements, in order, and that it terminates. • By induction on size of tree. Size=0: Easy. • Size >1: – Prints left subtree in order by induction. – Prints root, which comes after all elements in left subtree (still in order). – Prints right subtree in order (all elements come after root, so still in order). 15

Notice how we used the recursive definition of a tree in our inductive proof. We exploit the recursive structure of a tree, and this approach - which is general to all recursive definitions, and not restricted to trees - is called structural induction. 16

Searching in a BST Tree-Search(x, k) if x = NIL or k = key[x] then return x if k < key[x] then return Tree-Search(left[x], k) else return Tree-Search(right[x], k) 26 24 27 Height 3 200 28 18 12 Height 4 56 190 213 Height 2 Height 1 Time complexity is proportional with h is the height of the tree. 17

Inserting an element in a BST Tree-Insert(Tree T, int z) • Change the dynamic set represented by a BST. y NIL x root[T] • Ensure the binary search tree property holds after change. while x NIL do y x • Insertion is easier than deletion. if key[z] < key[x] then x left[x] else x right[x] p[z] y if y = NIL 56 26 28 18 12 24 then root[t] z else if key[z] < key[y] 200 190 213 then left[y] z else right[y] z 27 18

Analysis of Insertion Tree-Insert(T, z) y NIL • While loop in lines 3 -7 searches x root[T] for place to insert z, maintaining while x NIL do parent y. y x This takes time proportional with if key[z] < key[x] h then x left[x] • Initialization: constant time • Lines 8 -13 insert the value: constant time Þ TOTAL: C 1 + C 2 + C 3*h else x right[x] p[z] y if y = NIL then root[t] z else if key[z] < key[y] then left[y] z else right[y] z 19

Deletion • Goal: Delete a given node z from a binary search tree. • We need to consider several cases. – Case 1: z has no children. • Delete z by making the parent of z point to NIL, instead of to z. 15 15 5 16 3 12 10 13 18 delete 6 7 3 20 z 16 5 12 23 10 20 18 23 6 7 20

Deletion • Case 2: z has one child. – Delete z by making the parent of z point to z’s child, instead of to z. – Update the parent of z’s child to be z’s parent. 15 5 3 z 16 5 12 10 6 18 20 3 20 13 15 delete 12 18 23 10 23 6 7 7 21

Deletion delete • Case 3: z has two children. – z’s successor (y) is the minimum node in z’s right subtree. – y has either no children or one right child (but no left child). 6 z 5 16 3 20 12 10 13 18 23 y 6 7 • Why? – Delete y from the tree (via Case 1 or 2). – Replace z’s key and satellite data with y’s. 15 15 6 16 3 20 12 10 13 18 23 7 22

Successor node • The successor of node x is the node y such that key[y] is the smallest key greater than key[x]. • The successor of the largest key is NIL. • Search for a successor consists of two cases. – If node x has a non-empty right subtree, then x’s successor is the minimum in the right subtree of x. – If node x has an empty right subtree, then: • As long as we move to the left up the tree (move up through right children), we are visiting smaller keys. • x’s successor y is the node that x is the predecessor of (x is the maximum in y’s left subtree). • In other words, x’s successor y, is the lowest ancestor of x whose left child is also an ancestor of x. • We can define the predecessor node similarly. 23

Exercise: Sorting using BSTs Sort (A) for i 1 to n do tree-insert(A[i]) inorder-tree-walk(root) – What are the worst case and best case running times? – In practice, how would this compare to other sorting algorithms? 24

Wrap-up • Determining the complexity of algorithms is usually the first step in obtaining better algorithms. – Or in realizing we cannot do any better. • What should you know? – Inductive proofs on trees. – Binary search trees and operations on BSTs. – Height of trees and how it influences efficiency. 25