CSCI 3160 Design and Analysis of Algorithms Tutorial

CSCI 3160 Design and Analysis of Algorithms Tutorial 3 Chengyu Lin

Outline • Union-find data structure • Minimum spanning tree (MST) • Kruskal’s algorithm

Union-find • A data structure for disjoint sets • n = number of members, forming disjoint groups • Two members are in the same group if and only if they have a common leader • Operations: o Union: merge two groups o Find: name the leader of the group • We do not really care about who the leader is; we only want to tell one group from another

Union-find • Idea: use a forest (= collection of trees) o a group → a tree o leader → root of the tree • Example: o Group 1: Alice, Bob o Group 2: Carol, Dave, Eve • Store the height of each tree at the root A 1 B D 1 C E

Union-find • Find: return the root of the group • Union: make the leader of one group the boss of the other o Our heuristic: make the root of the shorter tree point to the root of the other tree o If both trees are of the same height h, then the resulting tree has height h+1

Union-find in action • Initialize o Everyone is his/her own boss A 0 B 0 C 0 D 0 E 0

Union-find in action • Union(A, B) o Find(A) returns A and Find(B) returns B o Make Alice the boss of Bob o Increase the height of the tree A 1 B 0 C 0 D 0 E 0

Union-find in action • Union(C, D) o Find(C) returns C and Find(D) returns D o Make Carol the boss of Dave o Increase the height of the tree A 1 B 0 C 1 D 0 E 0

Union-find in action • Union(B, D) o Find(B) returns A and Find(D) returns C o Make Alice the boss of Carol o Increase the height of the tree A 2 B 0 C 1 D 0 E 0

Your turn • Find(C)? A 2 B 0 C 1 D 0 E 0

Your turn • Find(C)? o Find(C) returns A A 2 B 0 C 1 D 0 E 0

Your turn • Union(B, E)? A 2 B 0 C 1 D 0 E 0

Your turn • Union(B, E)? o Find(B) returns A and Find(E) returns E o Make Alice the boss of Eve o No increase in the tree height (Why? ) A 2 B 0 C 1 D 0 E 0

Final result • Does this look more like a tree? A 2 B 0 C 1 D 0 E 0

Analysis • Height of each tree = O(log n) • Cost of Find(): O(log n) • Cost of Union(): O(log n) o Dominated by the cost of Find()

Minimum spanning tree • G = (V, E): undirected, connected, (non-negatively) weighted • Problem: find a subset of edges with minimum total weight such that all vertices in V are connected using these edges A 1 B 4 C 5 2 3 D 6 7 E Total weight = 1 + 2 + 3 + 6 = 12

Kruskal’s algorithm • In words: o Sort the edges in ascending order of weights o Initialize: set T = Ø o While T is not a spanning tree • Consider the next edge in the sorted list • If adding it to T does not cause a cycle, add it • The union-find structure helps us check for cycles o Adding an edge corresponds to putting the two endpoints into the same group o Connecting two vertices in the same group causes a cycle

Dry run • 1: Sort the edges Edge Weight (A, B) 1 (C, D) 2 (B, D) 3 (A, C) 4 (A, D) 5 (B, E) 6 (D, E) 7 A 4 C 5 2 1 B 3 D 6 7 E

Dry run • 2: Set T = Ø Edge Weight (A, B) 1 (C, D) 2 (B, D) 3 (A, C) 4 (A, D) 5 (B, E) 6 (D, E) 7 B A C D A 4 C 5 2 1 E B 3 D 6 7 E

Dry run • 3: Consider the first edge Edge Weight (A, B) 1 (C, D) 2 (B, D) 3 (A, C) 4 (A, D) 5 (B, E) 6 (D, E) 7 B A C D A 4 C 5 2 1 E B 3 D 6 7 E

Dry run • 4: Consider the second edge Edge Weight (A, B) 1 (C, D) 2 (B, D) 3 (A, C) 4 (A, D) 5 (B, E) 6 (D, E) 7 B A C D A 4 C 5 2 1 E B 3 D 6 7 E

Dry run • 5: Consider the third edge Edge Weight (A, B) 1 (C, D) 2 (B, D) 3 (A, C) 4 (A, D) 5 (B, E) 6 (D, E) 7 B A C D A 4 C 5 2 1 E B 3 D 6 7 E

Dry run • 6: Edge Weight (A, B) 1 (C, D) 2 (B, D) 3 (A, C) 4 (A, D) 5 (B, E) 6 (D, E) 7 B A C D A 4 C 5 2 1 E B 3 D 6 7 E

Dry run • 7: Edge Weight (A, B) 1 (C, D) 2 (B, D) 3 (A, C) 4 (A, D) 5 (B, E) 6 (D, E) 7 B A C D A 4 C 5 2 1 E B 3 D 6 7 E

Dry run • 8: Edge Weight (A, B) 1 (C, D) 2 (B, D) 3 (A, C) 4 (A, D) 5 (B, E) 6 (D, E) 7 B A C D A 4 C 5 2 1 E B 3 D 6 7 E

Dry run • 9: We have find an MST! Edge Weight (A, B) 1 (C, D) 2 (B, D) 3 (A, C) 4 (A, D) 5 (B, E) 6 (D, E) 7 B A C D A 4 C 5 2 1 E B 3 D 6 7 E

Analysis • Correctness o In each step, we keep a set of edges T that is a subset of an MST o Theorem: Let S be any tree in the forest (V, T). We can add the lightest edge from S to V-S to T, and the resulting set of edges is also a subset of an MST • Space complexity = O(|V|+|E|) • Time complexity = O(|E|log|V|) o Sorting alone takes time O(|E|log|V|) (Note that |E| = O(|V|2)) o Cycle checking takes O(log|V|) operations (Find()) o Adding an edge also takes O(log|V|) operations (Union())

Dry run revisited • 2: Set T = Ø Edge Weight (A, B) 1 (C, D) 2 (B, D) 3 (A, C) 4 (A, D) 5 (B, E) 6 (D, E) 7 B A C A 0 B 0 D C 0 E D 0 E 0

Dry run revisited • 3: Consider the first edge Edge Weight (A, B) 1 (C, D) 2 (B, D) 3 (A, C) 4 (A, D) 5 (B, E) 6 (D, E) 7 B A C A 1 B 0 D C 0 E D 0 E 0

Dry run revisited • 4: Consider the second edge Edge Weight (A, B) 1 (C, D) 2 (B, D) 3 (A, C) 4 (A, D) 5 (B, E) 6 (D, E) 7 B A C A 1 B 0 D C 1 E D 0 E 0

Dry run revisited • 5: Consider the third edge Edge Weight (A, B) 1 (C, D) 2 (B, D) 3 (A, C) 4 (A, D) 5 (B, E) 6 (D, E) 7 B A C A 2 B 0 D C 1 E D 0 E 0

Dry run revisited • 6: Edge Weight (A, B) 1 (C, D) 2 (B, D) 3 (A, C) 4 (A, D) 5 (B, E) 6 (D, E) 7 B A C A 2 B 0 D C 1 E D 0 E 0

Dry run revisited • 7: Edge Weight (A, B) 1 (C, D) 2 (B, D) 3 (A, C) 4 (A, D) 5 (B, E) 6 (D, E) 7 B A C A 2 B 0 D C 1 E D 0 E 0

Dry run revisited • 8: Edge Weight (A, B) 1 (C, D) 2 (B, D) 3 (A, C) 4 (A, D) 5 (B, E) 6 (D, E) 7 B A C A 2 B 0 D C 1 E D 0 E 0

Dry run revisited • 9: We have find an MST! Edge Weight (A, B) 1 (C, D) 2 (B, D) 3 (A, C) 4 (A, D) 5 (B, E) 6 (D, E) 7 B A C A 2 B 0 D C 1 E D 0 E 0

End • Questions