Algorithms and Data Structures Lecture XII Simonas altenis

Algorithms and Data Structures Lecture XII Simonas Šaltenis Aalborg University simas@cs. auc. dk November 13, 2003 1

This Lecture n n Weighted Graphs Minimum Spanning Trees n n n Greedy Choice Theorem Kruskal’s Algorithm Prim’s Algorithm November 13, 2003 2

Spanning Tree n A spanning tree of G is a subgraph which n n n is a tree contains all vertices of G How many edges are there in a spanning tree, if there are V vertices? November 13, 2003 3

Minimum Spanning Trees n n Undirected, connected graph G = ( V , E) Weight function W: E ® R (assigning cost or length or other values to edges) Spanning tree: tree that connects all the vertices Optimization problem – Minimum spanning tree (MST): tree T that connects all the vertices and minimizes November 13, 2003 4

Optimal Substructure MST(G’) = T – (u, v) MST(G) = T u n v ”u+v” ”Cut and paste” argument n If G’ would have a cheaper ST T’, then we would get a cheaper ST of G: T’ + (u, v) November 13, 2003 5

Idea for an Algorithm n n We have to make V – 1 choices (edges of the MST) to arrive at the optimization goal After each choice we have a sub-problem one vertex smaller than the original n n Dynamic programming algorithm, at each stage, would consider all possible choices (edges) If only we could always guess the correct choice – an edge that definitely belongs to an MST November 13, 2003 6

Greedy Choice n n Greedy choice property: locally optimal (greedy) choice yields a globally optimal solution Theorem n n n Let G=(V, E), and let S Í V and let (u, v) be min-weight edge in G connecting S to V – S : a light edge crossing a cut Then (u, v) Î T – some MST of G November 13, 2003 7

Greedy Choice (2) n Proof n n Suppose (u, v) is light, but (u, v) Ï any MST look at path from u to v in some MST T Let (x, y) – the first edge on path from u to v in T that crosses from S to V – S. Swap (x, y) with (u, v) in T. this improves T – contradiction (T supposed to be an MST) V-S S x u November 13, 2003 y v 8

Generic MST Algorithm Generic-MST(G, w) 1 A¬Æ // Contains edges that belong to a MST 2 while A does not form a spanning tree do 3 Find an edge (u, v) that is safe for A 4 A¬AÈ{(u, v)} 5 return A Safe edge – edge that does not destroy A’s property More. Specific-MST(G, w) 1 A¬Æ // Contains edges that belong to a MST 2 while A does not form a spanning tree do 3. 1 Make a cut (S, V-S) of G that respects A 3. 2 Take the min-weight edge (u, v) connecting S to V-S 4 A¬AÈ{(u, v)} 5 return A November 13, 2003 9

Pseudocode assumptions n Graph ADT with an operation n n V(): Vertex. Set E(): Edge. Set w(u: Vertex, v: Vertex): int – returns a weight of (u, v) A looping construct “for each v ÎV ”, where V is of a type Vertex. Set, and v is of a type Vertex ADT with operations: n n n adjacent(): Vertex. Set key(): int and setkey(k: int) parent(): Vertex and setparent(p: Vertex) November 13, 2003 10

Prim-Jarnik Algorithm n n Vertex based algorithm Grows one tree T, one vertex at a time A set A covering the portion of T already computed Label the vertices v outside of the set A with v. key() – the minimum weigth of an edge connecting v to a vertex in A, v. key() = ¥, if no such edge exists November 13, 2003 11

Prim-Jarnik Algorithm (2) MST-Prim(G, r) 01 for each vertex u Î G. V() 02 u. setkey(¥) 03 u. setparent(NIL) 04 r. setkey(0) 05 Q. init(G. V()) // Q is a priority queue ADT 06 while not Q. is. Empty() 07 u ¬ Q. extract. Min() // making u part of T 08 for each v Î u. adjacent() do 09 if v Î Q and G. w(u, v) < v. key() then 10 v. setkey(G. w(u, v)) 11 Q. modify. Key(v) 12 v. setparent(u) November 13, 2003 updating keys 12

Prim-Jarnik’s Example n Let’s do an example: B 4 A 12 8 8 C 3 6 H 4 D 5 3 9 13 I 1 n 6 F 10 E G Let’s keep track: n What is set A, which vertices are in Q, what cuts are we making, what are the key values November 13, 2003 13

Implementation issues MST-Prim(G, r) 01 for each vertex u Î G. V() 02 u. setkey(¥) 03 u. setparent(NIL) 04 r. setkey(0) 05 Q. init(G. V()) // Q is a priority queue ADT 06 while not Q. is. Empty() 07 u ¬ Q. extract. Min() // making u part of T 08 for each v Î u. adjacent() do 09 if v Î Q and G. w(u, v) < v. key() then 10 v. setkey(G. w(u, v)) 11 Q. modify. Key(v) 12 v. setparent(u) November 13, 2003 14

Priority Queues n A priority queue is an ADT for maintaining a set S of elements, each with an associated value called key n We need PQ to support the following operations n n init(S: Vertex. Set) extract. Min(): Vertex modify. Key(v: Vertex) To choose how to implement a PQ, we need to count how many times the operations are performed: n init is performed just once and runs in O(n) November 13, 2003 15

Prim-Jarnik’s Running Time n n Time = |V|T(extract. Min) + O(E)T(modify. Key) Binary heap implementation: n Time = O(V lg. V + E lg. V) = O(E lg. V) Q array binary heap Fibonacci heap November 13, 2003 T(extract. Min) O( V ) O(lg V) T(modify. Key) O(1) O(lg V) O(1) amortized Total O( V 2) O(E lg. V ) O(V lg. V +E ) 16

Greedy Algorithms n n One more algorithm design technique Greedy algorithms can be used to solve optimization problems, if: n n There is an optimal substructure We can prove that a greedy choice at each iteration leads to an optimal solution. November 13, 2003 17

Kruskal's Algorithm n n n Edge based algorithm Add the edges one at a time, in increasing weight order The algorithm maintains A – a forest of trees. An edge is accepted it if connects vertices of distinct trees (the cut respects A) November 13, 2003 18

Disjoint-Set ADT n We need a Disjoint-Set ADT that maintains a partition, i. e. , a collection S of disjoint sets. Operations: n make. Set(x: Vertex): Set. ID n n union(x: Vertex, y: Vertex) – n n S ¬ S È {{x}} X ¬ S. find. Set(x) Y ¬ S. find. Set(y) S ¬ S – { X , Y} È { X È Y} find. Set(x: Vertex): Set. ID n returns unique X Î S, where x Î X November 13, 2003 19

Kruskal's Algorithm n The algorithm keeps adding the cheapest edge that connects two trees of the forest MST-Kruskal(G) 01 A ¬ Æ 02 S. init() // Init disjoint-set ADT 03 for each vertex v Î G. V() 04 S. make. Set(v) 05 sort the edges of G. E() by non-decreasing G. w(u, v) 06 for each edge (u, v) Î E, in sorted order do 07 if S. find. Set(u) ¹ S. find. Set(v) then 08 A ¬ A È {(u, v)} 09 S. union(u, v) 10 return A November 13, 2003 20

Kruskal’s Example n Let’s do an example: B 4 A 12 8 8 C 3 6 H 4 D 5 3 9 13 I 1 n 6 F 10 E G Let’s keep track: n What is set A, what is a collection S , what cuts are we making November 13, 2003 21

Disjoint Sets as Lists n n n Each set – a list of elements identified by the first element, all elements in the list point to the first element Union – add a smaller list to a larger one Find. Set: O(1), Union(u, v): O(min{|C(u)|, |C(v)|}) 1 2 3 4 A B C Æ 1 2 November 13, 2003 3 4 Æ Æ 22

Kruskal Running Time n n Initialization O(V) time Sorting the edges Q(E lg E) = Q(E lg V) (why? ) O(E) calls to Find. Set Union costs n n Let t(v) be the number of times v is moved to a new cluster Each time a vertex is moved to a new cluster the size of the cluster containing the vertex at least doubles: t(v) £ log V Total time spent doing Union Total time: O(E lg V) November 13, 2003 23

Next Lecture n Shortest Paths in Weighted Graphs November 13, 2003 24