Greedy Algorithms MST min spanning tree Minimum Spanning

Greedy Algorithms

MST (min spanning tree)

Minimum Spanning Trees In a weighted, undirected graph, it is a tree formed by connecting all of the vertices with minimal cost. ◦ The MST is a tree because it’s acyclic. ◦ It’s spanning because it covers every vertex. ◦ And it’s minimum because it has minimum cost. 2 v 4 1 1 2 v 3 2 3 4 v 5 4 v 6 v 1 6 1 2 3 7 2 v v 10 7 v 8 5 v v 2 1 v 4 v 6 v 5 4 6 v 1 7

MSTs – proof that greedy works Let G be a graph with vertices in the set V partitioned into two sets V 1 and V 2. Then the minimum weight edge, e, that connects a vertex from V 1 to V 2 is part of a minimum spanning tree of G. Proof: Consider a MST T of G that does NOT contain the minimum weight edge e. ◦ This MUST have at least one edge in between a vertex from V 1 to V 2. (Otherwise, no vertices between those two sets would be connected. ) ◦ Let G contain edge f that connects V 1 to V 2. ◦ Now, add in edge e to T. ◦ This creates a cycle. In particular, there was already one path from every vertex in V 1 to V 2 and with the addition of e, there are two. ◦ Thus, we can form a cycle involving both e and f. Now, imagine removing f from this cycle. ◦ This new graph, T' is also a spanning tree, but it's total weight is less than or equal to T because we replaced e with f, and e was the minimum weight edge.

MSTs – proof that greedy works As a spanning tree is created ◦ If the edge that is added is the one of minimum cost that avoids creation of a cycle. ◦ Then the cost of the resulting spanning tree cannot be improved Because any replacement edge would have cost at least as much as an edge already in the spanning tree. This is why greedy works!

Kruskal’s Algorithm Let V = For i=1 to n-1, (where there are n vertices in a graph) V = V e, where e is the edge with the minimum edge weight not already in V, and that does NOT form a cycle when added to V. Return V Basically, you build the MST of the graph by continually adding in the smallest weighted edge into the MST that doesn't form a cycle. ◦ When you are done, you'll have an MST. ◦ You HAVE to make sure you never add an edge the forms a cycle and that you always add the minimum of ALL the edges left that don't.

Kruskal’s Given Graph G: Edges in sorted order: 4 v v 4 1 v v 4 v v 5 v 1 v 2 v v 3 4 v 3 7 v 8 4 v 4 1 v 6 v 7 1 2 v 1 v v 2 2 3 1 CYCLE! v 3 v v 6 Determine the MST: 4 5 2 v 5 4 8 v 4 7 6 7 2 10 v 6 2 v 2 3 v 6 5 7 1 v 4 5 3 1 6 7 v 1 2 v 3 v 2 v 4 4 4 5 6 v v 2 6 10 v v 3 5 v 1 7 All Vertices, we’re done!!

Kruskal’s The reason this works: ◦ is that each added edge is connecting between two sets of vertices, ◦ and since we select the edges in order by weight, ◦ we are always selecting the minimum edge weight that connects the two sets of vertices. Cycle detection: ◦ Keep track of disjoint sets. ◦ Initially, each vertex is in its own disjoint set. ◦ When you add an edge you are unioning two sets. ◦ A union cannot happen if the two vertices are already in the same set.

Prim’s Algorithm This is quite similar to Kruskal's with one big difference: ◦ The tree that you are "growing" ALWAYS stays connected. Whereas in Kruskal's you could add an edge to your growing tree that wasn't connected to the rest of it, here you can NOT do it. Here is the algorithm: 1) Set S = . 1) Pick any vertex in the graph. 2) Add the minimum edge incident to that vertex to S. 3) Continue to add edges into S (n-2 more times) using the following rule: Add the minimum edge weight to S that is incident to S but that doesn't form a cycle when added to S.

Prim’s Given Graph G: Edges in sorted order: v 4 1 v v 4 1 6 5 7 v 6 v v 6 2 v 5 v 1 v 2 v v 3 4 v 3 7 v 8 4 v 4 1 v 6 1 v v 2 2 3 1 CYCLE! v 3 v v 2 v 4 5 2 Determine the MST, using Prim’s starting with vertex. V 1: v 4 v 7 1 6 7 2 v 5 4 6 3 v 10 7 8 7 v 2 3 v 4 5 v 1 2 v 3 4 1 v v 1 4 v 2 v 4 4 4 5 6 v v 2 6 10 v v 3 5 v 1 7 All Vertices, we’re done!!

Huffman Encoding Compress the storage of data using variable length codes. ◦ For example, each character in a text file is stored using 8 bits. ◦ Nice and easy because we always read in 8 bits for a single character. Not the most efficient… ◦ What if ‘e’ is used 10 times more frequently than ‘q’. ◦ It would be more advantageous for us to use a 7 bit code for e and a 9 bit code for q.

Huffman Coding Finds the optimal way to take advantage of varying character frequencies in a particular file. ◦ On average, standard files can shrink them anywhere from 10% to 30% depending on the character distribution. The idea behind the coding is to give less frequent characters and groups of characters longer codes. Also, the coding is constructed in such a way that no two constructed codes are prefixes of each other. This property about the code is crucial with respect to easily deciphering the code.

Dijkstra’s Algorithm

Dijkstra’s Algorithm This algorithm finds the shortest path from a source vertex to all other vertices in a weighted directed graph without negative edge weights. 5 a 10 2 3 2 b 8 3 16 4 4 12 e d c 13

Dijkstra’s Algorithm For a Graph G ◦ Vertices V = {v 1, … vn} ◦ And edge weights wij, for edge connecting vi to vj. ◦ Let the source be v 1. Initialize a Set S = . ◦ Keep track of the vertices for which we’ve already computed their shortest path from the source. Initialize an array D of estimates of shortest distances. ◦ Initialize D[source]=0, everything else D[i] = . ◦ This says our estimate from the source to the source is 0, everything else is initially. While S!=V (or while we haven’t considered all vertices): 1) Find the vertex with the minimum dist (not already in S). 2) Add this vertex, vi to S 3) Recompute all estimates based on vi. Specifically compute D[i]+wij. If this < D[j] then set D[j] = D[i]+wij

Dijkstra’s Algorithm Dijkstra’s algorithm The shortest path from a to b is 10, relies on: Knowing that all shortest paths contain subpaths that are also shortest paths. (Convince yourself of this) a 8 10 c b 4 ◦ so the shortest path from a to d has to go through b ◦ so it will also contain the shortest path from a to b which is 10, ◦ plus the additional 2 = 12. 2 d

Greedy Algorithms Greedy algorithms work in phases. ◦ In each phase, a decision is made that appears to be good, without regard for future consequences. ◦ “Take what you can get now” ◦ When the algorithm terminates, we hope that the local optimum is equal to the global optimum. That’s why Dijkstra’s works out well as a greedy algorithm ◦ It’s greedy because we assume we have a shortest distance to a vertex before we ever examine all the edges that even lead into that vertex. ◦ It works since all shortest paths contain subpaths that are also shortest paths. ◦ This also works because we assume no negative edge weights.

References Slides adapted from Arup Guha’s Computer Science II Lecture notes: http: //www. cs. ucf. edu/~dmarino/ucf/cop 350 3/lectures/ Additional material from the textbook: Data Structures and Algorithm Analysis in Java (Second Edition) by Mark Allen Weiss Additional images: www. wikipedia. com xkcd. com