CSE 421 Algorithms Richard Anderson Dijkstras algorithm Single

  • Slides: 52
Download presentation
CSE 421 Algorithms Richard Anderson Dijkstra’s algorithm

CSE 421 Algorithms Richard Anderson Dijkstra’s algorithm

Single Source Shortest Path Problem • Given a graph and a start vertex s

Single Source Shortest Path Problem • Given a graph and a start vertex s – Determine distance of every vertex from s – Identify shortest paths to each vertex • Express concisely as a “shortest paths tree” • Each vertex has a pointer to a predecessor on shortest path 1 1 u 2 5 s 3 3 x 4 v u s x 3 v

Construct Shortest Path Tree from s d 2 a 4 s 1 5 4

Construct Shortest Path Tree from s d 2 a 4 s 1 5 4 -3 d 4 e c 3 2 7 e -2 3 g 6 b a f c s g b 3 f

Warmup • If P is a shortest path from s to v, and if

Warmup • If P is a shortest path from s to v, and if t is on the path P, the segment from s to t is a shortest path between s and t t • WHY? s v

Assume all edges have non-negative cost Dijkstra’s Algorithm S = {}; d[s] = 0;

Assume all edges have non-negative cost Dijkstra’s Algorithm S = {}; d[s] = 0; d[v] = infinity for v != s While S != V Choose v in V-S with minimum d[v] Add v to S For each w in the neighborhood of v d[w] = min(d[w], d[v] + c(v, w)) 0 s 1 u 1 2 4 2 v 4 y 3 1 1 x 2 2 3 2 z 5

Simulate Dijkstra’s algorithm (strarting from s) on the graph Round Vertex Added a 1

Simulate Dijkstra’s algorithm (strarting from s) on the graph Round Vertex Added a 1 s 3 1 1 c 2 1 4 3 4 6 b 1 3 2 d 4 5 s a b c d

Dijkstra’s Algorithm as a greedy algorithm • Elements committed to the solution by order

Dijkstra’s Algorithm as a greedy algorithm • Elements committed to the solution by order of minimum distance

Correctness Proof • Elements in S have the correct label • Key to proof:

Correctness Proof • Elements in S have the correct label • Key to proof: when v is added to S, it has the correct distance label. y x s u v

Proof • Let v be a vertex in V-S with minimum d[v] • Let

Proof • Let v be a vertex in V-S with minimum d[v] • Let Pv be a path of length d[v], with an edge (u, v) • Let P be some other path to v. Suppose P first leaves S on the edge (x, y) – – P = Psx + c(x, y) + Pyv Len(Psx) + c(x, y) >= d[y] Len(Pyv) >= 0 Len(P) >= d[y] + 0 >= d[v] y x s u v

Negative Cost Edges • Draw a small example a negative cost edge and show

Negative Cost Edges • Draw a small example a negative cost edge and show that Dijkstra’s algorithm fails on this example

Bottleneck Shortest Path • Define the bottleneck distance for a path to be the

Bottleneck Shortest Path • Define the bottleneck distance for a path to be the maximum cost edge along the path u 6 s 2 3 5 x 5 4 v

Compute the bottleneck shortest paths d 6 a 4 s 6 5 4 -3

Compute the bottleneck shortest paths d 6 a 4 s 6 5 4 -3 d 4 e c 3 2 7 e -2 3 g 6 b a f c s g b 4 f

How do you adapt Dijkstra’s algorithm to handle bottleneck distances • Does the correctness

How do you adapt Dijkstra’s algorithm to handle bottleneck distances • Does the correctness proof still apply?

Who was Dijkstra? • What were his major contributions?

Who was Dijkstra? • What were his major contributions?

http: //www. cs. utexas. edu/users/EWD/ • Edsger Wybe Dijkstra was one of the most

http: //www. cs. utexas. edu/users/EWD/ • Edsger Wybe Dijkstra was one of the most influential members of computing science's founding generation. Among the domains in which his scientific contributions are fundamental are – – – – algorithm design programming languages program design operating systems distributed processing formal specification and verification design of mathematical arguments

Shortest Paths • Negative Cost Edges – Dijkstra’s algorithm assumes positive cost edges –

Shortest Paths • Negative Cost Edges – Dijkstra’s algorithm assumes positive cost edges – For some applications, negative cost edges make sense – Shortest path not well defined if a graph has a negative cost cycle a 6 4 -4 -3 s 4 e c 3 2 -2 3 6 b g f 7 4

Negative Cost Edge Preview • Topological Sort can be used for solving the shortest

Negative Cost Edge Preview • Topological Sort can be used for solving the shortest path problem in directed acyclic graphs • Bellman-Ford algorithm finds shortest paths in a graph with negative cost edges (or reports the existence of a negative cost cycle).

Dijkstra’s Algorithm Implementation and Runtime S = {}; d[s] = 0; d[v] = infinity

Dijkstra’s Algorithm Implementation and Runtime S = {}; d[s] = 0; d[v] = infinity for v != s While S != V Choose v in V-S with minimum d[v] Add v to S For each w in the neighborhood of v d[w] = min(d[w], d[v] + c(v, w)) y u s a n Extract Mins x v z HEAP OPERATIONS b m Heap Updates Edge costs are assumed to be non-negative

Bottleneck Shortest Path • Define the bottleneck distance for a path to be the

Bottleneck Shortest Path • Define the bottleneck distance for a path to be the maximum cost edge along the path u 6 s 2 3 5 x 5 4 v

Compute the bottleneck shortest paths d 6 a 4 s 6 5 4 -3

Compute the bottleneck shortest paths d 6 a 4 s 6 5 4 -3 d 4 e c 3 2 7 e -2 3 g 5 b a f c s g b 7 f

Dijkstra’s Algorithm for Bottleneck Shortest Paths S = {}; d[s] = negative infinity; d[v]

Dijkstra’s Algorithm for Bottleneck Shortest Paths S = {}; d[s] = negative infinity; d[v] = infinity for v != s While S != V Choose v in V-S with minimum d[v] Add v to S For each w in the neighborhood of v d[w] = min(d[w], max(d[v], c(v, w))) 1 u s 2 v y 4 2 4 3 x 1 5 4 3 z a 1 3 b

Minimum Spanning Tree • Introduce Problem • Demonstrate three different greedy algorithms • Provide

Minimum Spanning Tree • Introduce Problem • Demonstrate three different greedy algorithms • Provide proofs that the algorithms work

Minimum Spanning Tree 15 t a 6 9 14 3 10 13 s 17

Minimum Spanning Tree 15 t a 6 9 14 3 10 13 s 17 1 e c 20 2 5 7 b u 4 8 12 16 v 11 g f 22

Greedy Algorithms for Minimum Spanning Tree • Extend a tree by including the cheapest

Greedy Algorithms for Minimum Spanning Tree • Extend a tree by including the cheapest out going edge • Add the cheapest edge that joins disjoint components • Delete the most expensive edge that does not disconnect the graph 4 b 20 c 5 11 8 a d 7 e 22

Greedy Algorithm 1 Prim’s Algorithm • Extend a tree by including the cheapest out

Greedy Algorithm 1 Prim’s Algorithm • Extend a tree by including the cheapest out going edge 15 t 3 10 Construct the MST with Prim’s algorithm starting from vertex a Label the edges in order of insertion 14 13 1 4 e c 20 2 5 7 b u 6 9 s 17 a 8 12 16 v 11 g f 22

Greedy Algorithm 2 Kruskal’s Algorithm • Add the cheapest edge that joins disjoint components

Greedy Algorithm 2 Kruskal’s Algorithm • Add the cheapest edge that joins disjoint components 15 t 3 10 Construct the MST with Kruskal’s algorithm Label the edges in order of insertion 14 13 1 4 e c 20 2 5 7 b u 6 9 s 17 a 8 12 16 v 11 g f 22

Greedy Algorithm 3 Reverse-Delete Algorithm • Delete the most expensive edge that does not

Greedy Algorithm 3 Reverse-Delete Algorithm • Delete the most expensive edge that does not disconnect the graph 15 t 3 10 Construct the MST with the reversedelete algorithm Label the edges in order of removal 14 13 1 4 e c 20 2 5 7 b u 6 9 s 17 a 8 12 16 v 11 g f 22

Why do the greedy algorithms work? • For simplicity, assume all edge costs are

Why do the greedy algorithms work? • For simplicity, assume all edge costs are distinct • Let S be a subset of V, and suppose e = (u, v) is the minimum cost edge of E, with u in S and v in V-S • e is in every minimum spanning tree

Proof • Suppose T is a spanning tree that does not contain e •

Proof • Suppose T is a spanning tree that does not contain e • Add e to T, this creates a cycle • The cycle must have some edge e 1 = (u 1, v 1) with u 1 in S and v 1 in V-S • T 1 = T – {e 1} + {e} is a spanning tree with lower cost • Hence, T is not a minimum spanning tree

Optimality Proofs • Prim’s Algorithm computes a MST • Kruskal’s Algorithm computes a MST

Optimality Proofs • Prim’s Algorithm computes a MST • Kruskal’s Algorithm computes a MST

Reverse-Delete Algorithm • Lemma: The most expensive edge on a cycle is never in

Reverse-Delete Algorithm • Lemma: The most expensive edge on a cycle is never in a minimum spanning tree

Dealing with the assumption of no equal weight edges • Force the edge weights

Dealing with the assumption of no equal weight edges • Force the edge weights to be distinct – Add small quantities to the weights – Give a tie breaking rule for equal weight edges

Dijkstra’s Algorithm for Minimum Spanning Trees S = {}; d[s] = 0; d[v] =

Dijkstra’s Algorithm for Minimum Spanning Trees S = {}; d[s] = 0; d[v] = infinity for v != s While S != V Choose v in V-S with minimum d[v] Add v to S For each w in the neighborhood of v d[w] = min(d[w], c(v, w)) 1 u s 2 v y 4 2 4 3 x 1 5 4 3 z a 1 3 b

Minimum Spanning Tree 15 t a 6 9 14 3 10 13 s 17

Minimum Spanning Tree 15 t a 6 9 14 3 10 13 s 17 1 4 e c 20 2 5 7 b u Undirected Graph G=(V, E) with edge weights 8 12 16 v 11 g f 22

Greedy Algorithms for Minimum Spanning Tree • [Prim] Extend a tree by including the

Greedy Algorithms for Minimum Spanning Tree • [Prim] Extend a tree by including the cheapest out going edge • [Kruskal] Add the cheapest edge that joins disjoint components • [Reverse. Delete] Delete the most expensive edge that does not disconnect the graph 4 b 20 c 5 11 8 a d 7 e 22

Why do the greedy algorithms work? • For simplicity, assume all edge costs are

Why do the greedy algorithms work? • For simplicity, assume all edge costs are distinct

Edge inclusion lemma • Let S be a subset of V, and suppose e

Edge inclusion lemma • Let S be a subset of V, and suppose e = (u, v) is the minimum cost edge of E, with u in S and v in V-S • e is in every minimum spanning tree of G – Or equivalently, if e is not in T, then T is not a minimum spanning tree e S V-S

e is the minimum cost edge between S and V-S Proof • Suppose T

e is the minimum cost edge between S and V-S Proof • Suppose T is a spanning tree that does not contain e • Add e to T, this creates a cycle • The cycle must have some edge e 1 = (u 1, v 1) with u 1 in S and v 1 in V-S S e V-S • T 1 = T – {e 1} + {e} is a spanning tree with lower cost • Hence, T is not a minimum spanning tree

Optimality Proofs • Prim’s Algorithm computes a MST • Kruskal’s Algorithm computes a MST

Optimality Proofs • Prim’s Algorithm computes a MST • Kruskal’s Algorithm computes a MST • Show that when an edge is added to the MST by Prim or Kruskal, the edge is the minimum cost edge between S and V-S for some set S.

Prim’s Algorithm S = { }; T = { }; while S != V

Prim’s Algorithm S = { }; T = { }; while S != V choose the minimum cost edge e = (u, v), with u in S, and v in V-S add e to T add v to S

Prove Prim’s algorithm computes an MST • Show an edge e is in the

Prove Prim’s algorithm computes an MST • Show an edge e is in the MST when it is added to T

Kruskal’s Algorithm Let C = {{v 1}, {v 2}, . . . , {vn}};

Kruskal’s Algorithm Let C = {{v 1}, {v 2}, . . . , {vn}}; T = { } while |C| > 1 Let e = (u, v) with u in Ci and v in Cj be the minimum cost edge joining distinct sets in C Replace Ci and Cj by Ci U Cj Add e to T

Prove Kruskal’s algorithm computes an MST • Show an edge e is in the

Prove Kruskal’s algorithm computes an MST • Show an edge e is in the MST when it is added to T

Reverse-Delete Algorithm • Lemma: The most expensive edge on a cycle is never in

Reverse-Delete Algorithm • Lemma: The most expensive edge on a cycle is never in a minimum spanning tree

Dealing with the assumption of no equal weight edges • Force the edge weights

Dealing with the assumption of no equal weight edges • Force the edge weights to be distinct – Add small quantities to the weights – Give a tie breaking rule for equal weight edges

Application: Clustering • Given a collection of points in an rdimensional space, and an

Application: Clustering • Given a collection of points in an rdimensional space, and an integer K, divide the points into K sets that are closest together

Distance clustering • Divide the data set into K subsets to maximize the distance

Distance clustering • Divide the data set into K subsets to maximize the distance between any pair of sets – dist (S 1, S 2) = min {dist(x, y) | x in S 1, y in S 2}

Divide into 2 clusters

Divide into 2 clusters

Divide into 3 clusters

Divide into 3 clusters

Divide into 4 clusters

Divide into 4 clusters

Distance Clustering Algorithm Let C = {{v 1}, {v 2}, . . . ,

Distance Clustering Algorithm Let C = {{v 1}, {v 2}, . . . , {vn}}; T = { } while |C| > K Let e = (u, v) with u in Ci and v in Cj be the minimum cost edge joining distinct sets in C Replace Ci and Cj by Ci U Cj

K-clustering

K-clustering