Data Structures Algorithms Shortest Paths Richard Newman based

Data Structures & Algorithms Shortest Paths Richard Newman based on book by R. Sedgewick and slides by S. Sahni

Minimum Cost Path • • Weighted, digraph G, or network Directed simple path p from s to t Cost(p) = sum of edge weights on p Minimum cost path p from s to t in G such that no other path p' from s to t in G has cost(p') < cost(p)

Minimum Cost Path 2 4 3 8 8 1 6 2 10 4 4 4 5 5 9 6 2 6 5 7 7 s = 1, t = 10 What is minimum cost path? 8 3 11

A Spanning Tree 2 4 3 8 8 1 6 2 10 4 4 4 5 5 9 6 2 6 5 7 7 path cost = 28 Is there a cheaper path? 8 3 11

A Spanning Tree 2 4 3 8 8 1 6 2 10 4 4 4 5 5 9 6 2 6 5 7 7 path cost = 25 Is there a cheaper path? 8 3 11

A Spanning Tree 2 4 3 8 8 1 6 2 10 4 4 4 5 5 9 6 2 6 5 7 7 path cost = 24 Is there a cheaper path? 8 3 11

Shortest Path Problems • No negative weight edges allowed! • s-t shortest path – Single source, destination – Fastest route to Epcot • Single Source Shortest Path – Best routes from A to anywhere • All Pairs Shortest Paths – Routing tables in network nodes

Shortest Path Algorithms • Dijkstra's Algorithm • Floyd's Algorithm • Bellman-Ford Algorithm

Shortest Path Algorithms • Dijkstra's Algorithm • Floyd's Algorithm • Bellman-Ford Algorithm

Dijkstra's Algorithm • Very similar to Prim's algorithm • Differences are that – Paths form a rooted tree (whereas MST is not rooted) – Graph is directed (not undirected) • Grow known shortest paths from source one edge at a time • Priority of edge is different

Dijkstra's Algorithm • Set Known nodes K = {s} • Set dist(s) = 0, dist(u) = ∞ for all other nodes u in G • Set pred(s) = s, pred(u) = NULL for all other nodes u in G • Set Seen nodes S = {neighbors of s} • Set pred(u) = s for all nodes in S

Dijkstra's Algorithm While |K| < V Find nearest node v in S Add v to K For each edge (v, w) in E If dist(v) + cost(v, w) < dist(w) Add w to S Pred(w) = v dist(w) = dist(v) + cost(v, w)

Dijkstra's Algorithm 4 4 0 1 2 7 5 2 2 11 8 3 6 17 6 8 5 4 4 129 5 7 6 4 5 6 2 7 13 9 22 10 5 8 11 3 19 21 5 14 7 Update best distances s=1 Grow SPT by adding node nearest to s

Dijkstra's Algorithm Correct Builds shortest path tree (SPT) Always adds nearest seen node v Path takes into account all nodes in K No other node x not in K can be closer than v • Hence no path through x could be shorter than the path we have to v • •

Dijkstra's Algorithm Complexity • • • Initialization – O(V) Select next node – O(V) linear list Update dist, pred – O(E) total Selection done V-1 times Total time – O(V 2 + E) = O(V 2) Using linear list for S

Dijkstra's Algorithm Complexity • Min. Heap – O((V+E) lg V) • V removals, E changes to S • What if G is dense – E is O(V 2)? • Worse!! • Fibonacci Heap – O(V lg V + E) • Even better!

Dijkstra's Algorithm • Solves single source shortest path • Builds SPT • For s-t shortest path, • Just stop when t is added to K • Also works for sink-trees (take edges in reverse direction)

Dijkstra's Algorithm • Does NOT work with negative edge weights! (Violates assumption needed by greedy method) • Can be used to solve all pairs shortest path • Build SPT from each node • Complexity of SSSP times V

Shortest Path Algorithms • Dijkstra's Algorithm • Floyd's Algorithm • Bellman-Ford Algorithm

Shortest Path Algorithms • Floyd's Algorithm • Single source shortest paths • Works like Warshall’s algorithm for reachability • Except takes path costs into account

Floyd’s Algorithm • Recall Warshall's Algorithm For each intermediate node i For each source s For each destination t s reaches t if s already reaches t or if s reaches i and i reaches t

Floyd’s Algorithm • Now just track distances For each intermediate node i For each source s For each destination t cost(s, t) = lesser of cost(s, t) and cost(s, i) + cost(i, t)

Floyd’s Algorithm • • Complexity of Floyd’s algorithm: O(V 3) Dynamic Programming and Relaxation • Build estimates • Improve estimates (node relax) • Converge

Floyd's Algorithm 0 41 29 45 36 3 1 0 19 5 21 36 0 32 51 1 0 1 2 3 3 32 50 0 29 51 0 45 5 32 50 0 4 2 5 41 2 4 4 32 29 Consider paths through nodes numbered <1, <2, <3, etc. 36 36 0 21 0

Floyd's Algorithm 0 41 29 45 36 3 1 0 19 5 21 36 0 32 51 1 0 1 2 3 0 32 50 5 51 86 4 2 5 29 0 45 4 41 2 4 3 32 50 0 32 36 29 Paths through nodes numbered <1 3 -0 -1 better, 3 -0 -5 worse 36 0 21 0

Floyd's Algorithm 0 41 29 45 36 3 1 0 29 5 21 36 0 32 51 1 0 1 2 3 32 50 73 0 51 32 86 137 4 2 5 5 92 0 45 4 41 2 4 3 32 29 29 50 0 118 36 80 Paths through nodes numbered <2 29+32 = 61 > 21 36 0 21 0

Floyd's Algorithm 0 41 29 45 36 3 1 0 29 5 21 36 0 32 51 1 0 1 2 3 32 50 73 0 51 101 32 86 137 4 2 5 5 92 142 0 45 4 41 2 4 3 32 29 29 50 0 118 36 0 80 130 21 Paths through nodes numbered <3 36 0

Floyd's Algorithm 0 41 29 45 36 3 1 0 29 5 21 36 32 51 4 32 50 2 1 2 3 4 5 0 0 41 92 142 73 1 146 0 51 101 32 137 2 95 136 3 45 4 81 122 5 175 29 0 50 168 86 86 137 0 118 36 29 32 36 0 72 80 130 21 0 Paths through nodes numbered <4

Floyd's Algorithm 0 41 29 45 36 3 1 0 29 5 21 36 32 51 4 32 50 2 1 2 0 0 41 1 113 3 4 5 92 109 73 0 51 32 106 2 95 136 0 50 168 88 3 45 86 137 0 118 36 4 81 122 5 102 29 68 29 32 36 0 72 53 57 21 0 Paths through nodes numbered <5

Floyd's Algorithm 0 41 29 45 36 3 1 0 29 5 21 36 32 51 4 32 50 2 1 2 3 4 5 0 0 41 82 86 50 1 113 0 51 68 32 106 2 95 117 0 3 45 65 89 0 57 36 4 81 101 32 36 0 72 53 57 21 0 5 102 29 50 109 Paths through nodes numbered <6 29 88

Floyd’s Algorithm • Update successor node when updating minimum cost path • Method of choice for APSP for dense graphs • Works even with negative weights • But not with negative cycles! • (can detect at least one)

Shortest Path Algorithms • Dijkstra's Algorithm • Floyd's Algorithm • Bellman-Ford Algorithm

Shortest Path Algorithms • Bellman-Ford Algorithm • SSSP • Compute best cost path by edge relaxation – checking all edges • Essentially considers paths of increasing potential length

Bellman-Ford Algorithm Initialize: dist[t] = {0 if t==s, else infinity} pred[v] = NULL for all v For (i = 1 to V-1) For each edge e=(u, v) in E If (dist[u] + cost[e] < dist[v]) dist[v] = dist[u] + cost[e] pred[v] = u

Bellman-Ford Algorithm 0 41 29 45 36 3 1 0 19 5 21 36 0 32 51 1 0 1 2 3 32 50 0 5 5 29 51 0 45 32 50 0 4 2 4 41 2 4 3 32 29 36 36 0 21 Consider paths of length <2, <3, etc. Adjacency matrix is paths of length <2 0

Bellman-Ford Algorithm 0 41 29 45 36 3 Leave here 1 0 19 5 21 36 32 51 4 Edge (0, 1) 0 1 2 3 2 0 45 4 5 29 51 0 32 50 86 4 5 3 41 2 32 50 0 1 0 32 36 29 Enter here 36 0 21 0

Bellman-Ford Algorithm 0 41 29 45 36 3 1 0 19 5 21 36 32 51 4 0 1 2 3 2 45 92 0 51 Edge (0, 5) doesn’t help Edge (1, 2) 4 32 50 86 137 0 32 36 29 80 5 29 0 4 5 3 41 2 32 50 0 1 36 0 21 0

Bellman-Ford Algorithm 0 41 29 45 36 3 1 0 19 5 21 36 32 51 4 Edge (2, 3) 0 1 2 3 2 45 4 92 142 0 51 101 5 29 32 0 50 86 137 0 32 36 0 80 130 21 4 5 3 41 2 32 50 0 1 29 36 0

Bellman-Ford Algorithm 0 41 29 45 36 3 1 0 19 5 21 36 32 51 4 32 50 Edge (3, 0) 2 1 2 3 4 5 0 0 41 92 142 1 146 0 51 101 2 95 0 50 3 45 86 137 0 4 81 32 36 0 5 175 80 130 21 29 29 32 36 0

Bellman-Ford Algorithm 0 41 29 45 36 3 1 0 19 5 21 36 32 51 4 32 50 Edge (3, 5) 2 1 2 3 4 5 0 0 41 92 142 29 1 146 0 51 101 32 137 2 95 0 50 86 3 45 86 137 0 36 4 81 32 36 0 72 5 175 80 130 21 0 29

Bellman-Ford Algorithm 0 41 29 45 36 3 1 0 19 5 21 36 32 51 4 32 50 Edge (4, 2) 2 1 2 3 4 5 0 0 41 92 142 29 1 146 0 64 101 32 137 2 95 0 50 86 3 45 86 137 0 36 4 81 32 36 0 72 5 175 53 130 21 0 29

Bellman-Ford Algorithm 0 41 29 45 36 3 1 0 19 5 21 36 32 51 4 32 50 Edge (4, 3) 2 1 2 0 0 41 1 146 0 2 3 4 5 92 128 29 64 68 32 137 95 0 50 86 3 45 86 137 0 36 4 81 32 36 0 72 5 175 53 57 21 0 29

Bellman-Ford Algorithm 0 41 29 45 36 3 1 0 19 5 21 36 32 51 4 32 50 Edge (5, 1) 2 1 2 3 4 5 0 0 41 1 146 0 64 68 32 137 2 95 115 0 50 86 3 45 65 137 0 36 4 81 101 5 175 29 92 128 29 32 36 0 72 53 57 21 0

Bellman-Ford Algorithm 0 41 29 45 36 3 1 0 19 5 21 36 32 51 4 32 50 Edge (5, 4) 2 1 2 0 0 41 1 146 3 4 5 92 128 50 0 64 32 137 2 95 115 0 3 45 4 81 101 5 175 65 137 29 68 50 107 29 86 0 57 36 32 36 0 72 53 57 21 0

Bellman-Ford Algorithm • Well, that was ONE pass – need to do that V times!

Shortest Path Algorithms • Dijkstra's Algorithm • Floyd's Algorithm • Bellman-Ford Algorithm