GRAPH ALGORITHMS BASICS Graph G V E V

GRAPH ALGORITHMS BASICS • Graph: G = {V, E} • V ={v 1, v 2, …, vn}, E = {e 1, e 2, … em}, eij=(vi, vj, lij) • A set of n nodes/vertices, and m edges/arcs as pairs of nodes: Problem sizes: n, m • When lij is present, it is a label, if it is a number it is the weight of an edge. V ={v 1, v 2, v 3, v 4, v 5} E = {e 1=(v 1, v 2), (v 3, v 1), (v 3, v 4), (v 5, v 2), e 5=(v 4, v 5)} v 1 v 3 v 2 degree(v 1)=2, degree(v 2)=2, … : number of arcs For directed graph: indegree, and outdegree may differ v 4 9/15/2020 v 5 (C) Debasis Mitra

GRAPH ALGORITHMS BASICS Adjacency List Representation: v 1: (v 2, 10), (v 3, 20) v 2: (v 1, 10), (v 5, 25) v 3: (v 1, 20), (v 4, 30) v 4: (v 3, 30), (v 5, 50) v 5: (v 2, 25), (v 4, 50) Weighted graph G: V ={v 1, v 2, v 3, v 4, v 5} E = {(v 1, v 2, 10), (v 3, v 1, 20), (v 3, v 4 , 30), (v 5, v 2 , 25), (v 4, v 5 , 50)} v 1 10 20 v 3 v 2 30 25 v 4 9/15/2020 50 v 5 v 1 v 2 v 3 v 4 v 5 Matrix representation: Matrix representation for directed Graph? Matrix representation for unweighted Graph? (C) Debasis Mitra v 1 0 10 20 Inf inf v 2 10 0 Inf 25 v 3 20 inf 0 30 inf v 4 Inf inf 30 0 50 v 5 inf 30 Inf 50 0

GRAPH ALGORITHMS BASICS • • • Directed graph: edges are ordered pairs of nodes. Weighted graph: each edge (directed/undirected) has a weight. Path between a pair of nodes vi, vk: sequence of edges with vi, vk at the two ends. Simple path: covers no node in it twice. Loop: a path with the same start and end node. Path length: number of edges in it. Path weight: total wt of all edges in it. Connected graph: there exists a path between every pair of nodes, no node is disconnected. Complete graph: edge between every pair of nodes [NUMBER OF EDGES? ]. Acyclic graph: a graph with no cycles. Etc. Graphs are one of the most used models of real-life problems for computersolutions. 9/15/2020 (C) Debasis Mitra

GRAPH ALGORITHMS BASICS v 1 • • • • • Algorithm 1: 10 For each node v in V do -Steps// (|V|) v 2 Algorithm 2: 25 For each node v in V do For each edge in E do -Steps- // (|V|*|E|) v 4 Algorithm 3: For each node v in V do For each edge e adjacent to v do -Steps// ( |E| ) with adjacency list, // but (|V|* |V| ) for matrix representation Algorithm 4: For each node v in V do -steps. For each edge e of v do -Steps// ( |V|+|E| ) or ( max{|V|, |E| }) 9/15/2020 (C) Debasis Mitra 20 v 3 30 50 v 5

TOPOLOGICAL SORT Problem 1 • Input: directed acyclic graph • Output: sequentially order the nodes without violating any arc ordering. • Note: you may have to check for cycles - depending on the problem definition (input: directed graph). • An important data: indegree of each node - number of arcs coming in (#courses pre-requisite to "this" course). 9/15/2020 (C) Debasis Mitra

TOPOLOGICAL SORT Problem 1 • Input: directed acyclic graph • Output: sequentially order the nodes without violating any arc direction. • A first-pass Algorithm: – Assign indegree values to each node – In each iteration: Eliminate one of the vertices with indegree=0 and its associated (outgoing) arcs (thus, reducing indegrees of the adjacent nodes) - after assigning an ordering-number (as output of topological-sort ordering) to these nodes, – keep doing the last step for each of the n number-of-nodes. – If at any stage before finishing the loop, there does not exist any node with indegree=0, then a cycle exists! [WHY? ] • A naïve way of finding a vertex with indegree 0 is to scan the nodes that are still left in the graph. • As the loop runs for n times this leads to (n 2) algorithm. 9/15/2020 (C) Debasis Mitra

TOPOLOGICAL SORT Problem 1 • • • • • Input: A directed graph Output: A sorting on the node without violating directions, or Failure Algorithm naïve-topological-sort 1 For each node v ϵ V calculate indegree(v); // (|E|=m) 2 counter = 0; 3 While V≠ empty do // (|V|=n) 4 find a node v ϵ V such that indegree(v) = = 0; // have to scan all nodes here: (n) // total: (n) x (while loop’s n) = O(n 2) 5 if there is no such v then return(Failure) else 6 ordering-id(v) = ++counter; 7 V = V – v; 8 For each edge (v, w) ϵ E do //nodes adjacent to v //adjacent nodes: (m), total for all nodes O(n or m) 9 decrement indegree(w); 10 E = E – (v, w); end if; end while; End algorithm. Complexity: dominating term O(n 2) 9/15/2020 (C) Debasis Mitra

TOPOLOGICAL SORT Problem 1 v 3 v 2 v 4 9/15/2020 v 5 Indegree(v 1)=0 Indegree(v 2)=1 Indegree(v 3)=1 Indegree(v 4)=2 Indegree(v 5)=1 Ordering-id(v 1) =1 Eliminate node v 1, and edges (v 1, v 2), (v 1, v 3) from G Update indegrees of nodes v 2 and v 3 (C) Debasis Mitra

TOPOLOGICAL SORT Problem 1 v 3 v 2 v 4 v 5 Indegree(v 2)=0 Indegree(v 3)=0 Indegree(v 4)=2 Indegree(v 5)=1 Id(v 1) =1, Id(v 2)=2 Eliminate node v 2, and edges (v 2, v 5) from G Update indegrees of nodes v 5 9/15/2020 (C) Debasis Mitra

TOPOLOGICAL SORT Problem 1 v 3 Indegree(v 3)=0 Indegree(v 4)=2 Indegree(v 5)=0 Id(v 1) =1, Id(v 2)=2, Id(v 3)=3 v 4 9/15/2020 v 5 Eliminate node v 3, and edge (v 3, v 4) from G Update indegrees of nodes v 4 (C) Debasis Mitra

TOPOLOGICAL SORT Problem 1 Indegree(v 4)=1 Indegree(v 5)=0 Id(v 1) =1, Id(v 2)=2, Id(v 3)=3, Id(v 5)=4 v 4 9/15/2020 v 5 Eliminate node v 5, and edge (v 5, v 4) from G Update indegrees of nodes v 4 (C) Debasis Mitra

TOPOLOGICAL SORT Problem 1 Indegree(v 4)=0 id(v 1) =1, id(v 2)=2, id(v 3)=3, id(v 5)=4, id(v 4)=5 Eliminate node v 4 from G, and V is empty now Resulting sort: v 1, v 2, v 3, v 5, v 4 9/15/2020 (C) Debasis Mitra

TOPOLOGICAL SORT Problem 1 v 3 v 2 v 4 v 5 However, now, Indegree(v 1)=0 Indegree(v 2)=2 Indegree(v 3)=1 Indegree(v 4)=2 Indegree(v 5)=1 id(v 1) =1 Eliminate node v 1, and edges (v 1, v 2), (v 1, v 3) from G Update indegrees of nodes v 2 and v 3 9/15/2020 (C) Debasis Mitra

TOPOLOGICAL SORT Problem 1 v 3 v 2 v 4 v 5 Indegree(v 2)=1 Indegree(v 3)=0 Indegree(v 4)=2 Indegree(v 5)=1 id(v 1) =1, id(v 3)=2 Eliminate node v 3, and edge (v 3, v 4) from G Update indegrees of nodes v 4 9/15/2020 (C) Debasis Mitra

TOPOLOGICAL SORT Problem 1 Indegree(v 2)=1 Indegree(v 4)=1 Indegree(v 5)=1 v 2 No node to choose before all nodes are ordered: Fail v 4 9/15/2020 v 5 (C) Debasis Mitra

TOPOLOGICAL SORT Problem 1 • A smarter way: notice indegrees become zero for only a subset of nodes in a pass, but all nodes are being checked in the above naïve algorithm. • Store those nodes (who have become “roots” of the truncated graph) in a box and pass the box to the next iteration. • An implementation of this idea may be done by using a queue: • Push those nodes whose indegrees have become 0’s (when you update those values), at the back of a queue; • Algorithm terminates on empty queue, • but having empty Q before covering all nodes: • we have got a cycle! 9/15/2020 (C) Debasis Mitra

TOPOLOGICAL SORT Problem 1 Algorithm Q-based-topological-sort 1 For each vertex v in G do // initialization 2 calculate indegree(v); // (m) with adjacency list 3 if indegree(v) = =0 then push(Q, v); end for; 4 counter = 0; 5 While Q is not empty do // indegree becomes 0 once & only once for a node // so, each node goes to Q only once: (n) 6 v = pop(Q); 7 ordering-id(v) = ++counter; 8 V = V – v; 9 For each edge (v, w) ϵ E do //nodes adjacent to v // two loops together (max{N, |E|}) // each edge scanned once and only once 10 E = E – (v, w); 11 decrement indegree(w); 12 if now indegree(w) = =0 then push(Q, w); end for; end while; 13 if counter != N then return(Failure); // not all nodes are numbered yet, but Q is empty // cycle detected; End algorithm. 9/15/2020 (C) Debasis Mitra Complexity: the body of the inner for-loop is executed at most once per edge, even considering the outer while loop, if adjacency list is used. The maximum queue length is n. Complexity is (m + n).

SOURCE TO ALL NODES SHORTEST PATH-LENGTH Problem 2 Path length = number of edges on a path. Compute shortest path-length from a given source node to all nodes on the graph. Strategy: starting with the source as the "current-nodes, " in each of the iteration expand children (adjacent nodes) of "current-nodes. " Assign the iteration# as the shortest path length to each expanded node, if the value is not already assigned. Also, assign to each child, its parent node-id in this expansion tree (to retract the shortest path if necessary). Input: Undirected graph, and source node Output: Shortest path-length to each node from source This is a breadth-first traversal, nodes are expanded as increasing distances from the source: 0, then 1, then 2, etc. 9/15/2020 (C) Debasis Mitra

SOURCE TO ALL NODES SHORTEST PATH-LENGTH Problem 2 Once again, the children are pushed at the back of a queue. Input: Undirected graph, and source node Output: Shortest path-length to each node from source Algorithm q-based-shortest-path 1 d(s) = 0; // source 2 enqueue only s in Q; // (1), no loop, constant-time 3 while (Q is not empty) do // each node goes to Q once and only once: (n) 4 v = dequeue Q; 5 for each vertex w adjacent to v do // (m) 6 if d(w) is yet unassigned then // this is a graph traversal 7 d(w) = d(v) + 1; 8 last_on_path(w) = v; 9 enqueue w in Q; end if; end for; end while; End algorithm. Complexity: (m + n), by a similar analysis as that of the previous queue-based algorithm. 9/15/2020 (C) Debasis Mitra

SOURCE TO ALL NODES SHORTEST PATH-LENGTH Problem 2 Source: v 1, length=0 Queue: v 1 v 3 v 2 Queue: v 2, v 3 , v 5 v 2, length=1 v 3, length=1 v 5, length=1 Q: v 3 , v 5 v 4 v 5, already assigned v 1, no v 4, length=2 Q: v 4 v 3, no Q: empty 9/15/2020 (C) Debasis Mitra v 1, no

SOURCE TO ALL NODES SHORTEST PATH-WEIGHT Problem 3 Input: Positive-weighted graph, a “source” node s Output: Shortest distance to all nodes from the source s v 1 10 20 70 v 3 v 2 25 Breadth-first search does not work any more! Say, source is v 1 Shortest-path-length(v 5) = 35, not 70 30 v 4 9/15/2020 50 v 5 However, the concept works: expanding distance front from s (C) Debasis Mitra

SINGLE SOURCE SHORTEST PATH: Djikstra’s Algorithm Input: Weighted graph Dsw , a “source” node s [can not handle negative cycle] Output: Shortest distance to all nodes from the source s Algorithm Dijkstra 1 s. distance = 0; // shortest distance from s 2 mark s as “finished”; //shortest distance to s from s found 3 For each node w do 4 if (s, w) ϵ E then w. distance= Dsw else w. distance = inf; // O(n) 5 While there exists any node not marked as finished do // (n), each node is picked once & only once 6 v = node from unfinished list with the smallest v. distance; // loop on unfinished nodes O(n): total O(n 2) 7 mark v as “finished”; // WHY? 8 For each edge (v, w) ϵ E do //adjacent to v: (|E|=m) 9 if (w is not “finished” & (v. distance + Dvw < w. distance) ) then 10 w. distance = v. distance + D vw; 11 w. previous = v; // v is parent of w on the shortest path so far end if; end for; end while; End algorithm 9/15/2020 (C) Debasis Mitra

DJIKSTRA’S ALG: SINGLE SOURCE SHORTEST PATH v 1 (0) 10 20 10 10 20 20 70 v 2 ( ) v 3 ( ) 70 v 2 (10) v 3 (20) 25 30 50 30 30 v 5 ( ) v 4 ( ) 50 v 5 (70) v 4 ( ) v 1 (0) 10 50 v 1 (0) 20 70 v 2 (10) 10 20 v 3 (20) 70 v 2 (10) v 3 (20) 25 25 30 v 4 (50) 9/15/2020 50 v 3 (20) 25 25 v 4 ( ) 70 v 2 (10) 30 v 5 (35) v 4 (50) (C) Debasis Mitra 50 v 5 (35)

DJIKSTRA’S ALG: COMPLEXITY Algorithm Dijkstra 1 s. distance = 0; // shortest distance from s 2 mark s as “finished”; //shortest distance to s from s found 3 For each node w do 4 if (s, w) ϵ E then w. distance= Dsw else w. distance = inf; // O(n) 5 While there exists any node not marked as finished do // (n), each node is picked once & only once 6 v = node from unfinished list with the smallest v. distance; // loop on nodes O(n): O(n 2) 7 mark v as “finished”; // WHY? 8 For each edge (v, w) ϵ E do //adjacent to v: (|E|) 9 if (w is not “finished” & (v. distance + Dvw < w. distance) ) then 10 w. distance = v. distance + D vw; 11 w. previous = v; End algorithm • Lines 2 -4, initialization: O(n); • Line 5: while loop runs n-1 times: [ (n)], • Line 6: find-minimum-distance-node v runs another (n) within it, thus the complexity is (n 2); << if we use heap? ? >> • Can you reduce this complexity by a Queue? • Line 8: for-loop within while-loop, for adjacency list graph data structure, runs for |E| (=m) times including the outside while loop. • Grand total: (m + n 2) = (n 2), as m is always n 2. 9/15/2020 (C) Debasis Mitra

DJIKSTRA’S ALG: COMPLEXITY • Djikstra complexity: (m + n 2) = (n 2), as |E|=m is always n 2. • Using a heap data-structure find-minimum-distance-node (line 6) may be log_n, inside while loop (line 5): total O(n log_n) • but as the distance values get changed the heap needs to be kept reorganized, thus, increasing the for-loop's complexity to (m log n). • Grand total: (m log_n + n log_n) = (m log_n), as m is most likely n in a connected graph. 9/15/2020 (C) Debasis Mitra

DJIKSTRA’S ALG: PROOF SKETCH Induction base: Shortest distance for the source s itself must have been found Induction Hypothesis: Suppose, it works up to the k-th iteration. This means, for k nodes the corresponding shortest paths from s have been already identified (say, set F, and V-F is U, the set of unfinished nodes). Induction Step: Let, node p has the shortest distance in U, in the (k+1)th iteration. (1) Each node in F has already tried to update or updated p. (2) Each other node in U has a distance ≥ that of p. Hence, cannot improve p any more (assuming edge weights are positive or non-negative) (3) Hence, p is ‘finished” and so, moved from U to F (4) Each node in F, except p, has its chance to update nodes in U. (5) Each other node in U has a distance ≥ that of p, so, cannot improve distances of nodes in U optimally (assuming edge weights are non-negative) (6) Hence, p is the only candidate to improve distances of nodes in U and so, updates only nodes in U in the (k+1)th iteration 9/15/2020 (C) Debasis Mitra

DJIKSTRA’S ALG: WITH NEGATIVE EDGE • Cost(v) may reduce by adding a new arc from any node p to any node v • Hence, the ‘finished’ marker does not work • The proof of above Djikstra depends on positive/non-neg edge weights • Assumption in Djikstra: a path’s cost is non-decreasing as new arc is added to it • This assumption is NOT true for negative-weighted edges 9/15/2020 (C) Debasis Mitra

DJIKSTRA’S ALG: WITH NEGATIVE EDGE Algorithm Dijkstra-negative Input: Weighted (undirected) graph G, and a node s in V Output: For each node v in V, shortest distance w. distance from s 1 Initialize a queue Q with the s; 2 While Q not empty do 3 v = pop(Q); 4 For each w such that (v, w) ϵ E do 5 If (v. distance + Dvw < w. distance) then 6 w. distance = v. distance +Dvw; 7 w. parent = v; 8 If w is not already in Q then push(Q, w); // if it is already in Q do not duplicate, w. dist is already updated // however, w may have been in Q before end if end for end while End Algorithm. 9/15/2020 (C) Debasis Mitra

DJIKSTRA’S ALG: WITH NEGATIVE EDGE • Negative cost on an edge or a path is all right, but negative cycle is a problem • A path may repeat itself on a loop, thus keep reducing negative cost on it • That will go on infinitely • No limit to minimizing the cost between a source to any node on a negative cycle • Shortest-path is not defined when there is a negative cycle. 9/15/2020 (C) Debasis Mitra

DJIKSTRA’S ALG: WITH NEGATIVE EDGE • Cost reduces on looping around negative cycles • Q-Djikstra does not prevent that: it may not terminate • But how many looping should we allow? • Can we predict that we are on an infinite loop here? • How many times a node may and should appear in the queue? – Each node v may only be once on a shortest path from s to x – There are n-1 shortest paths from s to each of other nodes – Node v may appear on n-1 shortest paths, or on the queue • In other words, What is the longest simple path in the graph ≡ What is the “diameter” of the graph? 9/15/2020 (C) Debasis Mitra

DJIKSTRA’S ALG: WITH NEGATIVE CYCLE • “Diameter” of a graph = n • No node should get a chance to update others more than n times! • Because, there cannot be a simple (loop free) path of length more than n • Line 8 of Q-Djikstra is guaranteed to happen once for any node, or, any node will be pushed to Q at least once • But not more than n times, unless it is on a negative loop • So, Q-Dijkstra should be terminated if a node v gets n+1 times in to the queue • If that happens: the graph has a negative-cost cycle with the node v on it • Otherwise, Q-Djikstra may go into an infinite loop (over the cycle with negative path-weight) Complexity of Q-Djikstra: O(n*m), because each node may be updated by each edge at most one time (if there is no negative cycle). Note: this may become close to O(n 3) for a dense graph, much more than naïve-Dijkstra 9/15/2020 (C) Debasis Mitra

ALL PAIRS SHORTEST PATH Problem 4 Djikstra’s Algorithm is a Greedy Algorithm: pick up the best node from U in each cycle Find all pairs shortest distance (Problem 4): For Each node v as source Run Djikstra(v, G) Complexity: O(m log n) for non-negative Dijkstra, O(n. m) for Q-based Djikstra (when non-negative edges are allowed in input) Floyd-Warshall’s Θ(n 3) algorithm finds all-pairs shortest path. It works with negative edge. It can also detect negative cycle easily: HOW? ? 9/15/2020 (C) Debasis Mitra

Problem 4: All Pairs Shortest Path • A variation of Djikstra’s algorithm. Called Floyd-Warshal’s algorithm. Good for dense graph. Algorithm Floyd 1. Copy the distance matrix in d[1. . n]; 2. for k=1 through n do //consider each vertex as updating candidate 1. for i=1 through n do 1. for j=1 through n do 2. if (d[i][k] + d[k][j] < d[i][j]) then 3. d[i][j] = d[i][k] + d[k][j]; 4. path[i][j] = k; // last updated via k End algorithm. • Θ(n 3), for 3 loops.

D 1(4, 5) = min{D 0(4, 5) , D 0(4, 1) + D 0(1, 5)} = min{inf, 2+3} 9/15/2020 (C) Debasis Mitra Source: Cormen et al.

CRITICAL PATH FINDING ALGORITHM Problem 5: modified Dijkstra Application in Project management Djikstra-like algorithm on Directed Acyclic Graph An Activity Graph with duration for each activity 9/15/2020 (C) Debasis Mitra

CRITICAL PATH FINDING ALGORITHM Corresponding Event Graph: Activities are edges 9/15/2020 (C) Debasis Mitra

CRITICAL PATH FINDING ALGORITHM Event Graph: Find Earliest Completion Time (EC(v)) for each node v EC(v 1) = 0; N = Topological sort of nodes on DAC G; For each node w (other than v 1) in the sorted list N for each edge (v, w) in E EC(w) = max (EC(v) + d(v, w); // d is the distances on arcs on G 9/15/2020 (C) Debasis Mitra

CRITICAL PATH FINDING ALGORITHM Find Latest Completion Time (LC(v)) for each node v LC(vn) = EC(vn); // last node becomes source now For each node v (other than vn) in reverse order of the sorted list N for each edge (v, w) in E LC(v) = min(LC(w) - d(v, w); // d is the distances on arcs on G 9/15/2020 (C) Debasis Mitra

CRITICAL PATH FINDING ALGORITHM EC: above each node LC: below each node Edges: Task id / duration / slack Find Slack Time for each edge (v, w) For each edge (v, w) in E Slack(v, w) = LC(w) – EC(v) - d(v, w); // the task on (v, w) has this much slack time Critical path: A path from v 1 to vn with slack on each edge as 0 Any delay on critical path will delay the whole project! 9/15/2020 (C) Debasis Mitra

Problem 6: Max-flow / Min-cut (Optimization) • Input: Weighted directed graph (weight = maximum permitted flow on an edge), a source node s without any incoming arc, a sink node n without any outgoing arc. • Objective function: schedule a flow graph such that no node holds any amount (influx=outflux), except for s and n, and no arc violets its allowed amount • Output: maximize the objective function (maximize the amount flowing from s to n) output: max Gflow G =>

Max-flow / Min-cut Greedy Strategy • Input: Weighted directed graph (weight = maximum permitted flow on an edge), a source node s without any incoming arc, a sink node n without any outgoing arc. • Output: maximize the objective function (maximize the amount flowing from s to n) • Greedy Strategy: – Initialize a flow graph Gf (each edge 0), and a residual graph Gr – In each iteration, find a “max path” p from s to n in residual Gr: “Max path” means a path allowing maximum flow from s->n, subject to the constraints – Add p to Gf, and subtract p from Gr, any edge having 0 in Gr is removed from Gr – Continue iterations until there exists no such path from s to n in Gr – Return Gf

Greedy Alg, initialization Input G Gflow Gresidual Source: Weiss 9/15/2020 (C) Debasis Mitra

G Greedy Alg, iteration 1 optimum output Gflow Terminates 9/15/2020 (C) Debasis Mitra Gresidual

Max-flow / Min-cut Greedy Strategy • Complexity: – – – How many paths may exist in a graph? m How many times any edge may be updated until it gets to 0? f. m for max-flow f Each iteration, looks for max-path: m log n, using heap Total: O(f. m 2 log n) Note: a pseudo-complexity because of f

Returned Result, max-flow = 3 Optimum Result, max-flow = 5 Greedy is sub-optimal algorithm here <= Capacities Under-utilized Capacity mis-used <= <= <= => 9/15/2020 (C) Debasis Mitra

Optimum Alg, with oracle, not-greedy, iteration 1 Start with not-max-path s->b->d->t (2 on each edge) Iteration 2 9/15/2020 (C) Debasis Mitra

Iteration 2 Iteration 3 Terminates with optimum flow=5 9/15/2020 (C) Debasis Mitra

Max-flow Backtracking Strategy Ford-Fulkerson’s Algorithm • Greedy Strategy is sub-optimal • A Backtracking optimal algorithm: – Initialize a flow graph Gf (each edge 0), and a residual graph Gr – In each iteration, pick any path p (greedy: max-flow path) from s to n in Gr – Add p to Gf, and Gr is updated with reverse flow opposite to p, and leaving residual amounts in forward direction => this allows backtracking if necessary – Continue iterations until either all edges from s are incoming, or all edges to n are outgoing – Return Gf • Guaranteed to terminate for rational numbers input • Complexity: If f is the max flow in returned Gf, – Then, number of times any edge may get updated is f , even though each arc never gets to 0 – O(f. m) • Random choice for a path p may run into bad input

Ford-Fulkerson, using backflow residual, Iteration 1 Iteration 2 Terminates, no outflow left from source, or inflow to sink 9/15/2020 (C) Debasis Mitra

Problem input for Ford-Fulkerson: May keep backtracking if starts with random path, say, s->a->b->t Complexity: If f is the max flow in returned Gf, O(f. m), or O(100000*m) Why bother with random path? Use greedy! 9/15/2020 (C) Debasis Mitra

Max-flow Backtracking with Djikstra Edmonds-Karp Algorithm • Another Backtracking optimal algorithm (may avoid problem of Ford. Fulkerson): – Initialize a flow graph Gf (each edge 0), and a residual graph Gr – In each iteration, find a “max path” p from s to n in Gr using Djikstra – Add p to Gf, and Gr is updated with reverse flow opposite to p, and leaving residual amounts in forward direction => allows backtracking if necessary – Continue iterations until either all edges from s are incoming, or all edges to n are outgoing – Return Gf • Complexity: O(cmaxm 2 log n), first term is max capacity of any edge [actually log_cmax in complexity!] Min-cut = Minimum total capacity from one partition (of G) S ϵ source to another T ϵ sink Which edges are to cut from G to isolate source from sink? Min-cut = Max-flow

Problem 6: Minimum Spanning Tree (Optimization) Prim’s Algorithm • Input: Weighted un-directed graph • Objective function: Total weight of a spanning tree over G • Output: minimize the objective function (minimum-wt spanning tree - MST) • Prim’s Algorithm: Djikstra’s strategy • Separate two sets of nodes F (in MST now) and U (not in MST yet), starting with any arbitrary node s ϵ F • In each iteration find a ‘closest’ node v ϵ U over each of the nodes pϵF; move v from U to F; add Epv to MST: {For all xϵU, For all yϵF, |Evy| ≤ |Exy| } • Continue until U is empty • How many nodes are on a spanning tree? • Return MST • • • 9/15/2020 How many arcs? How many spanning trees exist on a graph? Complexity of Prim’s Alg: ? (C) Debasis Mitra

Prim’s Algorithm for MST: Complexity • • How many nodes are on a spanning tree? How many arcs? How many spanning trees exist on a graph? Complexity of Prim’s Alg: ? Exactly same as Djikstra: O(m log n) 9/15/2020 (C) Debasis Mitra

Prim’s Algorithm for MST Prim’s Alg starting with a in F a 3 c 2 7 9 c 1 10 e 7 9 b 3 1 9 7 1 10 8 d e c 1 8 e Closest to a, b, e or c is d with Ebd=8 a 3 7 b 1 10 8 d a 3 2 9 b d Closest to a, b or e is c with Eac=3 b 7 9 10 e 2 2 c 8 a a 3 d Closest to a or b is e with Ebe=1 c 2 10 8 d a 3 b Closest to a is b with 1 on Eab e c 2 9 7 b 1 10 8 d e Done! All nodes in F. Eab+ Ebc + Ebd+ Eac= 14, is MST weight 9/15/2020 (C) Debasis Mitra

Proof sketch: Prims’s Algorithm Induction base: ? ? ? Induction hypothesis: on k-th iteration (1≤k<n), say, MST is correct for the subgraph of G with k nodes in F Induction Step: Epv is a shortest arc between all arcs from F to U After adding Epv to MST, it is still correct for the new subgraph with k+1 nodes 9/15/2020 (C) Debasis Mitra

Problem 6: MST Kruskal’s Algorithm • Input: Weighted un-directed graph • Objective function: Total weight of a spanning tree over G • Output: minimize the objective function (minimum-wt spanning tree - MST) • Kruskal’s Algorithm: Greedy strategy • 1) Sort all arcs ϵ E in low to high weights; 2)Initialize an empty MST • In each iteration 3)find the shortest arc eϵE; 4)add it to MST unless it causes a cycle (MST may be a forest until all iterations end); 5)E = Ee • Continue iterations until E is empty // will run for |E|=m times • • 9/15/2020 Complexity of Kruskal’s Alg? log_m sorting (Cycle checking in MST needs special data structure, e. g. ADT set merging may be needed to merge two trees) x m iterations O(m log m), which is O(m log n), WHY? (C) Debasis Mitra

Kruskal’s Algorithm for MST Kruskal’s Alg must start with Ebc in MST a a 3 c 2 3 7 9 b c 1 10 e 1 7 b e c 1 d e Add Ebd 7 b 1 10 8 d a 3 2 9 b 8 a 3 8 d 7 9 Eae creates cycle, fails to be added 1 10 2 10 e 2 9 c 8 a 3 c b d Add Eac a 3 7 9 10 8 d 2 e c 2 9 7 b 1 10 8 d e Done! All n-1 arcs in MST. Eab+ Ebc + Ebd+ Eac= 14, is MST weight 9/15/2020 (C) Debasis Mitra

Kruskal’s Algorithm for MST Kruskal’s Alg must start with Ebc in MST a a 2 c 3 2 7 9 b c 1 10 e 1 7 b e c 1 d e Add Ebd 7 b 1 10 8 d a 3 2 9 b 8 a 2 8 d 7 9 Eae creates cycle, fails to be added 1 10 3 10 e 3 9 c 8 a 2 c b d Add Eac a 2 7 9 10 8 d 3 e c 3 9 7 b 1 10 8 d e Done! All n-1 arcs in MST. Eab+ Ebc + Ebd+ Eac= 14, is MST weight 9/15/2020 (C) Debasis Mitra

Proof sketch: Kruskal’s Algorithm Induction does not work: growing forest is not necessarily MST until finished Contradiction: Suppose, returned T is not MST, but a different ST P is: {||T|| > ||P||}, double bar indicates weight Transform T to P, by adding arc Exy to T -> form a cycle -> remove Elm from the cycle already in T to create a new ST -> chain through this process -> P || Elm || must be <= || Exy ||, as Elm was picked up for T before Exy, from pre-sorted list Elm is in T but not Exy, Exy is in P but not Elm So, ||T|| <= ||P||. Contradiction! 9/15/2020 (C) Debasis Mitra

Depth First Traversal of Graphs • Creates a Spanning Forest on a finite graph • For connected graph, it will be a spanning tree • For dynamically developing graph (e. g. search tree), where finiteness is not guaranteed, DFS may not terminate • Space requirement is limited by the depth of the tree What is space complexity on BFS traversal? • Time: depends on branching factor b – have to come back to the same node b number of times • Stack is the typical data-structure for DFS • Many graph algorithms need DFS traversal 9/15/2020 (C) Debasis Mitra

Recursive Depth First Traversal of Graphs Input: graph (V, E) Output: a DFT spanning forest Algorithm dft(v) 1. mark node v as visited; 2. operate on v (e. g. , print); 3. for each node w adjacent to node v do 4. if w is not marked as visited then 5. dft(w); // an iterative version of this //will maintain its own stack end for; End algorithm. Starter algorithm on a given graph G call dfs(any/given node in G); End starter. Problem with above algorithm? 9/15/2020 (C) Debasis Mitra

Recursive Depth First Traversal of Graphs Input: graph (V, E) Output: a DFT spanning forest Algorithm dft(v) 1. mark node v as visited; 2. operate on v (e. g. , print); 3. for each node w adjacent to node v do 4. if w is not marked as visited then 5. dft(w); // an iterative version of this //will maintain its own stack End. Problem with above algorithm: Will terminate prematurely if the graph is disconnected. Repeat until all nodes are marked pick up next unmarked node s; dfs(s); end repeat; This will generate the required DFS-spanning forest 9/15/2020 (C) Debasis Mitra

Problem 7: Finding Articulation Points in a Graph A B C D G E F Input: Undirected, unweighted graph G Articulation Point: a node, removal of which from G will disconnect the graph Output: Articulation point(s) of G, if there exists any 9/15/2020 (C) Debasis Mitra

Finding Articulation Points in a Graph By ordered DF traversal Strategy: • Draw a dft-spanning tree T for the given graph G starting from any of its nodes. • Number them according to the pre-ordering of their traversal (=high(v)). • Def. of “low”: For each vertex v, find the lowest vertex-high where one can go to, from v by the directed edges on T & possibly by a single left-out edge of G (back-edge on T), in other words by only one cycle in G • Do post-ordering DF-traversal on the tree T now, & assign low of each node v: {low(v) =min{high(v), for all children w, low(w), high(w) for each node w with an arc from v in G} • You can assign both high and low in the same dft run. • If for any vertex v there is a child w such that high(v) low(w), then v is an articulation point, unless v is the root in the T (this condition is always true for the root!). • For the root v of T, if it has two or more children, then v is an articulation point. 9/15/2020 (C) Debasis Mitra

Finding Articulation Points in a Graph By ordered DF traversal Input: graph (V, E) Output: a DFT spanning forest Algorithm dft(v) 1. mark node v as visited; 2. pre-order number on v; 3. for each node w adjacent to node v do 4. if w is not marked as visited then 5. dft(w); // an iterative version of this //will maintain its own stack End. 9/15/2020 (C) Debasis Mitra

Finding Articulation Points in a Graph By ordered DFS traversal A B C D G E F B 1 A 2 C 3 G 7 D 4 E 5 F 6 DFT spanning tree: marked with pre-order numbering as “high” values Complexity: O(m + n) 9/15/2020 (C) Debasis Mitra

Finding Articulation Points (AP) in a Graph By ordered DFS traversal A B C D G E Initial low values were assigned same as high, during previous DFT Start, post-order traversal now, to update low, as min of all connected nodes Starting with G: 7 F B 1/1 A 2/2 C 3/3 G 7/7 D 4/4 E 5/5 9/15/2020 (C) Debasis Mitra F 6/6

Finding Articulation Points (AP) in a Graph By ordered DFS traversal A B C D G E F B 1/1 A 2/2 C 3/3 Starting with G: 7, low=7 F: low= min(D, E, F) = 4 G 7/7 D 4/4 E 5/5 9/15/2020 (C) Debasis Mitra F 6/4

Finding Articulation Points (AP) in a Graph By ordered DFS traversal A B C D G E F B 1/1 Starting with G: 7, low=7 F: low= min(D, E, F) = 4 A 2/2 C 3/3 G 7/7 D 4/4 E 5/4 9/15/2020 (C) Debasis Mitra F 6/4

Finding Articulation Points (AP) in a Graph By ordered DFS traversal A B C D G E F B 1/1 Starting with G: 7, low=7 F: low= min(D, E, F) = 4 E: low= min(E, F, D)=4 A 2/2 C 3/3 G 7/7 D 4/4 E 5/4 9/15/2020 (C) Debasis Mitra F 6/4

Finding Articulation Points (AP) in a Graph By ordered DFS traversal A B C D G E F B 1/1 A 2/2 C 3/3 Starting with G: 7, low=7 F: low= min(D, E, F) = 4 E: low= min(E, F, D)=4 D: low= min(D, E, F, B) = 1 … G 7/7 D 4/1 E 5/4 9/15/2020 (C) Debasis Mitra F 6/4

Finding Articulation Points (AP) in a Graph By ordered DFS traversal A B C D G E F B 1/1 A 2/1 C 3/1 G 7/7 D 4/1 E 5/4 F 6/4 Second DFT on the tree, marking “low” value of node, meaning lowest value it can get to by one cycle only Complexity: O(m + n), actually performed during the first DFT 9/15/2020 (C) Debasis Mitra

Finding Articulation Points in a Graph By ordered DFS traversal A B C D G E F B 1/1 A 2/1 C 3/1 G 7/7 D 4/1 F 6/4 E 5/4 Scan the Tree again, Check for, high(node) ≤ low(any child), those nodes are AP’s of the input graph, and that child will get disconnected for removing the node Complexity: O(m + n), Total complexity of the algorithm: O(m + n) 9/15/2020 (C) Debasis Mitra

Finding Articulation Points in a Graph By ordered DFS traversal A B C D G E F B 1/1 A 2/1 C 3/1 G 7/7 D 4/1 The condition high(node) ≤ low(any child) would always be satisfied for root, even if it were not an AP So, when is the root of the DFT spanning tree really an AP in the input graph? 9/15/2020 (C) Debasis Mitra E 5/4 F 6/4

Finding Articulation Points in a Graph By ordered DF-traversal A B C D G E F D 1/1 B 2/. . E. . /. . Start the scan from D, The condition high(node) ≤ low(any child) is not useful anymore. So, when is the root of the DFT spanning tree, is really an AP in the input graph? The root must have two or more children in the tree to be an AP (not true in this example, so, A is not an AP) 9/15/2020 (C) Debasis Mitra

Problem 7: Finding Strongly Connected Component (SCC) in a Directed Graph SCC = {Maximal subgraph | directed path exists from any node to any other} Note, SCC is different from just a cycle or a clique Source: Weiss, Fig 9. 74 9/15/2020 (C) Debasis Mitra

SCC-detection Algorithm Input: Directed Graph G Output: Set of subgraph(s) SCC of G, each subgraph in SCC is a strong connected component of G Note, the set SCC creates a partition over G Strategy: 1. Traverse G using DFT to create a spanning forest T of the graph G, numbering the nodes in the post-ordering traversal; 2. Create Gr, by reversing the directions of arcs in G; 3. Traverse using DFT on Gr, in a decreasing order of the numbers on nodes, 4. DFT spanning-forest from this second traversal creates the set SCC for G. 9/15/2020 (C) Debasis Mitra

SCC-detection Algorithm Input: graph (V, E) Output: a DFT spanning forest Algorithm dft(v) 1. mark node v as visited; 2. for each node w adjacent to node v do 3. if w is not marked as visited then 4. dft(w); // an iterative version of this //will maintain its own stack 5. post-order numbering on v; 6. return End. 9/15/2020 (C) Debasis Mitra

SCC-detection Algorithm: First Pass DFT, post-order numbering 9/15/2020 (C) Debasis Mitra

SCC-detection Algorithm: Second Pass DFT on Gr Now the DFT must follow strict ordering 1. Start DFT, stuck at G: G is a SCC 2. Start DFT again: B-A-C-F is a SCC 3. Start DFT again, stuck at D: D is a SCC 4. Start DFT again, stuck at E: E is a SCC 5. Start DFT again: H-I-J is a SCC 9/15/2020 (C) Debasis Mitra

Problem 8: Finding Euler Tour in a Undirected Graph Exists or not? O(m) time: Count degree of each node 9/15/2020 (C) Debasis Mitra

Problem 8: Finding Euler Circuit in a Undirected Graph Exists or not? O(m) time: Count degrees of nodes 9/15/2020 (C) Debasis Mitra

Finding Euler Circuit by Repeated use of modified-DFT Modified-DFT starting from 5: 5 -4 -10 -5 9/15/2020 (C) Debasis Mitra

Finding Euler Circuit by Repeated use of modified-DFT 5 -4 -10 -5 Left out graph: Still even degrees, but may not be connected Re-start from 4 now: 4 -1 -3 -7 -4 -11 -10 -7 -9— 3 -4 9/15/2020 (C) Debasis Mitra

Finding Euler Circuit by Repeated use of modified-DFT Done: 5 -4 -1 -3 -7 -4 -11 -10 -7 -9 -3 -4 -10 -5 Start from, say, 3 now: 3 -2 -8 -9 -6 -3 Done: 5 -4 -1 -3 -2 -8 -9 -6 -3 -7 -4 -11 -10 -7 -9 -3 -4 -10 -5 Start with, say, 9 now: 9 -10 -12 -9 You know what to do next! 9/15/2020 (C) Debasis Mitra

Modified-Depth First Traversal for Euler Circuit Input: connected undirected graph (V, E) Output: a loop in the graph Algorithm dft(v) 1. for each node w adjacent to node v do 2. if (v, w) is unmarked 3. mark (v, w) as visited; 4. parentof(w) = v; 5. degreeof(w) = count unmarked edges at w; // O(n)? Can we count only once and decrement? 6. if degreeof(w) = = 0 then enque w in Q; 7. dft(w); end if; end for; End algorithm. Euler-ckt algorithm 1. chose any node as starting node s and add it to Q; // Queue of nodes with non-zero degrees 2. until all edges are marked 3. v = pop(Q); 4. newloop = dft(v); 5. eckt = splice newloop into eckt at node v; 6. end until; 7. return eckt; End Algorithm. 9/15/2020 (C) Debasis Mitra

Modified-Depth First Traversal for Euler Circuit Euler-ckt algorithm 1. chose any node as starting node s and add it to Q; // Queue of nodes with non-zero degrees 2. until all edges are marked 3. v = pop(Q); // they should be nodes only on eckt 4. newloop = dft(v); 5. eckt = splice newloop into eckt at v; 6. end until; 7. return eckt; End Algorithm. Note, I am not counting degrees of all nodes in the beginning. Will this detect non-existence of Euler ckt? How to terminate then? No nodes on the eckt has >0 degree, but all edges are not marked? How to prove that this will never happen on a graph that has E-circuit? Complexity: Each edge is visited once, and possibly checked for being marked a second time O(m) 9/15/2020 (C) Debasis Mitra

Finding Euler Tour in a Undirected Graph Start with an odd degree node and end on the second odd degree node Previous algorithm should work <<Proof? >> 9/15/2020 (C) Debasis Mitra

Problem 9: Hamiltonian Tour problem: Traverse each node once and only once 9/15/2020 (C) Debasis Mitra

Problem 9: Hamiltonian Circuit problem: Traverse each node once and only once, and return to source Decision question: Does Ham-ckt exist? NP-complete Full problem: Find one, if it exists. 9/15/2020 (C) Debasis Mitra

Depth First Traversal for Hamiltonian Circuit problem Input: graph (V, E) Output: a Hamiltonian Circuit Algorithm dft(v, level) 1. if (level = = n) then return True; else 2. mark node v as visited; 3. for each node w adjacent to node v do 4. path = path -> w; 5. if w is not marked as visited then 6. found. HC = dft(w, level+1); 7. if (found. HC) then return True; // stop algorithm 8. else delink w from path; end if end for; return Fail; // none of the children lead to Hamiltonian tour, let the parent backtrack within for loop end if End algorithm. • Naïve Hamiltonian-circuit finding backtracking algorithm • found. HC = Fail; • for each node s • path = path -> s; • dft(s); 1. if (found. HC) then 2. l = last node on the returned Hamiltonian-tour path; 3. if (l, s) E then return Ham-circuit = Ham-path + s; 4. end if 5. end for 6. return Failure; 7. End starter. Complexity: O(n!) (or O(nn)) for naïve backtracking Note: this is NP-complete problem. So, ? ? ? 9/15/2020 (C) Debasis Mitra

Problem 10: Traveling Sales Person (TSP) problem: Minimum cost Hamiltonian Circuit, An optimization problem over Hamiltonian Circuits Decision question: Does Ham-ckt exist? NP-complete Full problem: Find best one, over all possible Ham-ckts. There is a dynamic programming algorithm exponential 9/15/2020 (C) Debasis Mitra