Data Structures Algorithms Digraphs and DAGs Richard Newman

Data Structures & Algorithms Digraphs and DAGs Richard Newman based on book by R. Sedgewick and slides by S. Sahni

Digraphs • Edges are directed • Number of possible undirected graphs is huge • 2 V(V+1)/2 • Number of possible directed graphs is … huger(? ) • 2 V^2

Digraphs V Undir Graphs • Graph enumeration 2 8 Digraphs 16 3 64 512 4 1, 024 65, 536 5 32, 768 33, 554, 432 6 2, 097, 152 68, 719, 476, 736 7 268, 435, 456 562, 949, 953, 421, 312 8 wow Wow wow

Digraphs • Defn. 19. 1: A digraph is a set of nodes V and a set of distinct directed edges E, each from one node to another node in V. (self-loop allowed) • Defn. 19. 2: A directed path in a digraph is a list of nodes for which there is an edge from each node to its successor. A node t is reachable from node s iff there is a d. p. from s to t.

Digraphs • Defn. 19. 3: A directed acyclic graph (DAG) is a digraph with no directed cycles (tours). A node with only outedges is a source; a node with only in -edges is a sink. • Defn. 19. 4: A digraph is strongly connected iff every node is reachable from every node.

DAGs • DAGs can be used to model many real-life problems • Scheduling • Precedence • Pre-requisite structures • Causality • Etc.

Digraphs • Prop. 19. 1: A digraph that is not strongly connected comprises a set of strongly connected components, which are maximal strongly connected subgraphs, and a set of directed edges that go from one component to another.

DAGs • Connected components • 0 -7 -4 -5 -3 2 • 2 -6 1 • 1 6 • 1 0 7 5 4 3

Digraphs • Prop. 19. 2: Given a digraph D, define another digraph K(D) with one node corresponding to each strongly connected component of D, and an edge from u to v iff there is one or more edge from the component corresponding to u to the component corresponding to v. K(D) is a DAG called the kernel DAG of D.

DAGs • DAG Components • 0: 0 -7 -4 -5 -3 • 1: 2 -6 • 2: 1 • Kernel DAG • Component 0 • Component 1 • Component 2 2 0 1 6 1 7 5 4 3 0 2

Digraphs • In undirected graph, we just say two nodes are connected if there is a path between them • In a digraph, node t is reachable from node s if there is a directed path from s to t. • In a digraph, s and t are strongly connected if they are mutually reachable.

Digraphs • Classify edges in DFS • Tree – recursive calls • Back – to ancestor (including parent!) • Down – to visited descendent • Cross – neither ancestor nor descendent (cousins)

DFS in Digraphs 2 0 1 6 7 5 4 3 01234567 pre 0 4 1 7 5 6 2 3 post 7 2 1 3 5 4 0 6 Stack: 0 down 0 572 2 7 576 4 57(2) back 6 1 541 cross 5 54 55(6 cross) 3 53(0 back) 5(4 back) (5 down)

DFS Algorithms • Cycle Detection • If we find a back edge, it represents a cycle – including link to parent! • Cross edges don’t make cycles! • Reachability • Start from one, DFS until find other (or complete DFS) • Weak connectivity • If DFS finds all the nodes, then yes!

DFS Algorithms • Convert digraph to DAG • Remove back edges! • Use to generate large DAGs from large digraphs • Note that DFS in a digraph only gives reachability from the start node, not from all nodes

Transitive Closure • Defn. 19. 5: The transitive closure of a digraph D is a digraph T with the same vertices but with an edge from s to t in T iff t is reachable from s in D. 2 0 1 6 2 7 5 4 3 0 1 6 7 5 4 3

Transitive Closure • Can also view (and compute) transitive closure by Boolean matrix multiplication • Use logical AND as x • Use logical OR as + • Ai represents (any) path of length i A 0 1 2 3 4 5 6 7 0 01 0 0 0 0 0 1 2 1 0 0 0 1 0 3 0 0 0 1 0 0 4 0 0 0 1 5 1 0 0 0 6 0 0 1 0 0 0 7 1 0 0 0 0 A 2 0 1 2 3 4 5 6 7 0 1 0 0 0 1 1 0 0 0 0 2 0 0 1 1 0 0 3 1 0 0 0 4 1 0 0 5 0 0 0 1 0 1 6 1 0 0 1 1 7 0 0 0 1 0 0

Transitive Closure • Can also view (and compute) transitive closure by Boolean matrix multiplication • Use logical AND as x • Use logical OR as + • Ai represents (any) path of length i A<3 0 1 2 3 4 5 6 7 0 01 1 1 0 0 0 0 1 2 1 0 1 1 1 0 3 1 0 0 0 1 1 0 0 4 1 0 0 1 0 1 5 1 0 0 1 1 1 0 1 6 1 0 1 1 7 1 0 0 A 3 0 1 2 3 4 5 6 7 0 0 1 0 1 1 0 0 0 1 2 1 0 0 1 1 3 0 0 0 1 0 1 4 1 0 0 0 1 1 0 1 5 1 0 0 0 1 1 0 0 6 1 0 1 7 1 0 0 0

Transitive Closure • Can also view (and compute) transitive closure by Boolean matrix multiplication • Use logical AND as x • Use logical OR as + • Ai represents (any) path of length i A<4 0 1 2 3 4 5 6 7 0 01 1 1 0 0 1 0 1 1 0 0 1 1 2 1 0 1 1 1 3 1 0 0 1 1 1 0 1 4 1 0 0 1 1 1 0 1 5 1 0 0 1 1 1 0 1 6 1 0 1 1 1 7 1 0 0 0 1 1 0 0 A 4 0 1 2 3 4 5 6 7 0 0 0 1 1 0 0 0 0 1 0 0 0 2 1 0 1 3 0 0 1 1 0 0 4 1 0 0 1 1 1 0 1 5 1 0 0 1 1 1 0 1 6 1 0 0 1 1 1 7 1 0 0 1 0 1

Transitive Closure • Can also view (and compute) transitive closure by Boolean matrix multiplication • Use logical AND as x • Use logical OR as + • Ai represents (any) path of length i A<5 0 1 2 3 4 5 6 7 0 01 1 1 0 0 1 0 1 1 0 0 1 1 2 1 0 1 1 1 3 1 0 0 1 1 1 0 1 4 1 0 0 1 1 1 0 1 5 1 0 0 1 1 1 0 1 6 1 0 1 1 1 7 1 0 0 1 1 1 0 1 A 5 0 1 2 3 4 5 6 7 0 0 1 1 1 0 0 0 1 0 1 2 1 0 0 1 1 1 3 0 0 0 1 1 1 0 1 4 1 0 0 1 1 1 0 1 5 1 0 0 1 1 1 0 1 6 1 0 1 7 1 0 0 0 1 1 0 0

Transitive Closure • Can also view (and compute) transitive closure by Boolean matrix multiplication • Use logical AND as x • Use logical OR as + • Ai represents (any) path of length i A<6 0 1 2 3 4 5 6 7 0 01 1 1 0 0 1 1 2 1 0 1 1 1 3 1 0 0 1 1 1 0 1 4 1 0 0 1 1 1 0 1 5 1 0 0 1 1 1 0 1 6 1 0 1 1 1 7 1 0 0 1 1 1 0 1 • Keep on multiplying and adding until… … reach fixed point Matrix does not change

Transitive Closure • Prop. 19. 5: We can compute the transitive closure of a digraph by adding self-loops, then computing AV, taking time V 4. • Self-loops allow path to be of any length up to exponent • Must reach fixed point by V – why? • Efficient approach: • A, A 2, A 4, A 8, … successive squaring • Takes lg V matrix multiplies, each V 3 • Total time is V 3 lg V

Transitive Closure • Even faster way! • Warshall’s algorithm: for (i = 0; i < V; ++i) for (s = 0; s < V; ++s) for (t = 0; t < V; ++t) if (A[s][i] && A[i][t]) A[s][t] = 1;

Transitive Closure Correctness by induction on i: Base: After first iteration, s-t or s-0 -t After second iteration, s-t, s-0 -t, s-1 -t, s-0 -1 -t, s-1 -0 -t. IH: After ith iteration – all paths w/o inner nodes > i Inductive step: path from s to t w/o i+1 (already there) or path via i+1 (tested by if statement) for (every intermediate node i) for (every source s) for (every destination t) if (s reaches i & i reaches t) s reaches t;

Transitive Closure • Prop. 19. 7: Warshall’s algorithm computes the transitive closure of a digraph in time V 3. • Obvious from structure of Warshall’s algorithm – three nested loops of V each: for (i = 0; i < V; ++i) for (s = 0; s < V; ++s) for (t = 0; t < V; ++t) if (A[s][i] && A[i][t]) A[s][t] = 1;

Transitive Closure • Prop. 19. 8: We can support constant-time reachability testing for a digraph with V nodes using space O(V 2) and preprocessing time O(V 3). • Can improve Warshall’s algorithm: for (i = 0; i < V; ++i) for (s = 0; s < V; ++s) for (t = 0; t < V; ++t) if (A[s][i] && A[i][t]) A[s][t] = 1;

Transitive Closure • We can improve Warshall’s algorithm by moving the test of A[s][i] out of the inner loop, avoiding innermost loop when s cannot reach i. for (i = 0; i < V; ++i) for (s = 0; s < V; ++s) if (A[s][i]) for (t = 0; t < V; ++t) if (A[i][t]) A[s][t] = 1;

Shortest Path • We can modify Warshall’s algorithm to compute shortest path, if A[][] contains the length of the minimum path from s to t (initialized with 1 for an edge and sentinel value V for no edge). for (i = 0; i < V; ++i) for (s = 0; s < V; ++s) for (t = 0; t < V; ++t) if (A[s][i] + A[i][t] < A[s][t]) A[s][t] = A[s][i] + A[i][t];

Reduction • Prop. 19. 9: We can use any transitiveclosure algorithm to compute the product of two Boolean matrices with at most a constant factor difference in running time. • Prf: Construct a 3 v x 3 v matrix using A, B, and Vx. V identity matrix I. TC is square. 2 I A 0 I A AB 0 I B 0 0 I = 0 0 I 0 B I

Reduction • What this means is that if we can perform transitive closure faster, then we can compute Boolean matrix products faster. • Likewise, a faster Boolean matrix multiply algorithm will speed up our TC algorithm. • Note that we can compute TC faster for sparse graphs – time O(V(E+V))

Topological Sort • Relabel: Given a DAG, relabel its nodes such that every directed edge points from a lower-numbered node to a highernumbered node. • Rearrange: Given a DAG, rearrange its nodes on a horizontal line such that all the directed edges point from left to right. • Key = turn partial order into total order that is consistent with the partial order

Topological Sort 2 Relabel: 0 1 7 6 2 7 5 4 1 2 3 6 Rearrange: 3 0 0 7 1 4 5 7 1 5 4 3 6 2

Reverse Topological Sort • Relabel: Given a DAG, relabel its nodes such that every directed edge points from a higher-numbered node to a lowernumbered node. • Rearrange: Given a DAG, rearrange its nodes on a horizontal line such that all the directed edges point from right to left. • Just reverse the regular topological sort

Topological Sort • Prop. 19. 11: Postorder numbering in DFS yields a reverse topological sort for any DAG. • It is easy to turn reverse topological sort into a regular topological sort

Topological Sort • Prop. 19. 12: Every DAG has at least one source and at least one sink. • Turn this into a topological sort algorithm: 1. Make indegree[V] vector initialized to 0 2. Scan through DAG (visiting every edge) and increment indegree[i] each time an edge to node i is found. 3. Scan through indegree[] and enqueue all nodes with indegree zero (the sources).

Topological Sort • Prop. 19. 12: Every DAG has at least one source and at least one sink. 4. Set current. ID = 0. 5. While the queue is non-empty, 6. remove a node x and label it current. ID 7. current. ID++ 8. Decrement indegree[j] for all edges from node x to node j 9. If indegree[j] == 0, enqueue node j

Topological Sort 2 0 1 6 2 7 7 5 4 3 0 1 3 6 7 2 5 4 1 3 indegree: 0 01 012 0 012 1023 10 01 Source Queue: 0 3 7 1 4 5 6 2

Topological Sort Application • Finding longest path from each node, longest path in DAG • “Critical path” for scheduling • What is the most urgent thing to do? • Reverse topological sort DAG • In RTS order, for each node v, longest path from v, LP[v] = 1 + max{LP[x] | (v, x) in E} • Guaranteed that LP[x] is known by time it is needed. Example of… Dynamic Programming!

Topological Sort 2 7 0 1 3 6 7 2 5 4 1 3 Longest path: 5 4 4 1 3 1 2 1

Transitive Closure Redux Reachability from each node in DAG • Reverse topological sort DAG • Row vector Reach[v] – initially self and successors • In RTS order, for each node v • Reach[v] = OR {Reach[x] | (v, x) in E} • Guaranteed that Reach[x] is known by time it is needed. Another example of… Dynamic Programming!

Transitive Closure Redux • Matrix approach takes time O(VE) • Direct recursive DFS approach: • No back edges (no cycles) • Tree edges – recursive call • Cross edges – do OR, no call • Down edges – skip, no call • Prop. 19. 13: Using DFS and DP, can compute TC of a DAG in time O(V 2+VX) where X is the number of cross edges

Strongly Connected Components • Kosaraju’s method • Do DFS on reversed digraph • Compute post-order numbering • Do DFS of original graph using reverse of postorder computed on reverse graph • Time and space are both linear in V+E!

Recap • • • Digraphs Strong connectivity Connected components Reachability Digraph kernel Transitive closure Shortest paths (special case) Reduction (from Boolean matrix multiply) Topological Sort