Graph Sparsifiers Nick Harvey Joint work with Isaac

Graph Sparsifiers Nick Harvey Joint work with Isaac Fung

5 6 s 6 5 4 4 3 5 5 7 • What is the max flow from s to t? 7 t 3

u s 15 15 10 15 t 15 • What is the max flow from s to t? • The answer in this graph is the same: it’s a Gomory-Hu tree. • What is capacity of all edges incident on u?

Can any dense graph be “approximated” by a sparse graph? • Approximating pairwise distances n = # vertices – Spanners: number of edges = O(n 1+2/®), distance approximated to within ®. [ADDJS’ 93] – Low-stretch trees: number of edges = n-1, “most” distances approximated to within log n. [FRT’ 04] • Approximating all cuts [BSS’ 09] – Sparsifiers: number of edges = O(n log n /² 2) , every cut approximated within 1+². [BK’ 96] • Spectral approximation [BSS’ 09] – Spectral sparsifiers: number of edges = O(n log n /² 2), entire spectrum approximated within 1+². [SS’ 08]

What is the point of all this? • Approximating pairwise distances – Spanners: Network routing, motion planning, etc. – Low-stretch trees: Approximating metrics by simpler metrics, approximation algorithms

What is the point of all this? n = # vertices m = # edges • Approximating all cuts – Sparsifiers: fast algorithms for cut/flow problem Problem Approximatio n Runtime Reference Min st Cut 1+² O~(n 2) BK’ 96 Sparsest Cut O(log n) O~(n 2) BK’ 96 Max st Flow 1 O~(m+nv) KL’ 02 O~(n 2) AHK’ 05 O~(m+n 3/2) KRV’ 06 Sparsest Cut O~(m+n 3/2+²) S’ 09 Perfect Matching in n/a Regular Bip. Graphs O~(n 1. 5) GKK’ 09 Sparsest Cut O~(m+n 1+²) M’ 10 Sparsest Cut O(log 2 n) v = flow value

What is the point of all this? • Spectral approximation – Spectral sparsifiers: solving diagonally-dominant linear systems in nearly linear time! Problem Runtime Reference Computing Fiedler Vector O~(m) ST’ 04 Computing Effective Resistances O~(m) SS’ 08 Sampling Random Spanning Trees KM’ 09 Max st Flow O~(m 4/3) CKMST’ 10 Min st Cut O~(m+n 4/3) CKMST’ 10 Dimensionality reduction in L 1

Graph Sparsifiers: Formal problem statement • Design an algorithm such that • Input: An undirected graph G=(V, E) • Output: A weighted subgraph H=(V, F, w), where FµE and w : F ! R • Goals: n = # vertices m = # edges between U and VU in G weight of edges between U and VU in H • |||± · ²² |±(U)| |±G(U)| |±(U)| w(±(U)) G(U)| - w(± H(U)) || · • |F| = O(n log n / ² 2) • Running time = O~( m / ² 2 ) 8 U µ V

Why should sparsifiers exist? • Example: G = Complete graph Kn • Sampling: Construct H by sampling every edge of G with probability p=100 log n/n • Properties of H: • # sampled edges = O(n log n) • Standard fact: H is connected • Stronger fact: p|±G(U)| ¼ |±H(U)| 8 U µ V • Output: – H with each edge given weight 1/p – By this, H is a sparsifier of G

• Chernoff Bound: Let X 1, X 2, . . . be {0, 1} random variables. Let X = i Xi and let ¹ = E[ X ]. For any ± 2[0, 1], Pr[ |X-¹| ¸ ±¹ ] · 2 exp( -± 2¹ / 3 ). • • Consider any cut ±G(U) with |U|=k. Then |±G(U)|¸kn/2. Let Xe = 1 if edge e is sampled. Let X = e 2 C Xe = |±H(U)|. Then ¹ = p |±(U)| ¸ 50 k log n. Key Ingredients Say cut fails if |X-¹| ¸ ¹/2. So Pr[ cut fails ] · 2 exp( - ¹/12 ) · n-4 k. Chernoff Bound on # small cuts # of cuts with |U|=k is. So Pr[ any cut fails ] · k n-4 k < k n-3 k < Union n-2. bound So, whp, every U has ||±H(U)| - p |±(U)|| < p |±(U)|/2.

Generalize to arbitrary G? Eliminate most of these Keep this • Can’t sample edges with same probability! • Idea [BK’ 96] Sample low-connectivity edges with high probability, and high-connectivity edges with low probability

Non-uniform sampling algorithm [BK’ 96] • Input: Graph G=(V, E), parameters pe 2 [0, 1] • Output: A weighted subgraph H=(V, F, w), where FµE and w : F ! R For i=1 to ½ For each edge e 2 E With probability pe, Add e to F Increase we by 1/(½pe) • Main Question: Can we choose ½ and pe’s to achieve sparsification goals? • Clearly running time is O(½m) (so want ½ · polylog(n)) • Clearly |F| = O(½ e pe) (so want e pe = O(n))

Non-uniform sampling algorithm [BK’ 96] • Input: Graph G=(V, E), parameters pe 2 [0, 1] • Output: A weighted subgraph H=(V, F, w), where FµE and w : F ! R For i=1 to ½ For each edge e 2 E With probability pe, Add e to F Increase we by 1/(½pe) • Claim: H perfectly approximates G in expectation! • For any e 2 E, E[ we ] = 1 ) For every UµV, E[ w(±H(U)) ] = |±G(U)| • Goal: Show every w(±H(U)) is tightly concentrated

Prior Work Assume ² is constant • Benczur-Karger ‘ 96 – Set ½ = O(log n), pe = 1/“strength” of edge e (max k s. t. e is contained in a k-edge-connected vertex-induced subgraph of G) – All cuts are preserved – e pe · n ) |F| = O(n log n) – Running time is O(m log 3 n) • Spielman-Srivastava ‘ 08 – Set ½ = O(log n), pe = “effective resistance” of edge e (view G as an electrical network where each edge is a 1 -ohm resistor) – – H is a spectral sparsifier of G ) all cuts are preserved e pe = n-1 ) |F| = O(n log n) O(m log 3 n) [Peng et al. ‘ 10] nd. See it here on Nov 22 50 Running time is O(m log n) Uses powerful tools from Geometric Functional Analysis

Our Work • Fung-Harvey ’ 10 Assume ² is constant (and independently Hariharan-Panigrahi ‘ 10) – Set ½ = O(log 2 n), pe = 1/edge-connectivity of edge e (min size of a cut that contains e) – – Advantage: Edge connectivities easy to compute All cuts are preserved e pe · n ) |F| = O(n log 2 n) Running time is O(m log 2 n) • Alternative Algorithm – Let H be union of ½ uniformly random spanning trees of G, where we is 1/(½¢(effective resistance of e)) – All cuts are preserved – |F| = O(n log 2 n) – Running time is

• Notation: kuv = min size of a cut separating u and v • Main ideas: – Partition edges into connectivity classes E = E 1 [ E 2 [. . . Elog n where Ei = { e : 2 i-1·ke<2 i }

• Notation: kuv = min size of a cut separating u and v • Main ideas: – Partition edges into connectivity classes E = E 1 [ E 2 [. . . Elog n where Ei = { e : 2 i-1·ke<2 i } – Prove weight of sampled edges that each cut takes from each connectivity class is about right – This yields a sparsifier U

Prove weight of sampled edges that each cut takes from each connectivity class is about right • Notation: • C = ±(U) is a cut • Ci = ±(U) Å Ei is a cut-induced set • Need to prove: C 1 C 2 C 3 C 4

• Notation: Ci = ±(U) Å Ei is a cut-induced set Prove 8 cut-induced set Ci • Key Ingredients • Chernoff bound: Prove small • Bound on # small cuts: Prove #{ cut-induced sets Ci induced by a small cut |C| } is small. • Union bound: sum of failure probabilities is small, so probably no failures. C 1 C 2 C 3 C 4

Counting Small Cut-Induced Sets • Theorem: Let G=(V, E) be a graph. Fix any BµE. Suppose ke¸K for all e in B. (kuv = min size of a cut separating u and v) Then, for every ®¸ 1, |{ ±(U) Å B : |±(U)|·®K }| < n 2®. • Corollary: Counting Small Cuts [K’ 93] Let G=(V, E) be a graph. Let K be the edge-connectivity of G. (i. e. , global min cut value) Then, for every ®¸ 1, |{ ±(U) : |±(U)|·®K }| < n 2®.

(Slightly unfair) Comparison • Theorem: Let G=(V, E) be a graph. Fix any BµE. Suppose ke¸K for all e in B. (kuv = min size of a cut separating u and v) Then |{ ±(U) Å B : |±(U)|·c }| < n 2 c/K 8 c¸ 1. • Corollary [K’ 93]: Let G=(V, E) be a graph. Let K be the edge-connectivity of G. (i. e. , global min cut value) Then, |{ ±(U) : |±(U)|·c }| < n 2 c/K 8 c¸ 1. • How many cuts of size 1? Theorem says < n 2, taking K=c=1. Corollary, says < 1, because K=0.

Algorithm For Finding Needle in Haystack • Input: A haystack • Output: A needle (maybe) • While haystack not too small – Pick a random handful – Throw it away • End While • Output whatever is left

Algorithm for Finding a Min Cut [K’ 93] • Input: A graph • Output: A minimum cut (maybe) • While graph has 2 vertices “Not too small” – Pick an edge at random “Random Handful” – Contract it “Throw it away” • End While • Output remaining edges • Claim: For any min cut, this algorithm outputs it with probability ¸ 1/n 2. • Corollary: There are · n 2 min cuts.

Finding a Small Cut-Induced Set • Input: A graph G=(V, E), and BµE • Output: A cut-induced subset of B • While graph has 2 vertices – If some vertex v has no incident edges in B • Split-off all edges at v and delete v – Pick an edge at random – Contract it • End While • Output remaining edges in B • Claim: For any min cut-induced subset of B, this algorithm outputs it with probability >1/n 2. • Corollary: There are <n 2 min cut-induced subsets of B

Sparsifiers from Random Spanning Trees • Let H be union of ½=log 2 n uniform random spanning trees, where we is 1/(½¢(effective resistance of e)) • Then all cuts are preserved and |F| = O(n log 2 n) • Why does this work? – Pr. T[ e 2 T ] = effective resistance of edge e – Similar to usual independent sampling algorithm, with pe = effective resistance of e – Key difference: edges in a random spanning tree are not independent. – But, they are negatively correlated ! That is enough to make Chernoff bounds work.

Conclusions • Graph sparsifiers important for fast algorithms and some combinatorial theorems. • Sampling by edge-connectivities gives a sparsifier with O(n log 2 n) edges • Also true for sampling by effective resistances. ) sampling O(log 2 n) random spanning trees gives a sparsifier. Questions • Improve log 2 n to log n? • Sampling o(log n) random trees gives a sparsifier?

Analysis of Min Cut Algorithm • While graph has 2 vertices – Pick an edge uv at random – Contract it • End While • Output remaining edges “Not too small” “Random Handful” “Throw it away” • Fix some min cut. Say it has k edges. • If algorithm doesn’t contract any edge in this cut, then the algorithm outputs this cut – When contracting edge uv, both u & v are on same side of cut • So what is probability that this happens?

• Initially there are n vertices. • Claim 1: # edges in min cut=k every vertex has degree k total # edges nk/2 • Pr[random edge is in min cut] = # edges in min cut / total # edges k / (nk/2) = 2/n

• Now there are n-1 vertices. • Claim 2: min cut in remaining graph is k • Why? Every cut in remaining graph is also a cut in original graph. • So, Pr[ random edge is in min cut ] 2/(n-1)

• In general, when there are i vertices left Pr[ random edge is in min cut ] 2/i • So Pr[ alg never contracts an edge in min cut ]