Submodular Set Function Maximization A MiniSurvey Chandra Chekuri

Submodular Set Function Maximization A Mini-Survey Chandra Chekuri Univ. of Illinois, Urbana-Champaign

Submodular Set Functions A function f : 2 N R is submodular if f(A) + f(B) ≥ f(A B) + f(A B) for all A, B N Equivalently, f(A+j) – f(A) ≥ f(A+i+j) – f(A+i) for all A N and i, j NA

Submodular Functions • Non-negative submodular set functions f(A) ≥ 0 for all A • Monotone submodular set functions f(ϕ) = 0 and f(A) ≤ f(B) for all A B • Symmetric submodular set functions f(A) = f(NA) for all A

Well-known Examples • Cut functions in undirected graphs and hypergraphs (symmetric non-negative) • Cut functions in directed graphs (non-negative) • Rank functions of matroids (monotone) • Coverage in set systems (monotone) • many others. . .

Cut Functions in Graphs • G=(V, E) undirected graph • f : 2 V R+ where f(S) = |δ(S)| S

Coverage in Set Systems • S 1, S 2, . . . , Sn subsets of set U • f : 2{1, 2, . . . , n} R+ where f(A) = | i in A Si | • w : U R+ and f(A) = w ( i in A Si) • . . .

Maximizing Submodular Set Functions Given f on a ground set N via a value oracle max S N f(S) S satisfies some constraints

Maximizing Submodular Set Functions Given f on a ground set N via a value oracle max S N f(S) S satisfies some constraints Motivation: • Many non-trivial applications (easy to miss!) • Generalize known results for modular functions

Unconstrained Problem max S N f(S) • Uninteresting for monotone f • NP-Hard for non-negative f (Max-Cut is a special case) • Very hard to approximate for arbitrary f (reduction from Set Packing)

Unconstrained Problem [Feige-Mirrokni-Vondrak’ 07] First O(1) approximation for non-negative f ! Easy O(1) algorithms, ½ for symmetric case Non-trivial 2/5 = 0. 4 approximation (slight improvement to 0. 41 [Vondrak])

Unconstrained Problem [Feige-Mirrokni-Vondrak’ 07] First O(1) approximation for non-negative f ! Easy O(1) algorithms, ½ for symmetric case Non-trivial 2/5 = 0. 4 approximation (slight improvement to 0. 41 [Vondrak]) Better than ½ requires exponential # of value queries Open Problem: Close gap between 0. 41 and ½

Unconstrained Problem [Feige-Mirrokni-Vondrak’ 07] Random set algorithm: • pick each i in N with prob ½, let R be random set • E[f(R)] ≥ OPT/4 • E [f(R)] ≥ OPT/2 for symmetric f Simple Local Search: • • Initialize S to best singleton S = local optimum for adding or deleting if improvement Output better of S and NS 1/3 approx for non-negative f and ½ for symmetric f

Local Search Analysis Lemma: If S is a local optimum then for any I S or S I, f(I) ≤ f(S). Proof: Say S I and f(I) > f(S) then by submodularity there exists i in IS s. t f(S+i) > f(S). Corollary: Let S* be an optimum solution and S be a local opt. f(S S*) ≤ f(S) and f(S S*) ≤ f(S)

Local Search Analysis f(S S*) ≤ f(S) and f(S S*) ≤ f(S) NS S S* f(S S*) + f(NS) ≥ f(S*S) + f(N) ≥ f(S*S) 2 f(S) + f(NS) ≥ f(S*S) + f(S* S) ≥ f(S*) implies max (f(S), f(NS)) ≥ f(S*)/3

Maximizing Submodular Set Functions with Constraints max S N f(S) S satisfies some constraints Question: what constraints? For maximization probs, packing constraints natural S I I is a downward-closed: A I, B A implies B I

Matroid and Knapsack Constraints • Combinatorial packing constraints • I is the intersection of some p matroids on N • Lemma: every downward-close family I on N is the intersection of p matroids on N (for some p) • Knapsack or matrix packing constraints • A is a m x n non-negative matrix, b is m x 1 vector • I = { x {0, 1}n| A x ≤ b } • Combination of matroid and knapsack constraints

Matroid Constraints • Uniform matroid: I = { S : |S| ≤ k } • Partition matroid: I = { S : |S Ni| ≤ ki, 1 ≤ i ≤ h } where N 1, . . . , Nh partition N, and ki are integers • Laminar matroid: I = { S : |S U| ≤ k(U), U in F } for a laminar family of sets F • Graphic matroid Matroid polytope is integral and hence one can hope to capture constraints via relaxation in polytope

Monotone Functions

Cardinality Constraint max f(S) such that |S| ≤ k • Max k-Cover problem is special case • Greedy gives (1 -1/e) approximation [Nemhauser. Wolsey-Fisher’ 78] • Unless P=NP no better approximation [Feige’ 98] • Many applications, routinely used

Greedy Algorithm 1. S = 2. While |S| < k do • i argmaxj f. S(j) • S S+i 3. Output S

Greedy Analysis • Sj : first j elements picked by Greedy • f(S) = δ 1 + δ 2. . . +δk • δj ≥ (OPT – f(Sj-1))/k (monotonicity and submod) • f(S) ≥ (1 -1/k)k OPT

A Different Analysis • S = {i 1, i 2, . . . , ik } picked by Greedy in order • S* = {i*1, i*2, . . . , i*k } an optimum solution • Form perfect matching between S and S* s. t elements in S S* are matched to themselves S S*

A Different Analysis • Form perfect matching between S and S* Renumber S* s. t ij is matched to i*j • For each j, because Greedy chose i*j instead of i*j δj = f(Sj-1+ ij) – f(Sj-1) ≥ f(Sj-1+ i*j) – f(Sj-1) ≥ f. S(i*j) • Summing up f(S) ≥ Σj f. S(i*j) ≥ f. S(S*) = f(S S*) – f(S) hence f(S) ≥ f(S S*)/2 ≥ f(S*)/2 if f is monotone

Why weaker analysis? Let M=(N, I) be any matroid on N Solve max f(S) such that S I [Nemhauser-Wolsey-Fisher’ 78] Theorem: Greedy is ½ approximation, and analysis is tight even for partition matroids Many applications, unfortunately unknown, till recently, to approximation algorithms community!

Base Exchange Theorem • B and B’ are distinct bases in a matroid M=(N, I) Strong Base Exchange Theorem: There are elements i BB’ and i’ B’B such that B-i+i’ and B’-i’+i are both bases. Corollary: There is a perfect matching between BB’ and B’B such that for each matched pair (i, i’), Bi+i’ is a base.

Proof for Greedy • B is the solution that Greedy outputs • B* is an optimum solution • B and B* are bases of M by monotonicity of f Same argument as before works by using perfect matching between B and B*

Multiple Matroids max f(S) such that S is in intersection of p matroids [Fisher-Nemhauser-Wolsey’ 78] Theorem: Greedy is 1/(p+1) approximation Generalize matching argument to match one element of Greedy to p elements of OPT Also works for p-systems

Non-negative functions? max f(S) such that S is in intersection of p matroids [Lee-Mirrokni-Nagarajan-Sviridenko’ 08] Theorem: For fixed p, local search based algorithm that achieves Ω(1/p) approximation. [Gupta-Roth-Schoenbeck-Talwar’ 09] For all p and with simple proof combining Greedy and unconstrained algorithm. Slightly worse constants.

Knapsack Constraints Similar ideas can be used with standard guessing large items etc

Can we do better? • ½ for one matroid, hardness is (1 -1/e) • 1/(p+1) for p matroids, hardness is (p/log p)

Can we do better? [Calinescu-C-Pal-Vondrak’ 07]+[Vondrak’ 08]=[CCPV’ 09] Theorem: There is a randomized (1 -1/e) approximation for maximizing a monotone f subject to a matroid constraint.

Can we do better? [Calinescu-C-Pal-Vondrak’ 07]+[Vondrak’ 08]=[CCPV’ 09] Theorem: There is a randomized (1 -1/e) approximation for maximizing a monotone f subject to a matroid constraint. [Lee-Sviridenko-Vondrak’ 09] Theorem: For fixed p≥ 2, there is a local-search based 1/(p+ε) approximation for intersection of p matroids New useful insight for (two) matroid intersection

Multilinear Extension of f Question: Is there a useful continuous relaxation of f such that it can be optimized? And can we round it effectively?

Multilinear Extension of f [CCPV’ 07] inspired by [Ageev-Sviridenko] For f: 2 N R+ define F: [0, 1]N R+ as x = (x 1, x 2, . . . , xn) [0, 1]|N| F(x) = Expect[ f(x) ] = S N f(S) px(S) = S N f(S) i S xi i NS (1 -xi)

Multilinear Extension of f For f: 2 N R+ define F: [0, 1]N R+ as F(x) = S N f(S) i S xi i NS (1 -xi) F is smooth submodular ([Vondrak’ 08]) • F/ xi ≥ 0 for all i (monotonicity) • 2 F/ xi xj ≤ 0 for all i, j (submodularity)

Optimizing F(x) [Vondrak’ 08] Theorem: For any down-monotone polytope P [0, 1]n max F(x) s. t x P can be optimized to within a (1 -1/e) approximation if we can do linear optimization over P Algorithm: Continuous-Greedy

Generic Approach Want to solve: max f(S) s. t S I Relaxation: max F(x) s. t x P(I ) • P(I) is a polytope that captures/relaxes S I • Can solve to within (1 -1/e) with continuous greedy • How to round? F is a non-linear function

Matroid Polytope Let M=(N, I) be a matroid • P(M) is the independent set polytope of M x≥ 0 x(S) ≤ rank. M(S) for all S N • B(M) is the base polytope x(N) = rank. M(N) of M: add

Rounding in Matroids Let M=(N, I) be a matroid • P(M) is the independent set polytope of M • B(M) is the base polytope of M Algorithm: • Run continuous greedy to obtain a point x B(M) such that F(x) ≥ (1 -1/e) OPT • Round x to a vertex of B(M) (a base)

Rounding in Matroids [CCPV’ 07] Theorem: Given any point x in B(M), there is a polynomial time algorithm to round x to a vertex x* (hence a base of M) such that F(x*) ≥ F(x). “Pipage” rounding technique of [Ageev-Sviridenko] adapted to matroids

New Rounding Method [C-Vondrak-Zenklusen’ 09] Randomized Swap-Rounding: • Express x = mj=1 βi Bi (convex comb. of bases) • B = B 1 , β = β 1 • For k = 2 to m do • Randomly Merge β B and βk Bk into (β + βk) B’ • Set B = B’, β = (β + βk) • Output B

Merging two Bases Merge B’ and B’’ into a random B that looks “half” like B’ and “half” like B’’

Merging two Bases Base Exchange. Theorem: B’ B’’ i B’ B’’ j B’ B’-i+j and B’’-j+i are both bases

Merging two Bases B’ i prob ½ B’ B’’ i B’ B’’ j B’ B’ prob ½ B’’ j B’ B’’

Swap Rounding for Matroids Theorem: Swap-Merging with input x in B(M) outputs a random base B such that 1. E[f(B)] ≥ F(x) and 2. Pr[ f(B) < (1 -δ) F(x)] ≤ exp(- F(x) δ 2/8) (concentration for lower tail of submod functions) and 3. For any vector a [0, 1]n , let μ = ax then • • Pr[a 1 B < (1 -δ) μ] ≤ exp(- μ δ 2/2) Pr[a 1 B > (1+δ) μ] ≤ ( e δ / (1+δ) δ ) μ (concentration for linear functions) Almost like independent random rounding of x

Applications Can handle matroid constraint plus packing constraints x P(M) and Ax ≤ b • (1 -1/e) approximation for submodular functions subject to a matroid plus O(1) knapsack/packing constraints (or many “loose” packing constraints) • Simpler rounding and proof for “thin” spanning trees in ATSP application ([Asadpour etal’ 10]) • . . .

Other Tools On-going work [C-Vondrak-Zenklusen] • Extension of continuous greedy to handle multiple submodular functions simultaneously • Depending rounding via swap method for polyhedra • matroid intersection • matchings and b-matchings in non-bipartite graphs • . . . Many applications of dependent randomized rounding[Arora-Frize-Kaplan, Srinivasan’ 01, . . , Asadpour etal, . . ]

Illustrative Application Maximum Bipartite Flow in Networks with Adaptive Channel Width [Azar-Madry-Moscibroda. Panigrahy-Srinivasan’ 09] • Problem motivated by capacity allocation in wireless networks • (1 -1/e) approximation via a specialized LP and complicated analysis • An easy O(log n) approximation

Problem Definition θ(B, C) : threshold capacity between B and C Base stations α(C): max flow that C desires from base stns β(B): total capacity of B to serve clients For each base station B, decide an operating point τ(B) ≤ β(B) • If τ(B) > θ(B, C) then u(B, C) capacity of link (B, C) is 0, otherwise u(B, C) = τ(B) • Maximize flow from base stations to clients Clients u(B, C) α(C) β(B)

Reduction Base stations B Copies of B one for each τ(B) Clients α(C) t u(τ(B), C) β(B) N : all copies on left f : 2 N R+ where f(S) = flow from S to t Partition Matroid constraint: can pick only one copy for each B

Summary • Substantial progress on submodular function maximization problems • Increased awareness and more applications • (New) tools and connections: continuous greedy, dependent rounding, local search, . . . • Several open problems still remain

Open Problems • Unconstrained for non-neg f. Close gap bet 0. 41 and ½ • max f(S) s. t S in intersection of p matroids • • • p=1 we have 1 -1/e approx which is tight unless P=NP for fixed p ≥ 2, 1/(p+ε) approx, otherwise 1/(p+1) hardness is Ω (p/log p) for large p when f is modular close gaps, most interesting for small p How to round F(x) for more than 1 matroid? Don’t know integrality gap for p=2! • max f(x) s. t x in P(M) and Ax ≤ b, A is k-columnsparse. Want Ω(1/k) approx. Known without matroid constraint [Bansal-Korula-Nagarajan-Srinivisan’ 10]