Submodular set functions Set function z on V
Submodular set functions • Set function z on V is called submodular if For all A, B µ V: z(A)+z(B) ¸ z(A[B)+z(AÅB) + AA[B B ¸ + AÅB • Equivalent diminishing returns characterization: + S Large improvement Submodularity: B A + S Small improvement For AµB, s B, z(A [ {s}) – z(A) ¸ z(B [ {s}) – z(B) 3
Submodularity and convexity For V = {1, …, n}, and A µ V, let 1 A = (w 1 A, …, wn. A) with wi. A = 1 if i 2 A, 0 otherwise Key result [Lovasz ’ 83]: Every submodular set function z induces a function g on Rn+, such that (i) z(A) = g(1 A) for all A µ V (ii) g(w) is convex (iii) min. A z(A) = minw g(w) s. t. w 2 [0, 1]n 11
The submodular polyhedron Pz Pz = {x 2 Rn: x(A) · z(A) for all A µ V} Example: V = {a, b} A ; {a} {b} {a, b} x(A) = i 2 A xi x{b} 2 Pz x({b}) · z({b}) 1 -2 -1 0 z(A) 0 -1 2 0 x({a, b}) · z({a, b}) 1 x{a} x({a}) · z({a}) 12
Evaluating g(w) = maxx 2 Pz w. Tx xw -2 -1 x{b} 2 1 w 0 1 x{a} Pz = {x 2 Rn: x(A) · z(A) for all A µ V} Theorem [Edmonds ’ 71, Lovasz ‘ 83]: For any given w, can get optimal solution xw to the LP using the following greedy algorithm: 1. Order V={e 1, …, en} so that w(e 1)¸ …¸ w(en) 2. Let xw(ei) = z({e 1, …, ei}) – z({e 1, …, ei-1}) Then w. T xw = g(w) = maxx 2 Pz w. T x 13
Ellipsoid algorithm [Grötschel, Lovasz, Schrijver ’ 81] Feasible region minc, w c s. t. c ¸ w. T x for all x 2 Pz w 2 [0, 1]n Separation oracle: Find most violated constraint: maxx w. T x – c s. t. x 2 Pz Can solve separation using the greedy algorithm!! Ellipsoid algorithm minimizes SFs in poly-time! 14
Minimizing submodular functions Ellipsoid algorithm not very practical Want combinatorial algorithm for minimization! Theorem [Iwata (2001)] There is a fully combinatorial, strongly polynomial algorithm for minimizing SFs, that runs in time O(n 8 log 2 n) Polynomial-time = Practical ? ? ? 15
A more practical alternative? [Fujishige ’ 91, Fujishige et al ‘ 06] x({a, b})=z({a, b}) x* [-1, 1] -2 -1 x{b} 2 1 Base polytope: Bz = Pz Å {x(V) = z(V)} 0 Minimum norm algorithm: 1 A ; {a} {b} {a, b} z(A) 0 -1 2 0 x{a} M 1. Find x* = argmin ||x||2 s. t. x 2 Bz 2. Return A* = {i: x*(i) < 0} x*=[-1, 1] A*={a} Theorem [Fujishige ’ 91]: A* is an optimal solution! Note: Can solve 1. using Wolfe’s algorithm Runtime finite but unknown!! 16
Empirical comparison Running time (seconds) Lower is better (log-scale!) [Fujishige et al ’ 06] Cut functions from DIMACS Challenge Minimum norm algorithm 64 128 256 512 1024 Problem size (log-scale!) Minimum norm algorithm orders of magnitude faster! Matlab implementation can solve n = 10 k in < 6 minutes! 17
- Slides: 18