Hypercontractivity Sums of Squares Ryan ODonnell Carnegie Mellon

Hypercontractivity & Sums of Squares Ryan O’Donnell Carnegie Mellon

Motivation: Optimization Max-Independent-Set Min-Vertex-Cover Colouring Sparsest-Cut Balanced-Separator Max-Cut Unique-Games Small-Set-Expansion

Max Independent-Set: LP relaxation MAX-IS(G) = max {|A| : A ⊆ V, no edge internal to A} For each v ∈ V, have variable… Independent-Set constraints… f(v) ∈ {0, 1} f(v)+f(w) ≤ 1 LP-OPT(G) = max [0, 1] ∀ (v, w)∈E

Max Independent-Set: Value Certification “I certify the max. independent set is ≤ LP-OPT(G)” G LP

Max Independent-Set: N-vertex complete graph KN Value Certification I certify the max. independent set is ≤ N/2 LP Truth: MAX-IS(KN) = 1

Max Independent-Set: Value Certification Recall: N-vertex I certify the Max-IS(G) = N − MIN-VERTEX -COVER (G) complete max. independent graph set is ≤ N/2 KN LP Truth: MAX-IS(KN) = 1

Min Vertex-Cover: N-vertex complete graph KN Value Certification I certify the min. vertex cover set is ≥ N/2 LP This is the famous factor-2 integrality gap for Vertex-Cover. Truth: MIN-VC(KN) = N− 1

Max Independent-Set: N-vertex complete graph KN Value Certification I certify the max. independent set is ≤ 1 SDP LP Truth: MAX-IS(KN) = 1

Max Independent-Set: I certify the max. independent set is ≤ N/3 Frankl–Rödl graph ¼ FR N Value Certification SDP [KG’ 95] Truth: ¼ FR N) ≤ N. 99 MAX-IS(FR [FR’ 87]

Max Independent-Set: I certify the max. independent set is ≤ N/3 Frankl–Rödl graph ¼ FR N Value Certification SDP [KG’ 95] Shoot, it might even be 3 -colourable…

Frankl–Rödl Graph ¼ FR N defined assuming N = 2 n (and ¼ n is even) V = {0, 1}n E = { (v, w) : Δ(v, w) = ¾ n } {0, 1}n

Frankl–Rödl Graph γ FR N defined assuming N = 2 n (and γ n is even) V = {0, 1}n E = { (v, w) : Δ(v, w) = (1−γ) n } Neighbours of v: all ways of flipping a γ-fraction of coords {0, 1}n 0<γ<½

Max Independent-Set: I certify the max. independent set is ≤ N/3 Frankl–Rödl graph ¼ FR N Value Certification SDP [KG’ 95] Truth: ¼ FR N) ≤ N. 99 MAX-IS(FR [FR’ 87]

Max Independent-Set: I certify the max. independent set is ≤ Frankl–Rödl graph γ FR N Value Certification SDP [KG’ 95] Truth: γ FR N) ≤ o(N) MAX-IS(FR (provided γ ≥ )

Max Independent-Set: Frankl–Rödl graph γ FR N Lovász– Schrijver LP Value Certification I certify the max. independent set is ≤ [ABL’ 02] Truth: γ FR N) ≤ o(N) MAX-IS(FR (provided γ ≥ )

Max Independent-Set: Frankl–Rödl graph γ FR N Lovász– Schrijver SDP Value Certification I certify the max. independent set is ≤ [GMPT’ 07] Truth: γ FR N) ≤ o(N) MAX-IS(FR (provided γ ≥ )

Max Independent-Set: Frankl–Rödl graph γ FR N Sherali– Adams SDP Value Certification I certify the max. independent set is ≤ [BCGM’ 11] Truth: γ FR N) ≤ o(N) MAX-IS(FR (provided γ ≥ )

Max Independent-Set: Frankl–Rödl graph Value Certification How did they certify it? ! γ FR N Truth: γ MAX-IS( FR N) ≤ o(N) [FR’ 87]

How Frankl & Rödl proved it It wasn’t particularly easy. Erdős gave them $250 for it. The central lemma at the very core of the proof was Harper’s Hypercube Vertex-Isoperimetry Theorem.

Hypercube Vertex-Isoperimetry Let A ⊆ {0, 1}n be not too small, say size ≥ N. 99. Then {w : Δ(w, A) ≤ ¼ n} is almost all of {0, 1}n. [Harper’ 66] gave the precise statement: For any γ in place of ¼, and any fixed size for A, the “worst” A is a Hamming ball.

Hypercube Vertex-Isoperimetry Harper’s proof uses “combinatorial shifting”. So arguably we can say that, as proof systems, LP SDP Lovász–Schrijver LP Lovász–Schrijver SDP Sherali–Adams SDP cannot “do” combinatorial shifting.

Balanced-Separator, Unique-Games, Sparsest-Cut, Small-Set Expansion… Khot–Vishnoi graph γ KV N SDP+Δ I think there’s a small set where most edges touching it are internal. Sherali– Adams… Truth: ∀ small sets, almost all edges touching it are crossing. [KV’ 05] [DKSV’ 06] [KS’ 09] [RS’ 09] …

Core theorem: Hypercube Small-Set Expansion [KKL’ 88] Let A ⊆ {0, 1}n be not too large. Pick v∈A at random, then pick w by flipping each coordinate with probability γ. Then almost surely, w∉A. (Asymptotically, Hamming balls are worst. ) Proof technique: Hypercontractive Inequality.

Hypercube SSE: 2 versions V = {0, 1}n, E = {(v, w) : Δ(v, w) ≈ γn}, 0 < γ < ½ I. Almost all edges escape II. Practically everywhere sees at least one edge A not too big ⇒ almost all touching edges go out A A not too small ⇒ ∃ edge to practically everywhere A

Hypercube SSE: 2 versions V = {0, 1}n, E = {(v, w) : Δ(v, w) ≈ γn}, 0 < γ < ½ I. Almost all edges escape II. Practically everywhere sees at least one edge A not too big ⇒ almost all touching edges go out Proof: Hypercontractive Inequality A not too small ⇒ ∃ edge to practically everywhere Proof: Combinatorial shifting

Hypercube SSE: 2 versions V = {0, 1}n, E = {(v, w) : Δ(v, w) ≈ γn}, 0 < γ < ½ I. Almost all edges escape A not too big ⇒ almost all touching edges go out Proof: Hypercontractive Inequality II′. Most places get at least a few edges A not too small ⇒ random edges will go almost everywhere Proof: Reverse Hypercontractive Inequality [MORSS’ 05], [BHM’ 12] for density Frankl–Rödl

Hypercube SSE: 2 versions V = {0, 1}n, E = {(v, w) : Δ(v, w) ≈ γn}, 0 < γ < ½ I. Almost all edges escape A II′. Most places get at least a few edges A Two core principles for “SDP integrality gaps”. LP, SDP, LS+, SA+, etc. cannot prove them.

What about SOS? SOS (“Sum of Squares”) is another SDP hierarchy / proof system. The “constant round/degree” version is poly-time. Known that SOS ≫ LP, SDP, LS+, SA+ [LRS’ 15]: Indeed, for “Max-CSPs”, SOS ≫ any poly-size SDP relaxation. Can SOS “do” the two Hypercube SSE principles?

What is the SOS proof system? It’s not hard to describe, but I won’t do so fully, for lack of time.

Max Indep. -Set: Polynomial formulation For each v ∈ V, have variable… Independent-Set constraints… f(v) ∈ {0, 1} f(v)f(w) = 0 ? ≥B f(v)2 = f(v) ∀ (v, w)∈E

The SOS idea Replace a polynomial inequality condition like “P(f 1, …, f. N) ≥ 0” (modulo constraints) with “P(f 1, …, f. N) is a sum of squares of polynomials of constant degree” (modulo constraints)

SOS and Hypercontractivity Can the Hypercontractive Inequality and the Reverse Hypercontractive Inequality be given a polynomial formulation, and then proved by the SOS method? Answers: Yes [BBHKSZ’ 12], [OZ’ 13], [KOTZ’ 14] and Yes [KOTZ’ 14], though the “degree” depends inversely on the γ parameter.

SOS and Hypercontractivity Corollaries: SOS gives satisfactory certifications for the Khot–Vishnoi-based hard instances of Unique-Games [BBHKSZ’ 12] and Balanced-Separator [OZ’ 13]. SOS gives somewhat satisfactory certifications for the Frankl–Rödl-based hard instances of Independent-Set & Vertex-Cover [KOTZ’ 14].

A little bit about the SOS formulation and proof for Reverse Hypercontractivity

Hypercube SSE: 2 versions V = {0, 1}n, E = {(v, w) : Δ(v, w) ≈ γn}, 0 < γ < ½ I. Almost all edges escape A not too big ⇒ almost all touching edges go out Proof: Hypercontractive Inequality II′. Most places get at least a few edges A not too small ⇒ random edges will go almost everywhere Proof: Reverse Hypercontractive Inequality [MORSS’ 05], [BHM’ 12] for density Frankl–Rödl

Hypercube SSE: 2 versions V = {0, 1}n, E = {(v, w) : Δ(v, w) ≈ γn}, 0 < γ < ½ II′. Most places get at least a few edges A not too small ⇒ random edges will go almost everywhere A Proof: Reverse Hypercontractive Inequality [MORSS’ 05], [BHM’ 12] for density Frankl–Rödl

Theorem: cor. of [MORSS’ 05] Let A, B ⊆ {0, 1}n have “volume” α each. (I. e. , |A|/2 n = |B|/2 n = α. ) Pick v ~ {0, 1}n uniformly, then flip each coord. with probability ¼ to form w∈{0, 1}n. Then I. e. , Pr [v∈A and w∈B] ≥ α 4 Pr [w∈B | v∈A] ≥ α 3 How to SOS-prove this? How to prove this at all? Proof by generalization.

Generalization: [MORSS’ 05] Let A, B ⊆ {0, 1}n have “volume” α each. (I. e. , |A|/2 n = |B|/2 n = α. ) Pick v ~ {0, 1}n uniformly, then flip each coord. with probability ¼ toγform w∈{0, 1}n. Then Pr [v∈A and w∈B] ≥ 1/γ 4 α

Further generalization: [MORSS’ 05] Let A, B ⊆ {0, 1}n have “volume” α respectively. , β Pick v ~ {0, 1}n uniformly, then flip each coord. with probability ¼ toγform w∈{0, 1}n. Then Pr [v∈A and w∈B] ≥ 4 p(α, β, γ) where p(α, β, γ) is a nice explicit expression.

Further further generalization: [B’ 82, MORSS’ 05] Let f, g : {0, 1}n → ℝ≥ 0. Let r, s ≥ 0 satisfy . Pick v ~ {0, 1}n uniformly, then flip each coord. with probability ¼ toγform w∈{0, 1}n. Then E [f(v) g(w)] ≥ ||f||1−r ||g||1−s The “(2 -function) Reverse Hypercontractive Inequality” Implies previous slide by taking f = 1 A, g = 1 B (and choosing r, s appropriately).

Further further generalization: [B’ 82, MORSS’ 05] Let f, g : {0, 1}n → ℝ≥ 0. Let r, s ≥ 0 satisfy . Pick v ~ {0, 1}n uniformly, then flip each coord. with probability ¼ toγform w∈{0, 1}n. Then E [f(v) g(w)] ≥ ||f||1−r ||g||1−s Given the n = 1 case, the general n case follows by an utterly trivial induction. But only because we have a 2 -function version!

Equivalent n=1 specialization: [B’ 82, MORSS’ 05] Let f, g : {0, 1} → ℝ≥ 0. Let r, s ≥ 0 satisfy . Pick v ~ {0, 1}n uniformly, then flip it coord. with probability ¼ toγform w∈{0, 1} Then E [f(v) g(w)] ≥ ||f||1−r ||g||1−s Fixing the “parameters” γ, r, s, this is just an inequality about 4 real numbers, namely f(0), f(1), g(0), g(1).

Final statement: [B’ 82, MORSS’ 05] Let f, g : {0, 1} → ℝ≥ 0. Let r, s ≥ 0 satisfy . Pick v ~ {0, 1}n uniformly, then flip it coord. with probability ¼ toγform w∈{0, 1} Then E [f(v) g(w)] ≥ ||f||1−r ||g||1−s Still not easy! Reverse Hölder inequality and homogeneity tricks reduce to an inequality about 1 real number. Then proved by Taylor series.

Final statement: [B’ 82, MORSS’ 05] Let f, g : {0, 1} → ℝ≥ 0. Let r, s ≥ 0 satisfy . Pick v ~ {0, 1}n uniformly, then flip it coord. with probability ¼ toγform w∈{0, 1} Then E [f(v) g(w)] ≥ ||f||1−r ||g||1−s Now how do we prove this using SOS? ! Not even a polynomial statement, due to non-integer norms.

Final statement: [B’ 82, MORSS’ 05] Let f, g : {0, 1} → ℝ≥ 0. Let r, s ≥ 0 satisfy . Pick v ~ {0, 1}n uniformly, then flip it coord. with probability ¼ toγform w∈{0, 1} Then E [f(v) g(w)] ≥ ||f||1−r ||g||1−s For the Independent-Set application, it’s enough to consider A, B of equal volume α. Implies it’s enough to consider r = s = 1− 2γ.

Final statement: [B’ 82, MORSS’ 05] Let f, g : {0, 1} → ℝ≥ 0. Let r, s ≥ 0 satisfy . Pick v ~ {0, 1}n uniformly, then flip it coord. with probability ¼ toγform w∈{0, 1} Then E [f(v) g(w)] ≥ ||f||2γ ||g||2γ Replace f, g with f 1/(2γ), g 1/(2γ).

Final statement: [B’ 82, MORSS’ 05] Let f, g : {0, 1} → ℝ≥ 0. Let r, s ≥ 0 satisfy . Pick v ~ {0, 1}n uniformly, then flip it coord. with probability ¼ toγform w∈{0, 1} Then E[f(v)1/(2γ)g(w)1/(2γ)] ≥ E[f]1/(2γ)E[g]1/(2γ) Assume 1/(2γ) = 2 k for some integer k. E. g. , γ = ¼ corresponds to k = 1.

Final statement: [B’ 82, MORSS’ 05] Let f, g : {0, 1} → ℝ Let k∈ℕ. Let r, s ≥ 0 satisfy . Pick v ~ {0, 1}n uniformly, then flip it 1/(4 k) coor with probability ¼ to form w∈{0, 1} Then E[f(v)2 k g(w)2 k] ≥ E[f]2 k E[g]2 k For each fixed k, this is a polynomial inequality in 4 real variables: f(0), f(1), g(0), g(1). By homogeneity, reduces to ineq. on 2 variables.

Final final statement: [B’ 82, MORSS’ 05] Let k∈ℕ. Then ≥ 0. We implicitly know this statement is true. What we want to show is that polynomial is a sum of squares of polynomials in a and b.

Final final statement: [B’ 82, MORSS’ 05] Let k∈ℕ. Then ≥ 0. For k = 1, the polynomial is equal to… (ab)2 + (a+b)2

Final final statement: [B’ 82, MORSS’ 05] Let k∈ℕ. Then ≥ 0. For k = 2, the polynomial is equal to…

Final final statement: [B’ 82, MORSS’ 05] Let k∈ℕ. Then ≥ 0. For k = 3, the polynomial is equal to…

Final final statement: [B’ 82, MORSS’ 05] Let k∈ℕ. Then ≥ 0. So we were pretty sure it was SOS for all k. But we could find no pattern.

One month later… We reduced it to showing, for each k, that a certain infinite sequence of ugly univariate polynomials are all SOS. Fact: A univariate polynomial is SOS iff it is nonnegative. So we just had to show all those univariate polynomials are nonnegative.

Another month later… We showed all those univariate polynomials are nonnegative. More accurately, using “automated combinatorial identity proving” tricks, we coaxed our computer to assist us in producing incomprehensible proofs of nonnegativity.

Another month later… We showed all those univariate polynomials are nonnegative.

Summary Can the Hypercontractive Inequality and the Reverse Hypercontractive Inequality be given a polynomial formulation, and then proved by the SOS method? Answers: Yes [BBHKSZ’ 12], [OZ’ 13], [KOTZ’ 14] and Yes [KOTZ’ 14], though the “degree” depends inversely on the γ parameter.

Summary Corollaries: SOS gives satisfactory certifications for the Khot–Vishnoi-based hard instances of Unique-Games [BBHKSZ’ 12] and Balanced-Separator [OZ’ 13]. SOS gives somewhat satisfactory certifications for the Frankl–Rödl-based hard instances of Independent-Set & Vertex-Cover [KOTZ’ 14].

Open Problems 1 & 2 Each corollary would be much more satisfactory if we could SOS-prove a certain simple Hypercube SSE statement. For each of the two desired statements, I can tell you an extremely simple “analytic” proof. (Ask me after the talk. ) Prove or disprove for either statement: There is a degree-4 SOS proof.

Open Problem 3 Give a human-comprehensible SOS-proof of that family of 2 -variable polynomial inequalities!

Thank you! �� ! Terima kasih! �����!