On the Unique Games Conjecture Subhash Khot Georgia

On the Unique Games Conjecture Subhash Khot Georgia Inst. Of Technology. At FOCS 2005

NP-hard Problems • • Vertex Cover MAX-3 SAT Bin-Packing Set Cover Clique MAX-CUT ……………. .

Approximability : Algorithms A C-approximation algorithm computes (C > 1), for problem instance I , solution A(I) s. t. Minimization problems : A(I) C OPT(I) Maximization problems : A(I) OPT(I) / C

Some Known Approximation Algorithms • Vertex Cover 2 - approx. • 8/7 - approx. Random assignment. MAX-3 SAT • Packing/Scheduling (1+ ) – approx. > 0 (PTAS) • Set Cover ln n • Clique n/log n • approx. [Boppana Halldorsson’ 92] Many more , ref. [Vazirani’ 01]

PCP Theorem [B’ 85, GMR’ 89, BFL’ 91, LFKN’ 92, S’ 92, ……] [PY’ 91] [FGLSS’ 91, AS’ 92 ALMSS’ 92] Theorem : It is NP-hard to tell whether a MAX-3 SAT instance is * satisfiable (i. e. OPT = 1) or * no assignment satisfies more than 99% clauses (i. e. OPT 0. 99). i. e. MAX-3 SAT is 1/0. 99 = 1. 01 hard to approximate. i. e. MAX-3 SAT and MAX-SNP-complete problems [PY’ 91] have no PTAS.

Approximability : Towards Tight Hardness Results • [Hastad’ 96] • [Hastad’ 97] • [Feige’ 98] Clique n 1 - MAX-3 SAT 8/7 - Set Cover (1 - ) ln n [Dinur’ 05] Combinatorial Proof of PCP Theorem !

Open Problems in Approximability – Vertex Cover (1. 36 vs. 2) [Dinur. Safra’ 02] – Coloring 3 -colorable graphs (5 vs. n 3/14) [Khanna. Linial. Safra’ 93, Blum. Karger’ 97] – Sparsest Cut (1 vs. (logn)1/2) [Arora. Rao. Vazirani’ 04] – Max Cut (17/16 vs 1/0. 878… ) [Håstad’ 97, Goemans. Williamson’ 94] ……………. .

Unique Games Conjecture [Khot’ 02] • • Implies these hardness results : Vertex Cover 2 - [KR’ 03] Coloring 3 -colorable (1) [DMR’ 05] graphs (variant of UGC) MAX-CUT 1/0. 878. . - [KKMO’ 04] Sparsest Cut, Multi-cut [KV’ 05, (1) CKKRS’ 04] Min-2 SAT-Deletion [K’ 02, CKKRS’ 04]

Unique Games Conjecture Led to … [MOO’ 05] Majority Is Stablest Theorem [KV’ 05] “Negative type” metrics do not embed into L 1 with O(1) “distortion”. Optimal “integrality gap” for MAX-CUT SDP with “Triangle Inequality”.

Integrality Gap : Definition Given : Maximization Problem + Specific SDP relaxation. • For every problem instance G, SDP(G) OPT(G) • Integrality Gap = Max G SDP(G) / OPT(G) • Constructing gap instance = negative result.

Overview of the talk • • The UGC Hardness of Approximation Results I hope UGC is true Attempts to Disprove : Algorithms Connections/applications : • Fourier Analysis • Integrality Gaps • Metric Embeddings

Unique Games Conjecture • A maximization problem called “Unique Game” is hard to approximate. • “Gap-preserving” reductions from Unique Game Hardness results for Vertex Cover, MAX-CUT, Graph-Coloring, …. .

Example of Unique Game OPT = max fraction of equations that can be satisfied by any assignment. x 1 + x 3 = 2 (mod k) 3 x 5 - x 2 = -1 (mod k) x 2 + 5 x 1 = 0 (mod k) UGC For large k, it is NP-hard to tell whether OPT 99% or OPT 1%

2 -Prover-1 -Round Game (Constraint Satisfaction Problem ) variables constraints

2 -Prover-1 -Round Game (Constraint Satisfaction Problem ) variables k labels Here k=4 constraints

2 -Prover-1 -Round Game (Constraint Satisfaction Problem ) variables k labels Here k=4 Constraints = Bipartite graphs or Relations [k]

2 -Prover-1 -Round Game (Constraint Satisfaction Problem ) Find a labeling that satisfies max # constraints variables k labels Here k=4 OPT(G) = 7/7

Hardness of Finding OPT(G) • Given a 2 P 1 R game G, how hard is it to find OPT(G) ? • PCP Theorem + Raz’s Parallel Repetition Theorem : For every , there is integer k( ), s. t. it is NP-hard to tell whether a 2 P 1 R game with k = k( ) labels has OPT = 1 or OPT In fact k = 1/poly( )

Reductions from 2 P 1 R Game • Almost all known hardness results (e. g. Clique, MAX-3 SAT, Set Cover, SVP, …. ) are reductions from 2 P 1 R games. • Many special cases of 2 P 1 R games are known to be hard, e. g. Multipartite graphs, Expander graphs, Smoothness property, …. What about unique games ?

Unique Game = 2 P 1 R Game with Permutations variable k labels Here k=4

Unique Game = 2 P 1 R Game with Permutations variable k labels Here k=4 Permutations : [k] or matchings

Unique Game = 2 P 1 R Game with Permutations Find a labeling that satisfies max # constraints OPT(G) = 6/7

Unique Games Considered before …… [Feige Lovasz’ 92] Parallel Repetition of UG reduces OPT(G). How hard is approximating OPT(G) for a unique game G ? Observation : Easy to decide whether OPT(G) = 1.

MAX-CUT is Special Case of Unique Game • Vertices : Binary variables x, y, z, w, ……. • Edges : Equations x + y = 1 (mod 2) • [Hastad’ 97] NP-hard to tell whether OPT(MAX-CUT) 17/21 or OPT(MAX-CUT) 16/21

Unique Games Conjecture For any , , there is integer k( , ), s. t. it is NP-hard to tell whether a Unique Game with k = k( , ) labels has OPT 1 - or OPT i. e. Gap-Unique Game (1 - , ) is NP-hard.

Overview of the talk • • The UGC Hardness of Approximation Results I hope UGC is true Attempts to Disprove : Algorithms Connections/applications : • Fourier Analysis • Integrality Gaps • Metric Embeddings

Case Study : MAX-CUT • Given a graph, find a cut that maximizes fraction of edges cut. • Random cut : 2 -approximation. • [GW’ 94] SDP-relaxation and rounding. min 0 < < 1 / (arccos (1 -2 ) / ) = 1/0. 878 … approximation. • [KKMO’ 04] Assuming UGC, MAX-CUT is 1/0. 878… - hard to approximate.

Reduction to MAX-CUT Unique Game • Completeness : OPT(UG) > 1 -o(1) Graph H - o(1) cut. • Soundness : OPT(UG) < o(1) No cut with size arccos (1 -2 ) / + o(1) • Hardness factor = / (arccos (1 -2 ) / ) - o(1) • Choose best to get 1/0. 878 … (= [GW’ 94])

Reduction from Unique Game Gadget constructed via Fourier theorem + Connecting gadgets via Unique Game instance [DMR’ 05] Gadget = “UGC reduces the analysis of the Basic gadget ---> entire. Bipartite construction the analysis gadgetto---> of the gadget”. gadget with permutation Bipartite

Basic Gadget A graph on {0, 1} k with specific properties (e. g. cuts, vertex covers, colorability) x = 011 k = # labels {0, 1} Y = 110 k

Basic Gadget : MAX-CUT Weighted graph, total edge weight = 1. Picking random edge : x R {0, 1} k y <-- flip every co-ordinate of x with probability ( 0. 8) x {0, 1} y k

MAX-CUT Gadget : Co-ordinate Cut Along Dimension i xi = 0 xi = 1 Fraction of edges cut = Pr(x, y) [xi yi ] = Observation : These are the maximum cuts.

Bipartite Gadget A graph on {0, 1} x = 011 k {0, 1} k (double cover of basic gadget) y’ = 110

Cuts in Bipartite Gadget {0, 1} k Matching co-ordinate cuts have size =

Bipartite Gadget with Permutation : [k] -> [k] Co-ordinates in second hypercube permuted via . Example : = reversal of co-ordinates. x = 011 (y’) = 011 Y ’ = 110

Reduction from Unique Game OPT 1 – o(1) or OPT o(1) Variables k labels Permutations : [k]

Instance H of MAX-CUT {0, 1} k Bipartite Gadget via Vertices Edges

Proving Completeness Unique Game Graph H (Completeness) : OPT(UG) > 1 -o(1) H has - o(1) cut.

Completeness : OPT(UG) 1 -o(1) Labels = [1, 2, 3] label = 1 label = 2 label = 3 label = 2 label = 1 label = 3

Completeness : OPT(UG) 1 -o(1) {0, 1} k Vertices Edges Hypercubes are cut along dimensions = labels. MAX-CUT - o(1)

Proving Soundness Unique Game Graph H (Soundness) : OPT(UG) < o(1) H has no cut of size arccos (1 -2 ) / + o(1)

MAX-CUT Gadget x {0, 1} k Cuts = Boolean functions f : {0, 1} k y {0, 1} f(x 1 x 2 ……. . xk) = xi Compare boolean functions * that depend only on single co-ordinate vs * where every co-ordinate has negligible “influence” functions) Influence (i. e. (i, f)“non-junta” = f(x Prx 1 x[2 f(x) = MAJORITY i) ] ……. . x k)f(x+e

Gadget : “Non-junta” Cuts How large can non-junta cuts be ? i. e. cuts with all influences negligible ? Random Cut : ½ Majority Cut : arccos (1 -2 ) / > ½ • [MOO’ 05] Majority Is Stablest (Best) Any cut slightly better than Majority Cut must have “influential” co-ordinate.

Non-junta Cuts in Bipartite Gadget {0, 1} k k [MOO’ 05] Any “special” cut with value arccos (1 -2 ) / + must define a matching pair of influential co-ordinates.

Non-junta Cuts in Bipartite Gadget {0, 1} k k f : {0, 1} k --> {0, 1} g : {0, 1} k --> {0, 1} cut > arccos (1 -2 ) / + i Infl (i, f), Infl (i, g) > (1)

Instance H of MAX-CUT {0, 1} k Bipartite Gadget via Vertices Edges

Proving Soundness • Assume arccos (1 -2 ) / + cut exists. • On /2 fraction of constraints, the bipartite gadget has arccos (1 -2 ) / + /2 cut. matching pair of labels on this constraint. This is impossible since OPT(UG) = o(1). Done !

Other Hardness Results • Vertex Cover Friedgut’s Theorem Every boolean function with low “average sensitivity” is a junta. • Sparsest Cut, Min-2 SAT Deletion Kahn. Kalai. Linial Every balanced boolean function has a co-ordinate with influence log n/n. Bourgain’s Theorem (inspired by Hastad-Sudan’s 2 -bit Long Code test) Every boolean function with low “noise sensitivity” is a junta. • Coloring 3 -Colorable Graphs [MOO’ 05] inspired.

Basic Paradigm by [BGS’ 95, Hastad’ 97] • Hardness results for Clique, MAX-3 SAT, ……. • Instead of Unique Games, use reduction from general 2 P 1 R Games (PCP Theorem + Raz). • Hypercube = Bits in the Long Code [Bellare Goldreich Sudan’ 95] • PCPs with 3 or more queries (testing Long Code). • Not enough to construct 2 -query PCPs.

Why UGC and not 2 P 1 R Games? Power in simplicity. “Obvious” way of encoding a permutation constraint. Basic Gadget ----> Bipartite Gadget with permutation.

Overview of the talk • • The UGC Hardness of Approximation Results I hope UGC is true Attempts to Disprove : Algorithms Connections/applications : • Fourier Analysis • Integrality Gaps • Metric Embeddings

I Hope UGC is True • Implies all the “right” hardness results in a unifying way. • Neat applications of Fourier theorems [Bourgain’ 02, KKL’ 88, Friedgut’ 98, MOO’ 05] • Surprising application to theory of metric embeddings and SDP-relaxations [KV’ 05]. • Mere coincidence ?

Supporting Evidence [Feige Reichman’ 04] Gap-Unique Game (C , ) is NP-hard. i. e. For every constant C, there is s. t. it is NP-hard to tell if a UG has OPT > C or OPT < . However C --> 0 as --> 0.

Supporting Evidence [Khot Vishnoi’ 05] SDP relaxation for Unique Game has integrality gap (1 - , ).

Overview of the talk • • The UGC Hardness of Approximation Results I hope UGC is true Attempts to Disprove : Algorithms Connections/applications : • Fourier Analysis • Integrality Gaps • Metric Embeddings

Disproving UGC means. . For small enough (constant) , given a UG with optimum 1 - , algorithm that finds a labeling satisfying (say) 50% constraints.

Algorithmic Results Algorithm that finds a labeling satisfying f( , k, n) fraction of constraints. [Khot’ 02] [Trevisan’ 05] [Gupta Talwar’ 05] [CMM’ 05] 1 - 1/5 k 2 1 - 1/3 log 1/3 n 1 - log n 1/k , 1 - 1/2 log 1/2 k None of these disproves UGC.

Quadratic Integer Program For Unique Game [Feige Lovasz’ 92] u 1 , u 2 , … , uk {0, 1} variable u : [k] v v 1 , v 2 , … , vk {0, 1} vi = 1 if Label(v) = i = 0 otherwise k labels

Quadratic Program for Unique Games Constraints on edge-set E. • Maximize • u (u, v) E i=1, 2, . . , k i [k], ui {0, 1} u i 2 i i≠j, = 1 ui uj = 0 ui vπ(i)

SDP Relaxation for Unique Games • • u • Maximize u (u, v) E i [k], i=1, 2, . . , k ui, vπ(i) ui is a vector. || ui ||2 i≠j [k], = 1 ui, uj = 0

[Feige Lovasz’ 92] • OPT(G) SDP(G) 1. • If OPT(G) < 1, then SDP(G) < 1. • SDP(Gm) = (SDP(G))m • Parallel Repetition Theorem for UG : OPT(G) < 1 OPT(Gm) 0

[Khot’ 02] Rounding Algorithm r uk u u 2 u 1 Label(u) = 2, r vk v 2 v Random r v 1 Label(v) = 2 Pr [ Label(u) = Label(v) ] > 1 - 1/5 k 2 Labeling satisfies 1 - 1/5 k 2 fraction of constraints in expected sense.

[CMM’ 05] Algorithm • Labeling that satisfies 1/k fraction of constraints. (Optimal [KV’ 05]) r uk r vk u 2 u 1 v 2 v 1 All i s. t. ui is “close” to r are taken as candidate labels to u. Pick one of them at random.

[Trevisan’ 05] Algorithm • Given a unique game with optimum 1 - 1/log n, algorithm finds a labeling that satisfies 50% of constraints. • Limit on hardness factors achievable via UGC (e. g. loglog n for Sparsest Cut).

[Trevisan’ 05] Algorithm Variables and constraints [Leighton Rao’ 88] Delete a few constraints and remaining graph has connected components of low diameter.

[Trevisan’ 05] Algorithm A good algorithm for graphs with low diameter.

Overview of the talk • • The UGC Hardness of Approximation Results I hope UGC is true Attempts to Disprove : Algorithms Connections/applications : • Fourier Analysis • Integrality Gaps • Metric Embeddings

Already Covered Let’s move on ….

Overview of the talk • • The UGC Hardness of Approximation Results I hope UGC is true Attempts to Disprove : Algorithms Connections/applications : • Fourier Analysis • Integrality Gaps • Metric Embeddings

[KV’ 05] Integrality Gaps for SDP-relaxations • MAX-CUT • Sparsest Cut • Unique Game Gaps hold for SDPs with “Triangle Inequality”.

Integer Program for MAX-CUT Given G(V, E) • • • Maximize i, ¼ (i, j) E |vi - vj |2 vi {-1, 1} i, j , k, + |vj - vk |2 |vi - v k|2 Triangle Inequality (Optional) : |vi - vj |2

Goemans-Williamson’s SDP Relaxation for MAX-CUT • Maximize • i, • ¼ v i R n, (i, j) E || vi - vj ||2 || vi || = 1 Triangle Inequality (Optional) : i, j , k, || vi - vj ||2 + || vj - vk ||2 || vi - v k||2

Integrality Gap for MAX-CUT • [Goemans Williamson’ 94] Integrality gap 1/0. 878. . • [Karloff’ 99] [Feige Schetchman ’ 01] Integrality gap 1/0. 878. . - SDP solution does not satisfy Triangle Inequality. Does Triangle Inequality make the SDP tighter ? NO if Unique Games Conj. is true !

Integrality Gap for Unique Games SDP v variables v 1 , v 2 , … , vk Orthonormal Bases for Rk k labels u Matchings [k] u 1 , u 2 , … , uk Unique Game G with OPT(G) = o(1) SDP(G) = 1 -o(1)

Integrality Gap for MAX-CUT with Triangle Inequality OPT(G) = o(1) PCP Reduction No large cut u 1 , u 2 , … , uk Good SDP solution u 1 u 2 u 3 ……… uk-1 uk {-1, 1}k

Overview of the talk • • The UGC Hardness of Approximation Results I hope UGC is true Attempts to Disprove : Algorithms Connections/applications : • Fourier Analysis • Integrality Gaps • Metric Embeddings

Metrics and Embeddings • Metric is a distance function on [n] such that d(i, j) + d(j, k) d(i, k). • Metric d embeds into metric with distortion 1 if i, j d(i, j).

Negative Type Metrics Given a set of vectors satisfying Triangle Inequality : i, j , k, || vi - vj ||2 + || vj - vk ||2 || vi - v k||2 d(i, j) = || vi - vj ||2 defines a metric. These are called “negative type metrics”. L 1 NEG METRICS

NEG vs L 1 Question [Goemans, Linial’ 95] Conjecture : NEG metrics embed into L 1 with O(1) distortion. O(1) Integrality Gap O(1) Approximation [Linial London Rabinovich’ 94] [Aumann Rabani’ 98] Unique Games Conjecture [Chawla Krauthgamer Kumar Rabani Sivakumar ’ 05] [KV’ 05] (1) hardness result Sparsest Cut

NEG vs L 1 Lower Bound (loglog n) integrality gap for Sparsest Cut SDP. [Khot. Vishnoi’ 05, Krauthgamer. Rabani’ 05] A negative type metric that needs distortion (loglog n) to embed into L 1.

Open Problems • (Dis)Prove Unique Games Conjecture. • Prove hardness results bypassing UGC. • NEG vs L 1 , Close the gap. (log n) vs ( log n loglog n) [Arora Lee Naor’ 04]

Open Problems • Prove hardness of Min-Deletion version of Unique Games. (log n approx. [GT’ 05]) • Integrality gaps with “k-gonal” inequalities. • Is hypercube (Long Code) necessary ?

Open Problems More hardness results, integrality gaps, embedding lower bounds, Fourier Analysis, …… “Gowers Uniformity, Influence of Variables, and PCPs”. [Samorodnitsky Trevisan’ 05] UGC Boolean k-CSP is hard to approximate within 2 k- log k Independent Set on degree D graphs is hard to approximate within D/poly(log D).

Open Problems in Approximability Traveling Salesperson Steiner Tree Max Acyclic Subgraph, Feedback Arc Set Bin-packing (additive approximation) ………… Recent progress on Edge Disjoint Paths Network Congestion Shortest Vector Problem Asymmetric k-center (log* n) Group Steiner Tree (log 2 n) Hypergraph Vertex Cover ………………

Linear Unique Games System of linear equations mod k. x 1 + x 3 = 2 3 x 5 - x 2 = -1 x 2 + 5 x 1 = 0 [KKMO’ 04] UGC in the special case of linear equations mod k.

Variations of Conjecture • 2 -to-1 Conjecture [K’ 02] [k] • -Conjecture [DMR’ 05] [k] NP-hard to color 3 -colorable graphs with O(1) colors.