Amplification and Derandomization Without Slowdown Dana Moshkovitz MIT

Amplification and Derandomization Without Slowdown Dana Moshkovitz MIT Joint work with Ofer Grossman (MIT)

Randomized Algorithms Amazingly successful endeavor: Parallel Matching, Minimum Spanning Tree, Approximate Max-Cut, Polynomial Identity Testing, … Trade certainty for efficiency: Sometimes won’t output right answer. efficiency certainty

Randomized Algorithms As A Pathway To Efficiency With Certainty always correct always efficient exp(-n) error probability error t n a t s n o c ity probabil

This Work: Positive Answers In Certain Cases To Two Questions: • Amplification: Can we decrease the error probability from 1/3 to exp(-k) without slowing down the algorithm by a factor k? • Derandomization: Can we convert a randomized algorithm into a deterministic nonuniform algorithm without a factor n slowdown? (Adleman: a randomized algorithm with error < exp(-n) gives a deterministic non-uniform n new connectio algorithm. ) to sketching!

Non-Uniform Algorithms (aka Circuits) • Sequence of algorithms, one for every input size. • Rationale: input size known in advance. • Popular computational model (sorting networks, circuit lower bounds, …). • Often non-uniform algorithms imply uniform algorithms! (e. g. , matrix mult, minimum spanning tree [Pe. Ra], …) • Can amortize the cost of finding good randomness over many inputs. input 1 input 2 input 3 . . .

Common Wisdom vs. Our Ideas via efficient testing, amplification without slowdown via sketching, deterministic non-uniform amplification multiplies time by n 1/3 deterministic nonuniform [Adleman] 2 -n error probability (decreasing)

Clique Example • Given a graph G on n vertices that has a clique on n vertices, find a set of n vertices with edge density 1 -. • Based on Goldreich-Goldwasser-Ron: Las Vegas algorithm with time O(n 221/ 3 2) and constant error probability. • Thm 1: Las Vegas algorithm with time O(n 221/ 3 2) and error probability exp small in n. • Thm 2: Non-uniform deterministic algorithm with similar run-time. • In the paper: similar results for Free Games (constraint problems on dense bipartite graphs), Max-Cut in dense graphs, Reed-Muller list decoding. Previous work: For clique, constructive Frieze-Kannan regularity lemma det algorithm with worse dependence on , [doesn’t apply to free games!]

Derandomization From Sketching inputs w/ same sketch randomness inputs Is the algorithm correct?

It’s the verifier that doesn’t distinguish inputs w/ same sketch – not the algorithm! Example: There is a clique C on |V| vertices. Algorithm samples S V. Does S contain ( - )|S| vertices in C? – Most samples do. – Verifier can tell if knows clique (|V| bits rather than|V|2). S

Derandomization From Sketching inputs w/ same sketch inputs Def: oblivious verifier gets sketch and randomness. 1) input x, randomness r, if verifier accepts on sketch(x) and r, then algorithm correct on x, r. 2) For every sketch, Pr(verifier rejects)Is <the. algorithm randomness correct? < 2 -sketch-size r x algorithm correct Pseudorandom generators vs. this method: shrinking randomness vs. shrinking inputs.

Previous Work • Method of conditional probabilities derandomizes without slowdown, but only in special cases. • Many existing connections between derandomization, circuit lower bounds, learning, compression. – All based on pseudorandom generators. • Countless instances of saving in union bound by considering representatives. E. g. , – [Gopalan, Meka, Reingold CCC’ 12]: improved pseudorandom generator against DNFs by noticing that can sparsify DNFs. • Work on derandomizing sub-linear time algorithms. – Deterministic Frieze-Kannan [Dellamonica et al’ 15] doesn’t apply to problems like free games. – [Zimand CCC’ 07]: deterministic average case algorithms with slowdown. Our contribution is in defining and designing oblivious verifiers.

Randomized Clique Algorithm [GGR] • Let C V, |C|= n, be the unknown clique. • Sample S V, |S|=100/ 2, and go over each of its S subsets U, one of them is U=S C. • Which are the other vertices in C? U – Connected to all of U. – Most of them are connected to most other vertices that are connected to all of U. • Basic Algorithm (constant error probability): Take n vertices connected to U with largest fraction of neighbors among vertices connected to U. • Lem: With prob 2/3 average is > 1 -.

The Sketch: Pick random R V, |R|=polylogn. Sketch is all edges touching R. R connected to U • Intuition: the vertices in R connected to U “represent” the vertices connected to U, – i. e. , let one estimate for every U and v, what fraction of vertices connected to U are v’s neighbors? • Size of sketch is ~n. For derandomization need an algorithm with error probability exp small in ~n. • Will show to amplify the basic randomized algorithm without a slowdown. • We’ll argue: sketch suffices to verify this more complicated algorithm.

Amplification: The Biased Coin Problem The bias of a coin is the probability it falls on head. You know that 2/3 of the coins have bias 0. 9. How many coin tosses do you need to find a coin with bias 0. 9 with probability 1 -exp(- (N))?

Simple Case: The bias of every coin is either at least 0. 9 or at most 0. 7 • Pick a random coin, and toss it k=2 i times for i=100, …, log. N. For each k, if less than 0. 8 fraction are heads, restart with a new coin. Analysis: Probability of restarting at phase k is exp(- (k)), and means we wasted O(k) tosses. So, have 1 -exp(- (N)) certainty after O(N) tosses.

General Case • Pick a random coin, and toss it k=2 i times for i=loglog. N, …, log. N. For each k, if less than 0. 9 -i/10 log. N fraction are heads, restart with a new coin. Analysis: Probability of restarting at phase k is exp(- (k)/log 2 N), and means we wasted O(k) tosses. So, have 1 -exp(-N) certainty after O(Nlog 2 N) tosses.

Amplification From Biased Coins Testing Algorithm Gives Amplification in Nearly Linear Time Biased Coins coin bias of coin most coins are biased Randomized Algorithms random choice of algorithm quality of random choice most random choices are good find a biased coin with prob 1 -exp(-N) coin toss find a good random choice with prob 1 -exp(-N) test random choice

Finding Approximate Clique With Error Probability 2 - (n) • Sample S V, |S|=100/ 2. • k 1000 log 2 n. • While k < n do • For each U S connected to at least n vertices, • Sample V’ V, |V’|=k. • Find k’ vertices in V’ connected to U with largest fraction of neighbors connected to U. • If for all U, average fraction < 0. 9 -i/10 logn, restart. • Double k. • Search for an approximate clique among the sets defined by U S ( n vertices connected to U with largest fraction of neighbors among vertices connected to U).

Oblivious Verifier For Amplified Algorithm • Main challenge: algorithm branches depending on input (if… then. . ). Verifier can’t follow. • Solution: Verifier tries all possible branches. – Oblivious verifier is inefficient! – Maintains set of inputs consistent with each branch. Branch is feasible if set is non-empty. – Final verifier checks: do all branches succeed? Use low error probability per branch.

Summary: Derandomization From Sketching • Requires: – Randomized algorithm with low error (biased coin protocol can help with that). – Sketch of input and oblivious verifier that given sketch & randomness decides if algorithm succeeds. • Yields non-uniform deterministic algorithms; can sometimes convert to uniform. Applications in paper: approximate clique, max-cut on dense graphs, free games (constraint satisfaction problems on dense graphs), Reed. Muller list decoding to unique decoding. More? ?