Extracting Randomness From Few Independent Sources Boaz Barak

Extracting Randomness From Few Independent Sources Boaz Barak, IAS Russell Impagliazzo, UCSD Avi Wigderson, IAS

Plan: 1. Discuss problem and model 2. State our result 3. Introduce main tool – Thm by [BKT, K] 4. * Prove our main theorem.

Randomness Extraction Randomness is central to CS (c. f. , randomized algorithms, cryptography, distributed computing) How do you execute randomized algorithms and protocols? Solution: sample some “random” physical data (coin tossing, thermal noise, hard disk movement, …) Problem: data from physical sources is not a sequence of ideal coin tosses.

Randomness Extractors “Definition”: E: {0, 1}n {0, 1}0. 1 k is an extractor if 8 r. v. X with entropy ¸k , E(X) is close to U 0. 1 k Idea: X high entropy data E extractor uniform output randomized algorithm / protocol

Randomness Extractors “Definition”: E: {0, 1}n {0, 1}0. 1 k is an extractor if 8 r. v. X with entropy ¸k , E(X) is close to U 0. 1 k Problem: No extractor exists. Thm: 8 E: {0, 1}n {0, 1}0. 1 k there’s a r. v. X w/ entropy ¸n-1 s. t. first bit of E(X) is constant. Proof Sketch: Assume wlog |{ x | E 1(x)=0 }| ¸ 2 n/2 let X be the uniform dist over this set.

Solution 1: Seeded Extractors d {0, 1} 0. 1 k is 0. 1 k n {0, 1} Def: E: {0, 1}n£{0, 1} a (seeded) extractor “Definition”: E: {0, 1} is an extractor ifif 88 r. v. min-entropy r. v. X w/ X with entropy¸k¸k , E(X) is close to U 0. 1 k | E(X, Ud) – U 0. 1 k |1 < 1/100. Many exciting applications and connections X has results, min-entropy ¸k (denoted H(X)¸k) if -k. [Z, NZ, Ta, Tr, RSW, STV, TSZ, SU 8 x Pr[ X=x ] · 2, …]. Thm [LRVW]: For every n, k there’s a seeded extractor Every such dist is convex comb of “flat” dist – with d=O(log n) dist on set of size ¸ 2 k. uniform Corollary: Any probabilistic algorithm can be simulated In this talk: entropy = min-entropy w/ weak random source + polynomial overhead.

Solution 1: Seeded Extractors Thm [LRVW]: For every n, k there’s a seeded extractor with d=O(log n) Corollary: Any probabilistic algorithm can be simulated w/ weak random source + polynomial overhead. Question: What about other uses of randomness? For example, can we use this for cryptography? Answer: No! For example, if we concatenate encryptions according to all possible seeds this won’t be secure! Need to use seedless extractors!

Seedless Extractors Idea: Bypass impossibility result by making additional assumption on the high entropy input. Long history and many results [v. N, P, B, SV, CW, TV, KZ, . . ] In this work: We assume that input comes from few independent distributions ([CG]). Def: E: {0, 1}nc {0, 1}0. 1 k is a c-sample extractor if 8 ind. r. v. X 1, …, Xc w/ min-entropy ¸k | E(X 1, …, Xc) – U 0. 1 k |1 < 1/100 2 - (k) Motivation: mathematically clean and plausible model.

Def: E: {0, 1}nc {0, 1}0. 1 k is a c-sample extractor if 8 ind. r. v. X 1, …, Xc w/ min-entropy ¸k | E(X 1, …, Xc) – U 0. 1 k |1 < 2 - (k) Optimal (non-explicit) construction: c=2 , every k¸ (log n) Previous best explicit construction [SV, V, CG, ER, DEOR]: c=2 , every k¸(1+ )n/2 Obtained by variants of following 1 -bit output extractor: E(x, y) = <x, y> Problematic, since natural entropy sources often have entropy less than n/2.

Def: E: {0, 1}nc {0, 1}0. 1 k is a c-sample extractor if 8 ind. r. v. X 1, …, Xc w/ min-entropy ¸k | E(X 1, …, Xc) – U 0. 1 k |1 < 2 - (k) Optimal (non-explicit) construction: c=2 , every k¸ (log n) Previous best explicit construction [SV, V, CG, ER, DEOR]: c=2 , every k¸(1+ )n/2 Our Result: For every >0 c=poly(1/ ) , k= n Main Thm: 8 >0 9 c=poly(1/ ) and poly-time E: {0, 1}nc {0, 1}n s. t. if 8 ind. r. v. X 1, …, Xc w/ min-entropy ¸ n | E(X 1, …, Xc) – Un |1 < 2 - (n)

Main Thm: 8 >0 9 c=poly(1/ ) and poly-time E: {0, 1}nc {0, 1}n s. t. if 8 ind. r. v. X 1, …, Xc w/ min-entropy ¸ n | E(X 1, …, Xc) – Un |1 < 2 - (n) Plan: 1. Discuss problem and model 2. State our result 3. Introduce main tool – Thm by [BKT, K] Show BKT (almost) immediately implies dispersers. 4. Prove our main theorem.

Main Thm: 8 >0 9 c=poly(1/ ) and poly-time E: {0, 1}nc {0, 1}n s. t. if 8 ind. r. v. X 1, …, Xc w/ min-entropy ¸ n | E(X 1, …, Xc) – Un |1 < 2 - (n) Our main tool is the following result: Thm 1 [BKT, K]: 9 absolute constant >0 s. t. for prime field F, and set AµF, max{ |A+A| , |A ¢ A| } ¸ min{ |A|1+ , |F| } 1. Finite field analog of a theorem by [ES]. A+A = { a+b | a, b 2 A } 2. Note false if F} had non-trivial subfields. A ¢ AThm = 1{ would a¢b |bea, b 2 A 3. Note if A is arithmetic (resp. geometric) sequence, then |A+A| (resp. |A¢ A|) is small.

Thm 1 [BKT, K]: 9 absolute constant >0 s. t. for prime field F, and set AµF, max{ |A+A| , |A ¢ A| } ¸ |A|1+ How is this related to extractors? Disperser Lemma [BKT]: Let >0 and F a prime field, then 9 c=poly(1/ ) and poly-time E: Fc F s. t. if X 1, …, XcµF satisfy |Xi|¸ |F| , then E(X 1, …, Xc) = F Corollary: Identify {0, 1}n w/ prime field F of size 2 n. Then, we get poly-time E s. t. if r. v. ’s X 1, …, Xc have entropy ¸ n, then Supp{E(X 1, …, Xc)}={0, 1}n This is called a disperser.

Thm 1 [BKT, K]: 9 absolute constant >0 s. t. for prime field F, and sets set AA, B, C µF, (with |A|=|B|=|C|) max{ |A¢B+C| |A+A| ¸ |A| , |A 1+¢ A| } ¸ |A|1+ How is this related to extractors? Disperser Lemma [BKT]: Let >0 and F a prime field, then 9 c=poly(1/ ) and poly-time E: Fc F s. t. if X 1, …, XcµF satisfy |Xi|¸ |F| , then E(X 1, …, Xc) = F Proof: Use lemma of Rusza to get “asymmetric” version of Thm 1. Lemma [R, N]: If A, B µG w/ |A|=|B|=M, and |AB| B| · M 1+ , then |AA| A| · M 1+O( ) We let E be recursive application of a, b, c a¢b+c with depth |A ¢ A| large ) |A ¢ B| large ) |A¢B+C| large O(log(1/ )). ) |A+C| large ) |A¢B+C| large |A+A| large

Thm 1 [BKT, K]: 9 absolute constant >0 s. t. for prime field F, and sets A, B, CµF, (with |A|=|B|=|C|) |A¢B+C| ¸ |A|1+ ¢+ a 1 , a 2, … . . . ¢ + ¢+ ¢ +. . . ¢+ . . . ¢ ¢+ + ¢+ ¢+ apoly(1/delta)

Plan: 1. Discuss problem and model 2. State our result 3. Introduce main tool – Thm by [BKT, K] Show BKT (almost) immediately implies dispersers. 4. Prove our main theorem.

Distributional Version of [BKT] Thm 1 [BKT, K]: 9 absolute constant >0 s. t. for prime field F, and sets A, B, CµF, (with |A|=|B|=|C|) |A¢B+C| ¸ |A|1+ Our Main Lemma: 9 absolute constant >0 s. t. for prime field F, and distributions A, B, CµF, (with H(A)=H(B)=H(C)), the distribution A¢B+C is 2 - H(A) close to having entropy ¸ (1+ )H(A) ( The distribution A¢B+C assigns to x the prob that a¢b+c=x with a 2 RA , b 2 RB , c 2 RC ) Main Lemma ) Main Theorem.

Our Main Lemma: 9 absolute constant >0 s. t. for prime field F, and distributions A, B, CµF, (with H(A)=H(B)=H(C)), the distribution A¢B+C is 2 - H(A) close to having entropy ¸ (1+ )H(A) Main Lemma ) Main Theorem. ¢+ a 1 , a 2, … . . . ¢ + ¢+ ¢ +. . . ¢+ . . . ¢ ¢+ + ¢+ ¢+ apoly(1/delta)

Our Main Lemma: 9 absolute constant >0 s. t. for prime field F, and distributions A, B, CµF, (with H(A)=H(B)=H(C)), the distribution A¢B+C is 2 - H(A) close to having entropy ¸ (1+ )H(A) Plan: Prove Main Lemma by reducing to [BKT]. We use “magic lemmas” of Gowers & Rusza in the reduction.

Our Main Lemma: 9 absolute constant >0 s. t. for prime field F, and distributions A, B, CµF, (with H(A)=H(B)=H(C)), the distribution A¢B+C is 2 - H(A) close to having entropy ¸ (1+ )H(A) Detailed Plan: 1. Introduce collision probability – a different entropy measure. 2. Rephrase Main Lemma in terms of C. P. 3. Show naïve approach to proving, and show counterexample 4. Use Gowers’ & Rusza’s lemmas to show counterexample essentially captures all cases

Collision Probability cp(X) = Prx, x’ X[ x= x’ ] = x px 2 Fact 1: If H(X)¸k then cp(X)· 2 -k Fact 2: If cp(X)· 2 -k(1+ ) then is 2 - k/2 close to having min-entropy at least k(1+ /2). Notation: If D is r. v. , then the 2 -entropy of D is H 2(D) = log(1/cp(D)) Fact 1 + Fact 2 ) H 2(D) ~ H(D) Fact 3: If X is convex combination of X 1, …, Xm then cp(X) · max { cp(X 1), … , cp(Xm) }

Main Lemma: 9 >0 s. t. for prime field F, dists A, B, CµF, (with H(A)=H(B)=H(C), the distribution A¢B+C is 2 - H(A) close to entropy ¸ (1+ )H(A) Main Lemma (CP version): 9 >0 s. t. for prime field F, and sets A, B, CµF (with |A|=|B|=|C| ), the distribution A¢B+C is |A|- close to having 2 -entropy ¸ (1+ )log |A| Thus, it is sufficient to prove CP version.

Main Lemma (CP version): 9 >0 s. t. for prime field F, and sets A, B, CµF (with |A|=|B|=|C| ), the distribution A¢B+C is |A|- close to having 2 -entropy ¸ (1+ )log |A| Detailed Plan: 1. Introduce collision probability – a different entropy measure. 2. Rephrase Main Lemma in terms of C. P. 3. Show naïve approach to proving, and show counterexample 4. Use Gower’s and Rusza’s lemmas to show counterexample essentially captures all cases

Naïve Approach Prove direct analog to BKT “Conjecture”: 9 >0 s. t. for prime F, and set AµF max { H 2(A+A) , H 2(A¢ A) } ¸ (1+ )log|A| Counter Example: A=AG [AA AG - geometric seq. AA - (disjoint) arithmetic seq. cp(A+A), cp(A¢A)¸ 1/10|A| hence H 2(A+A), H 2(A¢A)·log|A|+O(1) However, in this case H 2(A¢ A+A) ¸ (1+ )log |A|

Naïve Approach Counter Example: A=AG [ AA AG - geometric seq. AA - (disjoint) arithmetic seq. Claim: H 2(A¢A + A) ¸ (1+ )log |A| Sketch: A¢A+A is convex comb of AA ¢A+A and AG¢A+A. cp(AA ¢A+A)· cp(AA¢A) which is low since A¢ is an arithmetic seq AG¢A+A is convex comb of AGa+A but cp(AGa+A) is low since AGa is a geometric seq

Main Lemma: 9 absolute constant >0 s. t. for prime field F, and sets A, B, CµF (with |A|=|B|=|C| ), the distribution A¢B+C is |A|- close to having c. p. · |A|-(1+ ) Detailed Plan: 1. Introduce collision probability – a different entropy measure. 2. Rephrase Main Lemma in terms of C. P. 3. Show naïve approach to proving, and show counterexample 4. Use Gowers’ and Rusza’s lemmas to show counterexample essentially captures all cases

Proof of Main Lemma (CP version): 9 absolute constant >0 s. t. for prime field F, and sets A, B, CµF (with |A|=|B|=|C| ), the distribution A¢B+C is |A|- close to having 2 -entropy ¸ (1+ )log |A| (Loose) Notations: Let M=|A|=|B|=|C| and fix some >0 (e. g. , BKT’s divided by 100) A number ¸ M 1+ is called “large” A number ¸ M 1 - ( ) is called “not-too-small” A distribution D has “high 2 -entropy” if H 2(D) ¸ (1+ )log M Our Goal: Prove that A¢B+C is close to having high 2 -entropy. (i. e. , it is close to having c. p. · 1/M 1+ )

Tools: Thm 1 [BKT, K]: If AµF is not too small then either |A¢A| or |A+A| is large. Lemma [R, N]: If |AA| A| is large then |AB| B| is large. Magic Lemma [G, BS]: Either H 2(AB) B) is large or 9 not-too-small subsets A’µA, B’µB s. t. |A’B’| is not large.

A First Distributional Analog: Cor [BKT+R]: If 9 not-too-small B s. t. |A¢B| is not large then |A+C| is large 8 not-too-small C. Proof: |A¢B| is not large ) |A¢A| is not large [R] ) |A+A| is large [BKT] ) |A+C| is large [R]. Natural Analog: If 9 not-too-small B s. t. H 2(A¢B) is not large then H 2(A+C) is large 8 not-too-small C. This is false: e. g. , A=B=C=AG [AA However, the following is true: PF Lemma: If 9 not-too-small B s. t. |A¢B| is not large then H 2(A+C) is large 8 not-too-small C.

PF Lemma: If 9 not-too-small B s. t. |A¢B| is not large then H 2(A+C) is large 8 not-too-small C. Proof: If H 2(A+C) is not large then by Gowers’s Lemma 9 not-too-small A’µA, C’µC s. t. |A’+C’| is not large. By Rusza’s lemma |A’+A’| is not large ) by BKT |A’¢A’| is large. Since A’µA , |A¢A| is also large ) by Rusza’s lemma |A¢B| is large – contradiction! Def: A not-too-small set AµF is “plus friendly” if H 2(A+C) is large 8 not-too-small set C. 1. A plus-friendly, b 2 F ) Ab plus-friendly. 2. A’ , A’’ plus-friendly, disjoint ) A’[A’’ plus-friendly.

Our Goal: Prove A¢B+C close to having “low c. p. ”. Assume H 2(A¢B+C) not large. We’ll show A=A+[A¢ s. t. A+, A¢are disjoint and 1) A+ is “plus friendly” (or A+ is empty) 2) H 2(A¢ ¢B) is large (or |A¢|· M 1 - ) 1+2 ) contradiction since A¢B+C is M- close to convex comb of A+¢B+C and A¢¢B+C, but a) H 2(A+¢B+C) is large since convex comb of A+b+C and A+b is plus-friendly. b) H 2(A¢¢B+C) is large since convex comb of A¢B+c which are permutations of A¢B.

Our Goal: Prove A¢B+C close to having “low c. p. ”. Assume H 2(A¢B+C) not large. We’ll show A=A+[A¢ s. t. A+, A¢ disjoint and 1) A+ is “plus friendly” (or A+ is empty) 2) H 2(A¢ ¢B) is large (or |A¢|· M 1 - ) We build partition iteratively. Initially A+=; , A¢=A. Assume A¢ is not-too-small (o/w we’re done). Assume H 2(A¢¢B) is not large (o/w we’re done). By Gowers’ lemma, 9 not-too-small subsets A’µA¢, B’µB s. t. |A’¢B’| not large. By PF Lemma A’ is plus-friendly, remove A’ from A¢ and add it to A+.

This finishes the proof of the Main Lemma and hence the Main Theorem. Main Lemma: 9 absolute constant >0 s. t. for prime field F, and distributions A, B, CµF, (with H(A)=H(B)=H(C)<0. 8 log|F|), the distribution A¢B+C is 2 - H(A) close to having entropy ¸ (1+ )H(A) Main Thm: 8 >0 9 c=poly(1/ ) and poly-time E: {0, 1}nc {0, 1}n s. t. if 8 ind. r. v. X 1, …, Xc w/ min-entropy ¸ n | E(X 1, …, Xc) – Un |1 < 2 - (n) 2 -10 n

Another Result: A disperser for the case that all samples come from same distribution, which only requires (log n) entropy (using [EH]).

Open Problems n n n Extractors/Dispersers with lower entropy requirement (k=n (1) or even k= (log n) ) Improvement for the case of two samples (related to constructing Ramsey graphs). More applications of results/techniques.