Invariance Principles in Theoretical Computer Science Ryan ODonnell
- Slides: 41
Invariance Principles in Theoretical Computer Science Ryan O’Donnell Carnegie Mellon University
Talk Outline 1. Describe some TCS results requiring variants of the Central Limit Theorem. 2. Show a flexible proof of the CLT with error bounds. 3. Open problems and an advertisement.
Talk Outline 1. Describe some TCS results requiring variants of the Central Limit Theorem. 2. Show a flexible proof of the CLT with error bounds. 3. Open problems and an advertisement.
Linear Threshold Functions
Linear Threshold Functions
Learning Theory Servedio’ 08] Thm : [O- Can learn LTFs f in poly(n) time, just from correlations Key: when all |c i | ≤ ϵ. E[f( x)xi ].
Property Testing Servedio’ 09] Thm : [Matulef-O-Rubinfeld- Can test if ϵ-close to an LTF with poly(1/ Key: when all |c i | ≤ ϵ. ϵ) querie
Derandomization Zuckerman’ 10] Thm : [Meka- PRG for LTFs with seed length O(log(n) log(1/ ϵ)). Key: even when xi ’s not fully independent.
Multidimensional CLT? For when all small compared to
Derandomization+ Zuckerman’ 10] Thm : [Gopalan-O-Wu- PRG for “functions of O(1) LTFs” with seed length O(log(n) log(1/ Key: ϵ)). Derandomized multidimensional CLT.
Property Testing+ [Blais -O’ 10] Thm : Testing if Majority of k bits needs k is a Ω(1) queries. Key: assuming E[Xi ] = E[Yi ], Var[ Xi ] = Var[ Yi ], and some other conditions. (actually, a multidimensional version)
Social Choice, Inapproximability Oleszkiewicz’ 05] Thm : [Mossel-O- a) Among voting schemes where no voter has unduly large influence, Majority is most robust to noise. b) Max-Cut is UG-hard to. 878 -approx. Key: If P is a low-deg. multilin. polynomial, assuming P has “small coeffs. on each coord. ”
Talk Outline 1. Describe some TCS results requiring variants of the Central Limit Theorem. 2. Show a flexible proof of the CLT with error bounds. 3. Open problems and an advertisement.
Gaussians Standard Gaussian: G ~ N(0, 1). Mean 0, Var Anti-concentration: Pr[ G ∈ [u− ϵ, u+ ϵ] ] ≤ O a + b G also a “Gaussian”: N(a, b 2) Sum of independent Gaussians is Gaussian: If G ~ N(a, b 2 ), H ~ N(c, d 2 ) are independen then G + H ~ N(a+c, b 2 +d 2 ).
Central Limit Theorem (CLT) X 1 , X 2 , X 3 , … independent, ident. distrib. , mean 0, variance σ 2,
CLT with error bounds X 1 , X 2 , …, Xn independent, ident. distrib. , mean 0, variance 1/n, X 1 + · · · + Xn is “close to” N(0, 1), assuming Xi is not too wacky:
Niceness of random variables Say E[X] = 0, stddev[ X] = σ. def : ( ≥ σ). “ def ”: X is “ nice ” if eg : not nice : ± 1. N(0, 1). Unif on [-a, a].
Niceness of random variables Say E[X] = 0, stddev[ X] = σ. def : ( def : eg : not nice : ≥ σ). X is “ C-nice ” if ± 1. N(0, 1). Unif on [-a, a].
Berry-Esseen Theorem X 1 , X 2 , …, Xn independent, ident. distrib. , mean 0, variance 1/n, X 1 + · · · + Xn is ϵ-close to N(0, 1), assuming Xi is C-nice , where [Shevtsova’ 07]: Y “ ϵ-close” to Z: . 7056
General Case X 1 , X 2 , …, Xn independent, ident. distrib. , mean 0, X 1 + · · · + Xn is ϵ-close to N(0, 1), assuming Xi is C-nice ,
Berry-Esseen: How to prove? X 1 , X 2 , …, Xn indep. , mean 0, S = X 1 + · · · + Xn ϵ-close to G ~ N(0, 1). 1. “Characteristic functions” 2. “Stein’s method” 3. “Replacement” = think like a cryptographer
Indistinguishability of random variables S “ ϵ-close” to G:
Indistinguishability of random variables S “ ϵ-close” to G: u
Indistinguishability of random variables S “ ϵ-close” to G: t u
Indistinguishability of random variables S “ ϵ-close” to G:
Replacement method S “ ϵ-close” to G: u δ
Replacement method X 1 , X 2 , …, Xn indep. , mean 0, S = X 1 + · · · + Xn G ~ N(0, 1) For smooth
Replacement method Hybrid argument X 1 , X 2 , …, Xn indep. , mean 0, S = X 1 + · · · + Xn G = G 1 + · · · + Gn For smooth
Invariance principle X 1 , X 2 , …, Xn Y 1 , Y 2 , …, Yn indep. , mean 0, Var[ Xi ] = Var[ Yi ] = SX = X 1 + · · · + X n S Y = Y 1 + · · · + Yn For smooth
Hybrid argument X 1 , X 2 , …, Xn , Y 1 , Y 2 , …, Yn , independent matching means and variances. SX = X 1 + · · · + Xn vs. SY = Y 1 + · · · + Yn Def : Zi = Y 1 + · · · + Yi + Xi+1 + · · · + Xn S X = Z 0 , S Y = Zn
Hybrid argument X 1 , X 2 , …, Xn , Y 1 , Y 2 , …, Yn , independent matching means and variances. Zi = Y 1 + · · · + Yi + Xi+1 + · · · + Xn Goal:
Zi = Y 1 + · · · + Yi + Xi+1 + · · · + Xn where U = Y 1 + · · · + Yi− 1 + Xi+1 + · · · + Xn Note: U, Xi , Yi independent. Goal:
− = ∴ by indep. and matching means/variances!
Variant Berry-Esseen: Say If X 1 , X 2 , …, Xn & Y 1 , Y 2 , …, Yn indep. and have matching means/variances, then
Usual Berry-Esseen: If X 1 , X 2 , …, Xn indep. , mean 0, Hack u δ
Usual Berry-Esseen: If X 1 , X 2 , …, Xn indep. , mean 0, Variant Berry-Esseen + Hack Usual Berry-Esseen except with error O( ϵ 1/4 )
Extensions are easy! Vector-valued version: Use multidimensional Taylor theorem. Derandomized version: If X 1 , …, Xm C-nice, 3 -wise indep. , then X 1 +···+ Xm is O(C)-nice. Higher-degree version: X 1 , …, Xm C-nice, indep. , Q is a deg. -d poly. , then Q( X 1 , …, Xm ) is O(C) d -nice.
Talk Outline 1. Describe some TCS results requiring variants of the Central Limit Theorem. 2. Show a flexible proof of the CLT with error bounds. 3. Open problems, advertisement, anecdote?
Open problems 1. Recover usual Berry-Esseen via the Replacement method. 2. Vector-valued: Get correct dependence on test sets K. ( Gaussian surface area 3. Higher-degree: improve (? ) the exponential dependence 4. Find more applications on degree d. in TCS. ? )
Do you like LTFs and PTFs ? Do you like probability and geometry ? Oct. 21 -22 (“ just before FOCS ”) workshop at the Princeton Intractability Center: Analysis and Geometry of Boolean Threshold Functions Diakonikolas! Kane! Meka! Rubinfeld! Servedio! Shpilka! Vempala! And more! http: //intractability. princeton. edu/blog/2010/08/workshop-ltfptf/
- Tiffany odonnell
- Great theoretical ideas in computer science
- Great theoretical ideas in computer science
- Great theoretical ideas in computer science
- Computer science graph theory
- Great theoretical ideas in computer science
- Great theoretical ideas in computer science
- Great theoretical ideas in computer science
- Steven rudich
- Vocabularize
- Great theoretical ideas in computer science
- Great theoretical ideas in computer science
- Invariance property
- Phase invariance
- Semtools measurement invariance
- Bilinear transformation example
- Equivariance
- What is your favourite lesson
- Sequencing ap csp
- What are the 3 fundamental principles of fingerprints
- Applied principles of horticultural science
- Principles of science
- Principles of animal science
- Social science vs natural science
- What are the branches of natural science
- Natural and physical science
- Applied science vs pure science
- Natural science and social science similarities
- Wwwk-6.thinkcentral
- Rule of 70 in population growth
- Windcube
- Soft science definition
- Computer security principles and practice
- Computer security principles and practice solutions
- Build a modern computer from first principles
- Principles of network applications in computer networks
- Principles of human computer interaction
- Congestion control principles
- Computer clusters for scalable parallel computing
- Computer security principles and practice 4th edition
- What is input design in system analysis and design
- Principles of computer security