Two Decades of Property Testing Madhu Sudan Harvard

Two Decades of Property Testing Madhu Sudan Harvard April 8, 2016 Two Decades of Property Testing 1 of 29

Kepler’s Big Data Problem n Tycho Brahe (~1550 -1600): n n Wished to measure planetary motion accurately. To confirm sun revolved around earth … (+ other planets around sun) Spent 10% of Danish GNP Johannes Kepler (~1575 -1625 s): n n n Believed Copernicus’s picture: planets in circular orbits. Addendum: Ratio of orbits based on Löwner-John ratios of platonic solids. “Stole” Brahe’s data (1601). Source: Michael Fowler, “Galileo & Einstein”, U. Virginia Worked on it for nine years. Disproved Addendum; Confirmed Copernicus (circle -> ellipse); discovered laws of planetary motion. n Nine April 8, 2016 Years? Two Decades of Property Testing 2 of 29

The challenge of analyzing big data n n Standard method: n Propose concept class. n LEARN (parameters of) best fitting concept in class to data in hand. n TEST to see if this is a good enough fit. Bottleneck n LEARNing is expensive; wasted if TEST rejects. n n Can we TEST before we LEARN? Don’t be Ridiculous! Yes: This is PROPERTY TESTING!! April 8, 2016 Two Decades of Property Testing 3 of 29

Property Testing n April 8, 2016 Two Decades of Property Testing 4 of 29

Example 1: Polling n April 8, 2016 Two Decades of Property Testing 5 of 29

Example 2: Linearity n April 8, 2016 Two Decades of Property Testing 6 of 29

Linearity Analysis n April 8, 2016 Two Decades of Property Testing 7 of 29

A subtle change n April 8, 2016 Two Decades of Property Testing 8 of 29

Key step of BLR analysis n April 8, 2016 Two Decades of Property Testing 9 of 29

History (slightly abbreviated) n n n [Blum, Luby, Rubinfeld – S’ 90] n Linearity + application to program testing [Babai, Fortnow, Lund – F’ 90] n Multilinearity + application to PCPs (MIP). [Rubinfeld+S. ] n Low-degree testing + Definition [Goldreich, Goldwasser, Ron] n Graph property testing + systematic study Since then … many developments n More graph properties, statistical properties, matrix properties, properties of Boolean functions … n More algebraic properties April 8, 2016 Two Decades of Property Testing 10 of 29

n April 8, 2016 Two Decades of Property Testing 11 of 29

Example 4: Long code/Junta testing n April 8, 2016 Two Decades of Property Testing 12 of 29

Example 5: Distribution Testing n April 8, 2016 Two Decades of Property Testing 13 of 29

What is Property Testing? Graphs + Regularity Algebra Matrices + Linear algebra April 8, 2016 Two Decades of Property Testing Statistics + CLT 14 of 29

(Dense) Graph Property Testing n April 8, 2016 Two Decades of Property Testing 15 of 29

Contrast with Low-degree testing n Why no unification? April 8, 2016 Two Decades of Property Testing 16 of 29

Aside: Importance of Low-degree Testing n April 8, 2016 Two Decades of Property Testing 17 of 29

Some (introspective) questions n n What is qualitatively novel about linearity testing relative to classical statistics? Why are the mathematical underpinnings of different themes so different? Why is there no analog of “graph property testing” (broad class of properties, totally classified wrt testability) in algebraic world? n What is the context for low-degree testing? Answer to all: Invariance! April 8, 2016 Two Decades of Property Testing 18 of 29

Invariance? n April 8, 2016 Two Decades of Property Testing 19 of 29

Invariances (contd. ) n April 8, 2016 Two Decades of Property Testing 20 of 29

What is Property Testing? Algebra=? ? April 8, 2016 Two Decades of Property Testing 21 of 29

Abstracting algebraic properties n April 8, 2016 Two Decades of Property Testing 22 of 29

Testing Linear Properties R is a field F; P is linear! Universe: Universe {f: D R} P Must accept Don’t care P Must reject Algebraic Property = Code! (usually) April 8, 2016 Two Decades of Property Testing 23 of 29

Why study affine-invariance? n n n Common abstraction of properties studied in [BLR], [RS], [ALMSS], [AKKLR], [KL], [JPRZ]. n (Variations on low-degree polynomials) Hopes n Unify existing proofs n Classify/characterize testability n Find new testable codes (w. novel parameters) Rest of the talk: Brief summary of findings April 8, 2016 Two Decades of Property Testing 24 of 29

Results 1: AKKLR Conjecture n April 8, 2016 Two Decades of Property Testing 25 of 29

Results 2: Accidental +ve n April 8, 2016 Two Decades of Property Testing 26 of 29

Results 3: Lifting n Bad News + Bad News = Good News! April 8, 2016 Two Decades of Property Testing 27 of 29

Result 3: Lifting (contd. ) n April 8, 2016 Two Decades of Property Testing 28 of 29

Conclusions n April 8, 2016 Two Decades of Property Testing 29 of 29

Thank You April 8, 2016 Two Decades of Property Testing 30 of 29