COMS E 6998 9 F 15 Lecture 2

COMS E 6998 -9 F 15 Lecture 2: Median trick + Chernoff, Distinct Count, Impossibility Results 1

Administrivia, Plan • Website moved: sublinear. wikischolars. columbia. edu/main • Piazza: sign-up! • Plan: – Median trick, Chernoff bound (from Tue) – Distinct Elements Count – Impossibility Results 2

Last Lecture IP Frequency 160. 39. 142. 2 3 • Morris: Failure prob: 0. 1 use Chernoff 3

“Median trick” • 4

Using Chernoff for Median trick • 5

Problem: Distinct Elements • IP Frequency 1 3 2 2 3 0 4 9 5 0 … 0 1 6

Algorithm for approximating DE • Where from? Will return later… 7

Analysis • 7 5 2 8

Analysis 2 • 9

Alternative: Bottom-k • 10

Analysis for Bottom-k • 11

Hash functions in Streaming • 12

Issue 2: bounded randomness • 13

Pairwise-Independent: example • 14

Impossibility Results • 15

Deterministic Exact Won’t Work • 1 3 5 6 7 8 9 1 0 1 1 1 0 10 1 2 16

Deterministic Exact Won’t Work • 1 3 5 6 7 8 9 10 1 2 17

Deterministic Exact Won’t Work • 18

Deterministic Approx Won’t Too • 1 1 0 1 1 1 0 3 5 6 7 8 9 0 1 0 0 1 1 1 2 10 4 7 8 9 10 11 19

Concluding Remarks • 20