Weak Learning DNF under uniform distribution A parity
Weak Learning DNF under uniform distribution • A parity function weakly approximates f – Find this function • KM algorithm – Form a tree, pruning a node if there are no large coefficients starting with that substring
Strong learning: Boosting • Recall Freund’s algo – Construct weak hypotheses h 1, h 2, h 3… – At step i: Distribution Di: more weight to x on which hi, …, hi-1 were wrong Form hypo hi on Di – Combine hypotheses using some rule • Does parity approximate f on Di? – Yes. . Answer on next slide • Needs a distribution-independent weak learner – We don’t have one for DNF
Strong learning DNF • Let f be a DNF having s-terms • Lemma: f has a fourier coefficient of value at least 1/(2 s+1)
Can we tweak KM? • Cool fact: KM works for real-valued functions as well • Idea: Construct function g such that • Depends on L (g) in running time
Converting (f, D) to (g, U) • Notice that • We can learn on dist D is same as on uniform on U!! • Need MQ oracle for g => MQ oracle for D • L (g) should not be too large “D close to uniform”
An appropriate Boosting algorithm • Final hypothesis – majority of his • Di = D(x) i(x) has to be normalized i(x) prob that hypotheses are almost equally divided over x • Stop if Dis become too small • Notice: easily computable • Close to uniform – within a small factor
- Slides: 6