Online Algorithms Lecturer Yishay Mansour Elad Walach Alex

Online Algorithms Lecturer: Yishay Mansour Elad Walach Alex Roitenberg

Introduction Up until now, our algorithms start with input and work with it suppose input arrives a little at a time, need instant response

Oranges example

Introduction

Linear Separators

Linear saperator

Perceptron

The perceptron algorithm

The perceptron algorithm

Mistake Bound Theorem

Mistake Bound Proof

Proof Cont.

Proof Cont. From Claim 1: From Claim 2: Also: Since Combining:

The world is not perfect What if there is no perfect separator?

The world is not perfect

The world is not perfect

Perceptron for maximizing margins

Perceptron Algorithm (maximizing margin)

Mistake Bound Theorem similar to the perceptron proof. Claim 1 remains the same: We only have to bound

Mistake bound proof

Proof Cont.

Proof Cont.

The mistake bound model

Con Algorithm

CON Algorithm

The bounds of CON

HAL – halving algorithm

HAL –halving algorithm

Mistake Bound model and PAC Generates strong online algorithms In the past we have seen PAC Restrictions for mistake bound are much harsher than PAC If we know that A learns C in mistake bound model , should A learn C in PAC model?

Mistake Bound model and PAC

Conservative equivalent of Mistake Bound Algorithm

Building Apac inconsistent …

Building Apac

Disjunction of Conjuctions

Disjunction of Conjunctions We have proven that every algorithm in mistake bound model can be converted to PAC Lets look at some algorithms in the mistake bound model

Disjunction Learning

Example

Mistake Bound Analysis

Mistake Bound Analysis

Mistake analysis proof At first mistake we eliminate n literals At any further mistake we eliminate at least 1 literal L 0 has 2 n literals So we can have at most n+1 mistakes

k-DNF

k-DNF classification

Winnow

Winnow

Mistake bound analysis

Winnow Proof: Definitions

Winnow Proof: Positive Mistakes

Winnow Proof: Positive Mistakes

Winnow Proof: Negative Mistakes

Winnow Proof: Cont.

What should we know? I

What should you know? II

Questions?