Linear Classifiers Based on slides by William Cohen

Linear Classifiers Based on slides by William Cohen, Andrej Karpathy, Piyush Rai

Linear Classifiers • Let’s simplify life by assuming: – Every instance is a vector of real numbers, x=(x 1, …, xn). (Notation: boldface x is a vector. ) – First we consider only two classes, y=(+1) and y=(-1) • A linear classifier is vector w of the same dimension as x that is used to make this prediction:

w X 1 . w Visually, x · w is the distance you get if you “project x onto w” x 2 X 2 . w X 1 -W In 3 d: line plane In 4 d: plane hyperplane … The line perpendicular to w divides the vectors classified as positive from the vectors classified as negative.

w -W Wolfram Math. World Mediaboost. com

w Notice that the separating hyperplane goes through the origin…if we don’t want this we can preprocess our examples: -W or where b=w 0 is called bias

Back to Image Classification

3072 numbers in total reshaped into a column vector x

Interactive Web Demo: http: //vision. stanford. edu/teaching/cs 231 n/linear-classify-demo/

Perceptron learning A instance xi B ^ yi ^ Compute: yi = sign(wk. xi ) If mistake: wk+1 = wk + yi xi yi • 1957: The perceptron algorithm by Frank Rosenblatt • 1960: Perceptron Mark 1 Computer – hardware implementation • 1969: Minksky & Papert book shows perceptrons limited to linearly separable data • 1970’s: learning methods for two-layer neural networks

Question