Support Vector Machine II ECE5424 G CS5824 JiaBin

Support Vector Machine II ECE-5424 G / CS-5824 Jia-Bin Huang Virginia Tech Spring 2019

Administrative • Please use piazza. No emails. • HW 2 released.

Support Vector Machine • Cost function • Large margin classification • Kernels • Using an SVM

Support vector machine • Slide credit: Andrew Ng

SVM decision boundary • Slide credit: Andrew Ng

SVM decision boundary: Linearly separable case Slide credit: Andrew Ng

SVM decision boundary: Linearly separable case margin Slide credit: Andrew Ng

Why large margin classifiers? • margin

Vector inner product • Slide credit: Andrew Ng

SVM decision boundary • Slide credit: Andrew Ng

SVM decision boundary • Slide credit: Andrew Ng

Rewrite the formulation margin •

Data not linearly separable? • NP-hard

Convex relaxation NP-hard

Hinge loss Image credit: https: //math. stackexchange. com/questions/782586/how-do-you-minimize-hinge-loss

Support Vector Machine • Cost function • Large margin classification • Kernels • Using an SVM

Non-linear classification •

Non-linear classification

Kernel •

Why Kernels matter? •

Example •

Example • Slide credit: Maria-Florina Balcan

Example kernels •

Constructing new kernels •

Non-linear decision boundary • Slide credit: Andrew Ng

Kernel • Slide credit: Andrew Ng

• Slide credit: Andrew Ng

Choosing the landmarks • Slide credit: Andrew Ng

SVM with kernels • Slide credit: Andrew Ng

SVM with kernels

Support vector machines (Primal/Dual) • Primal form • Lagrangian dual form

SVM (Lagrangian dual) •

SVM parameters • Slide credit: Andrew Ng

SVM Demo • https: //cs. stanford. edu/people/karpathy/svmjs/demo/

SVM song • https: //www. youtube. com/watch? v=g 15 bqtyid. Zs Video source:

Support Vector Machine • Cost function • Large margin classification • Kernels • Using an SVM

Using SVM • Slide credit: Andrew Ng

Kernel (similarity) functions • Note: not all similarity functions make valid kernels. • Many off-the-shelf kernels available: • Polynomial kernel • String kernel • Chi-square kernel • Histogram intersection kernel Slide credit: Andrew Ng

Multi-classification • Slide credit: Andrew Ng

Logistic regression vs. SVMs • Slide credit: Andrew Ng

Things to remember • margin