ECE 5424 Introduction to Machine Learning Topics Regression
- Slides: 28
ECE 5424: Introduction to Machine Learning Topics: – Regression Readings: Barber 17. 1, 17. 2 Stefan Lee Virginia Tech
Administrativia • HW 1 – 39 Submissions – 58 Students Enrolled – 19 MIA (? ? ? ) • Project Proposal – Due: 09/21, 11: 55 pm – <= 2 pages, NIPS format (LESS THAN A WEEK!) • HW 2 – – (C) Dhruv Batra Out today Due on Wednesday 09/28, 11: 55 pm Please please start early Implement Linear Regression, Naïve Bayes, Logistic Regression 2
Recap of last time (C) Dhruv Batra 3
Learning a Gaussian • Collect a bunch of data – Hopefully, i. i. d. samples – e. g. , exam scores • Learn parameters – Mean – Variance (C) Dhruv Batra 4
MLE for Gaussian • Prob. of i. i. d. samples D={x 1, …, x. N}: • Log-likelihood of data: (C) Dhruv Batra Slide Credit: Carlos Guestrin 5
Your second learning algorithm: MLE for mean of a Gaussian • What’s MLE for mean? (C) Dhruv Batra Slide Credit: Carlos Guestrin 6
Learning Gaussian parameters • MLE: (C) Dhruv Batra 7
Bayesian learning of Gaussian parameters • Conjugate priors – Mean: Gaussian prior – Variance: Inverse Gamma or Wishart Distribution • Prior for mean: (C) Dhruv Batra Slide Credit: Carlos Guestrin 8
MAP for mean of Gaussian (C) Dhruv Batra Slide Credit: Carlos Guestrin 9
New Topic: Regression (C) Dhruv Batra 10
1 -NN for Regression • Often bumpy (overfits) (C) Dhruv Batra Figure Credit: Andrew Moore 11
(C) Dhruv Batra Slide Credit: Greg Shakhnarovich 12
(C) Dhruv Batra Slide Credit: Greg Shakhnarovich 13
(C) Dhruv Batra Slide Credit: Greg Shakhnarovich 14
Linear Regression • Demo – http: //hspm. sph. sc. edu/courses/J 716/demos/Least. Squares/L east. Squares. Demo. html (C) Dhruv Batra 15
(C) Dhruv Batra Slide Credit: Greg Shakhnarovich 16
Plan for Today • Regression – Linear Regression Recap • Some matrix calculus review – Connections with Gaussians • The outlier problem – Robust Least Squares (C) Dhruv Batra 17
(C) Dhruv Batra Slide Credit: Greg Shakhnarovich 18
(C) Dhruv Batra Slide Credit: Greg Shakhnarovich 19
(C) Dhruv Batra Slide Credit: Greg Shakhnarovich 20
(C) Dhruv Batra Slide Credit: Greg Shakhnarovich 21
(C) Dhruv Batra Slide Credit: Greg Shakhnarovich 22
(C) Dhruv Batra Slide Credit: Greg Shakhnarovich 23
But, why? • Why sum squared error? ? ? • Gaussians, Watson, Gaussians… (C) Dhruv Batra 24
(C) Dhruv Batra Slide Credit: Greg Shakhnarovich 25
MLE Under Gaussian Model • On board (C) Dhruv Batra 26
Is OLS Robust? • Demo – http: //www. calpoly. edu/~srein/Stat. Demo/All. html • Bad things happen when the data does not come from your model! • How do we fix this? (C) Dhruv Batra 27
Robust Linear Regression • y ~ Lap(w’x, b) • On board (C) Dhruv Batra 28
Cs 282
Cost funtion
Linear regression with multiple features
K nearest neighbors python
Linear regression with multiple variables machine learning
Simple linear regression and multiple regression
Multiple linear regression
Survival analysis vs logistic regression
Logistic regression vs linear regression
Concept learning task in machine learning
Analytical learning in machine learning
Pac learning model in machine learning
Machine learning t mitchell
Inductive and analytical learning in machine learning
Inductive and analytical learning problem
Instance based learning in machine learning
Inductive learning machine learning
First order rule learning in machine learning
Eager vs lazy
Cmu machine learning
Introduction to machine learning ethem alpaydin
Introduction to machine learning andrew ng
Introduction to machine learning andrew ng
Mike mozer
Introduction to machine learning ethem alpaydin
Andrew ng machine learning slides
Introduction to machine learning slides
Introduction to machine learning and data mining
Lstm lecture
