Linear regression with one variable Model representation Machine

Linear regression with one variable Model representation Machine Learning Andrew Ng

Housing Prices (Portland, OR) Price (in 1000 s of dollars) 500 400 300 200 100 0 0 500 1000 1500 Size 2000 (feet 2) 2500 Supervised Learning Regression Problem Given the “right answer” for each example in the data. Predict real-valued output 3000 Andrew Ng

Training set of housing prices (Portland, OR) Size in feet 2 (x) 2104 1416 1534 852 … Price ($) in 1000's (y) 460 232 315 178 … Notation: m = Number of training examples x’s = “input” variable / features y’s = “output” variable / “target” variable Andrew Ng

How do we represent h ? Training Set Learning Algorithm Size of house h Estimated price Linear regression with one variable. Univariate linear regression. Andrew Ng

Andrew Ng

Linear regression with one variable Cost function Machine Learning Andrew Ng

Training Set Size in feet 2 (x) 2104 1416 1534 852 … Price ($) in 1000's (y) 460 232 315 178 … Hypothesis: ‘s: Parameters How to choose ‘s ? Andrew Ng

3 3 3 2 2 2 1 1 1 0 0 1 2 3 Andrew Ng

y x Idea: Choose so that is close to for our training examples Andrew Ng

Andrew Ng

Linear regression with one variable Cost function intuition I Machine Learning Andrew Ng

Hypothesis: Simplified Parameters: Cost Function: Goal: Andrew Ng

(for fixed y , this is a function of x) (function of the parameter 3 3 2 2 1 1 0 0 1 x 2 3 ) 0 -0. 5 0 0. 5 1 1. 5 2 2. 5 Andrew Ng

Andrew Ng

Linear regression with one variable Cost function intuition II Machine Learning Andrew Ng

Hypothesis: Parameters: Cost Function: Goal: Andrew Ng

(for fixed , this is a function of x) (function of the parameters ) 500 400 Price ($) 300 in 1000’s 200 100 0 0 1000 Size in 2000 feet 2 (x) 3000 Andrew Ng

Andrew Ng

(for fixed , this is a function of x) (function of the parameters ) Andrew Ng

Andrew Ng

Linear regression with one variable Machine Learning Gradient descent Andrew Ng

Have some function Want Outline: • Start with some • Keep changing to reduce until we hopefully end up at a minimum Andrew Ng

J( 0, 1) 0 1 Andrew Ng

Gradient descent algorithm Correct: Simultaneous update Incorrect: Andrew Ng

Andrew Ng

Linear regression with one variable Gradient descent intuition Machine Learning Andrew Ng

Gradient descent algorithm Andrew Ng

Andrew Ng

If α is too small, gradient descent can be slow. If α is too large, gradient descent can overshoot the minimum. It may fail to converge, or even diverge. Andrew Ng

at local optima Current value of Andrew Ng

Gradient descent can converge to a local minimum, even with the learning rate α fixed. As we approach a local minimum, gradient descent will automatically take smaller steps. So, no need to decrease α over time. Andrew Ng