Linear regression with one variable Cost function Machine
Linear regression with one variable Cost function Machine Learning Andrew Ng
Training Set Size in feet 2 (x) 2104 1416 1534 852 … Price ($) in 1000's (y) 460 232 315 178 … Hypothesis: ‘s: Parameters How to choose ‘s ? Andrew Ng
3 3 3 2 2 2 1 1 1 0 0 1 2 3 Andrew Ng
y x Idea: Choose so that is close to for our training examples Andrew Ng
Linear regression with one variable Cost function intuition I Machine Learning Andrew Ng
Hypothesis: Simplified Parameters: Cost Function: Goal: Andrew Ng
(for fixed , this is a function of x) y (function of the parameter ) 3 3 2 2 1 1 0 0 1 x 2 3 0 -0. 5 0 0. 5 1 1. 5 2 2. 5 Andrew Ng
(for fixed , this is a function of x) y (function of the parameter ) 3 3 2 2 1 1 0 0 1 x 2 3 0 -0. 5 0 0. 5 1 1. 5 2 2. 5 Andrew Ng
(for fixed , this is a function of x) y (function of the parameter ) 3 3 2 2 1 1 0 0 1 x 2 3 0 -0. 5 0 0. 5 1 1. 5 2 2. 5 Andrew Ng
Linear regression with one variable Cost function intuition II Machine Learning Andrew Ng
Hypothesis: Parameters: Cost Function: Goal: Andrew Ng
(for fixed , this is a function of x) (function of the parameters ) 500 400 Price ($) 300 in 1000’s 200 100 0 0 1000 2000 Size in feet 2 (x) 3000 Andrew Ng
Andrew Ng
(for fixed , this is a function of x) (function of the parameters ) Andrew Ng
(for fixed , this is a function of x) (function of the parameters ) Andrew Ng
(for fixed , this is a function of x) (function of the parameters ) Andrew Ng
(for fixed , this is a function of x) (function of the parameters ) Andrew Ng
Linear regression with one variable Machine Learning Gradient descent Andrew Ng
Have some function Want Outline: • Start with some • Keep changing to reduce until we hopefully end up at a minimum Andrew Ng
J( 0, 1) 0 1 Andrew Ng
J( 0, 1) 0 1 Andrew Ng
Gradient descent algorithm Correct: Simultaneous update Incorrect: Andrew Ng
Linear regression with one variable Gradient descent intuition Machine Learning Andrew Ng
Gradient descent algorithm Andrew Ng
Andrew Ng
If α is too small, gradient descent can be slow. If α is too large, gradient descent can overshoot the minimum. It may fail to converge, or even diverge. Andrew Ng
at local optima Current value of Andrew Ng
Gradient descent can converge to a local minimum, even with the learning rate α fixed. As we approach a local minimum, gradient descent will automatically take smaller steps. So, no need to decrease α over time. Andrew Ng
Linear regression with one variable Gradient descent for linear regression Machine Learning Andrew Ng
Gradient descent algorithm Linear Regression Model Andrew Ng
Andrew Ng
Gradient descent algorithm update and simultaneously Andrew Ng
Gradient descent example X y 1 2 3 6 5 10 h error h-y (h-y)x h error Andrew Ng
J( 0, 1) 0 1 Andrew Ng
Andrew Ng
(for fixed , this is a function of x) (function of the parameters ) Andrew Ng
(for fixed , this is a function of x) (function of the parameters ) Andrew Ng
(for fixed , this is a function of x) (function of the parameters ) Andrew Ng
(for fixed , this is a function of x) (function of the parameters ) Andrew Ng
(for fixed , this is a function of x) (function of the parameters ) Andrew Ng
(for fixed , this is a function of x) (function of the parameters ) Andrew Ng
(for fixed , this is a function of x) (function of the parameters ) Andrew Ng
(for fixed , this is a function of x) (function of the parameters ) Andrew Ng
(for fixed , this is a function of x) (function of the parameters ) Andrew Ng
Logistic Regression Classification Machine Learning Andrew Ng
Classification Email: Spam / Not Spam? Online Transactions: Fraudulent (Yes / No)? Tumor: Malignant / Benign ? 0: “Negative Class” (e. g. , benign tumor) 1: “Positive Class” (e. g. , malignant tumor) Andrew Ng
Classification: y = 0 or 1 can be > 1 or < 0 Logistic Regression: Andrew Ng
Logistic Regression Hypothesis Representation Machine Learning Andrew Ng
Logistic Regression Model Want 1 0. 5 Sigmoid function Logistic function 0 Andrew Ng
Logistic regression 1 z Suppose predict “ “ if Andrew Ng
Logistic Regression Cost function Machine Learning Andrew Ng
Training set: m examples How to choose parameters ? Andrew Ng
Cost function Linear regression: “non-convex” “convex” Andrew Ng
Logistic regression cost function If y = 1 0 1 Andrew Ng
Logistic regression cost function If y = 0 0 1 Andrew Ng
Logistic Regression Simplified cost function and gradient descent Machine Learning Andrew Ng
Logistic regression cost function Andrew Ng
Logistic regression cost function To fit parameters : To make a prediction given new : Output Andrew Ng
Gradient Descent Want : Repeat (simultaneously update all ) Andrew Ng
Gradient Descent Want : Repeat (simultaneously update all ) Algorithm looks identical to linear regression! Andrew Ng
Gradient Descent Want : Repeat (simultaneously update all ) Andrew Ng
Gradient Descent Want : Repeat (simultaneously update all ) Algorithm looks identical to linear regression! Andrew Ng
Andrew Ng
Chain rule Andrew Ng
Andrew Ng
Derivation of logistic regression Andrew Ng
Now Derive From Andrew Ng
Andrew Ng
theta = function [j. Val, gradient] = cost. Function(theta) j. Val = [ code to compute ]; gradient(1) = [code to compute ]; gradient(2) = [code to compute ]; gradient(n+1) = [ code to compute ]; Andrew Ng
- Slides: 69