Linear regression with one variable Cost function Machine

Linear regression with one variable Cost function Machine Learning Andrew Ng

Training Set Size in feet 2 (x) 2104 1416 1534 852 … Price ($) in 1000's (y) 460 232 315 178 … Hypothesis: ‘s: Parameters How to choose ‘s ? Andrew Ng

3 3 3 2 2 2 1 1 1 0 0 1 2 3 Andrew Ng

y x Idea: Choose so that is close to for our training examples Andrew Ng

Linear regression with one variable Cost function intuition I Machine Learning Andrew Ng

Hypothesis: Simplified Parameters: Cost Function: Goal: Andrew Ng

(for fixed , this is a function of x) y (function of the parameter ) 3 3 2 2 1 1 0 0 1 x 2 3 0 -0. 5 0 0. 5 1 1. 5 2 2. 5 Andrew Ng

Linear regression with one variable Cost function intuition II Machine Learning Andrew Ng

Hypothesis: Parameters: Cost Function: Goal: Andrew Ng

(for fixed , this is a function of x) (function of the parameters ) 500 400 Price ($) 300 in 1000’s 200 100 0 0 1000 2000 Size in feet 2 (x) 3000 Andrew Ng

Andrew Ng

(for fixed , this is a function of x) (function of the parameters ) Andrew Ng

Linear regression with one variable Machine Learning Gradient descent Andrew Ng

Have some function Want Outline: • Start with some • Keep changing to reduce until we hopefully end up at a minimum Andrew Ng

J( 0, 1) 0 1 Andrew Ng

Gradient descent algorithm Correct: Simultaneous update Incorrect: Andrew Ng

Linear regression with one variable Gradient descent intuition Machine Learning Andrew Ng

Gradient descent algorithm Andrew Ng

Andrew Ng

If α is too small, gradient descent can be slow. If α is too large, gradient descent can overshoot the minimum. It may fail to converge, or even diverge. Andrew Ng

at local optima Current value of Andrew Ng

Gradient descent can converge to a local minimum, even with the learning rate α fixed. As we approach a local minimum, gradient descent will automatically take smaller steps. So, no need to decrease α over time. Andrew Ng

Linear regression with one variable Gradient descent for linear regression Machine Learning Andrew Ng

Gradient descent algorithm Linear Regression Model Andrew Ng

Andrew Ng

Gradient descent algorithm update and simultaneously Andrew Ng

Gradient descent example X y 1 2 3 6 5 10 h error h-y (h-y)x h error Andrew Ng

J( 0, 1) 0 1 Andrew Ng

Andrew Ng

(for fixed , this is a function of x) (function of the parameters ) Andrew Ng

Logistic Regression Classification Machine Learning Andrew Ng

Classification Email: Spam / Not Spam? Online Transactions: Fraudulent (Yes / No)? Tumor: Malignant / Benign ? 0: “Negative Class” (e. g. , benign tumor) 1: “Positive Class” (e. g. , malignant tumor) Andrew Ng

Classification: y = 0 or 1 can be > 1 or < 0 Logistic Regression: Andrew Ng

Logistic Regression Hypothesis Representation Machine Learning Andrew Ng

Logistic Regression Model Want 1 0. 5 Sigmoid function Logistic function 0 Andrew Ng

Logistic regression 1 z Suppose predict “ “ if Andrew Ng

Logistic Regression Cost function Machine Learning Andrew Ng

Training set: m examples How to choose parameters ? Andrew Ng

Cost function Linear regression: “non-convex” “convex” Andrew Ng

Logistic regression cost function If y = 1 0 1 Andrew Ng

Logistic regression cost function If y = 0 0 1 Andrew Ng

Logistic Regression Simplified cost function and gradient descent Machine Learning Andrew Ng

Logistic regression cost function Andrew Ng

Logistic regression cost function To fit parameters : To make a prediction given new : Output Andrew Ng

Gradient Descent Want : Repeat (simultaneously update all ) Andrew Ng

Gradient Descent Want : Repeat (simultaneously update all ) Algorithm looks identical to linear regression! Andrew Ng

Gradient Descent Want : Repeat (simultaneously update all ) Andrew Ng

Gradient Descent Want : Repeat (simultaneously update all ) Algorithm looks identical to linear regression! Andrew Ng

Andrew Ng

Chain rule Andrew Ng

Andrew Ng

Derivation of logistic regression Andrew Ng

Now Derive From Andrew Ng

Andrew Ng

theta = function [j. Val, gradient] = cost. Function(theta) j. Val = [ code to compute ]; gradient(1) = [code to compute ]; gradient(2) = [code to compute ]; gradient(n+1) = [ code to compute ]; Andrew Ng