Machine Learning Introduction Study on the Coursera All

• Machine Learning - Grew out of work in AI(Artificial Intelligence) - New

Machine Learning Definition • Tom Mitchell (1998) Well-posed Learning Problem: A computer program is

• Suppose your email program watches which emails you do or do not

Machine Learning Algorithms - Supervised learning - Unsupervised learning - Others: - Reinforcement learning

Supervised Learning & Unsupervised Learning x 2 x 1 Supervised Learning Unsupervised Learning

Linear Regression with one Variable Housing Prices (Portland, OR) Price (in 1000 s of

Training set of housing prices Size in feet 2 (x) 2104 1416 1534 852

Size in feet 2 (x) 2104 1416 1534 852 … Training Set Hypothesis: ‘s:

y x Idea: Choose so that is close to for our training examples

Cost Function Hypothesis: Parameters: Cost Function: Goal: Simplified:

Price ($) in 1000’s Size in feet 2 (x) Question: How to minimize J?

Gradient Descent Have some function Want Outline: • Start with some • Keep changing

Gradient descent algorithm Correct: Simultaneous update Incorrect:

Gradient descent algorithm Notice : α is the learning rate.

If α is too small, gradient descent can be slow. If α is too

at local optima Current value of Unchange Gradient descent can converge to a local

Gradient Descent for Linear Regression Gradient descent algorithm Linear Regression Model

Gradient descent algorithm update and simultaneously

(for fixed , this is a function of x) (function of the parameters )

Linear Regression with multiple variables Hypothesis: Parameters: Cost function: Gradient descent: Repeat (simultaneously update

New algorithm Repeat Gradient Descent Previously (n=1): : Repeat (simultaneously update ) for

Examples: 1 1 Size (feet 2) Number of bedrooms Number of floors Age of

Summarize • This is a briefly Introduction about Supervised Learning(Classification)in Machine Leaning. • There

Slides: 34

Download presentation

• Machine Learning - Grew out of work in AI(Artificial Intelligence) - New capability for computers • Examples: - Database mining • Large datasets from growth of automation/web. • Web click data, medical records, biology, engineering - Applications can’t program by hand. • Handwriting recognition, most of Natural Language Processing (NLP), Computer Vision.

Machine Learning Definition • Tom Mitchell (1998) Well-posed Learning Problem: A computer program is said to learn from experience E with respect to some task T and some performance measure P, if its performance on T, as measured by P, improves with experience E.

• Suppose your email program watches which emails you do or do not mark as spam, and based on that learns how to better filter spam. What is the task T in this setting? T : Classifying emails as spam or not spam E : Watching you label emails as spam or not spam P: The number of emails correctly classified as spam/not spam “A computer program is said to learn from experience E with respect to some task T and some performance measure P, if its performance on T, as measured by P, improves with experience E. ”

Machine Learning Algorithms - Supervised learning - Unsupervised learning - Others: - Reinforcement learning - Recommender systems.

Supervised Learning & Unsupervised Learning x 2 x 1 Supervised Learning Unsupervised Learning

Linear Regression with one Variable Housing Prices (Portland, OR) Price (in 1000 s of dollars) Size (feet 2) Supervised Learning Regression Problem Given the “right answer” for each example in the data. Predict real-valued output

Training set of housing prices Size in feet 2 (x) 2104 1416 1534 852 … Price ($) in 1000's (y) 460 232 315 178 … Notation: Training Set m = Number of training examples Learning Algorithm x’s = “input” variable / features y’s = “output” variable / “target” variable Size of house Question : How to describe h? h Estimated price

Size in feet 2 (x) 2104 1416 1534 852 … Training Set Hypothesis: ‘s: Parameters How to choose ‘s ? Price ($) in 1000's (y) 460 232 315 178 …

y x Idea: Choose so that is close to for our training examples

Cost Function Hypothesis: Parameters: Cost Function: Goal: Simplified:

Price ($) in 1000’s Size in feet 2 (x) Question: How to minimize J?

Gradient Descent Have some function Want Outline: • Start with some • Keep changing to reduce until we hopefully end up at a minimum

Gradient descent algorithm Correct: Simultaneous update Incorrect:

Gradient descent algorithm Notice : α is the learning rate.

If α is too small, gradient descent can be slow. If α is too large, gradient descent can overshoot the minimum. It may fail to converge, or even diverge.

at local optima Current value of Unchange Gradient descent can converge to a local minimum, even with the learning rate α fixed. As we approach a local minimum, gradient descent will automatically take smaller steps. So, no need to decrease α over time.

Gradient Descent for Linear Regression Gradient descent algorithm Linear Regression Model

Gradient descent algorithm update and simultaneously

J( 0, 1) 1 0

(for fixed , this is a function of x) (function of the parameters )

Linear Regression with multiple variables Hypothesis: Parameters: Cost function: Gradient descent: Repeat (simultaneously update for every )

New algorithm Repeat Gradient Descent Previously (n=1): : Repeat (simultaneously update ) for

Examples: 1 1 Size (feet 2) Number of bedrooms Number of floors Age of home (years) Price ($1000) 2104 1416 1534 852 5 3 3 2 1 2 2 1 45 40 30 36 460 232 315 178 simultaneously update

Summarize • This is a briefly Introduction about Supervised Learning(Classification)in Machine Leaning. • There is still a lot of things in this subject, such as Clustering, Support Vector Machine(SVM), Dimensionality Reduction, ETC. The Core Idea of MS is very similar, hope you will be fond of the Machine Learning！ Thanks for Listening !