KTH ROYAL INSTITUTE OF TECHNOLOGY Machine learning IV

  • Slides: 21
Download presentation
KTH ROYAL INSTITUTE OF TECHNOLOGY Machine learning IV Artificial Neural Networks

KTH ROYAL INSTITUTE OF TECHNOLOGY Machine learning IV Artificial Neural Networks

On the Shoulder of Giants Much of the material in this slide set is

On the Shoulder of Giants Much of the material in this slide set is based upon: - ”Automated Learning techniques in Power Systems” by L. Wehenkel, Université Liege - ”Machine Learning” course by Andrew Ng, Associate Professor, Stanford University 2

Contents Repeating from last time Artificial Neural Networks 3

Contents Repeating from last time Artificial Neural Networks 3

Power Systems Analysis: An automated learning approach P(X) input output Source: Automatic Learning techniques

Power Systems Analysis: An automated learning approach P(X) input output Source: Automatic Learning techniques in Power Systems, L. Wehenkel 4

Contents Repeating from last time Artificial Neural Networks 5

Contents Repeating from last time Artificial Neural Networks 5

Artificial Neural Networks - Introduction Inspired by the Human Nerve system A close resemblance?

Artificial Neural Networks - Introduction Inspired by the Human Nerve system A close resemblance? (perceptron) Source: Automatic Learning techniques in Power Systems, L. Wehenkel 6

Regression VS classification Regression (linear and polynomial): relationship between the independent variable and the

Regression VS classification Regression (linear and polynomial): relationship between the independent variable and the dependent variable (e. g. , predict continuous price based on size) Classification: identifying to which of a set of categories a new observation belongs Source: ”Machine learning ” course, Andrew Ng 7

Linear regression: bias unit and normalization Source: ”Machine learning ” course, Andrew Ng 8

Linear regression: bias unit and normalization Source: ”Machine learning ” course, Andrew Ng 8

Linear regression: Over/under fitting - In over fitting, if we have too many features,

Linear regression: Over/under fitting - In over fitting, if we have too many features, the learned hypothesis may fit the training set very well, but fail to generalize to new examples (predict prices on new examples). Source: ”Machine learning ” course, Andrew Ng 9

Classification (logistic regression) - Non-linear perceptron normally uses a threhold function for the output,

Classification (logistic regression) - Non-linear perceptron normally uses a threhold function for the output, to limit the extreme values. Source: ”Machine learning ” course, Andrew Ng 10

Over/under fitting in classification: - In the next lecture, we will discuss how to

Over/under fitting in classification: - In the next lecture, we will discuss how to cope with the over fitting problem (e. g. , by using Regularization technique). Source: ”Machine learning ” course, Andrew Ng 11

Example (one-layer perceptron): AND function Hint: in the next lecture we learn how to

Example (one-layer perceptron): AND function Hint: in the next lecture we learn how to tune the weights optimally Source: ”Machine learning ” course, Andrew Ng 12

Example (one-layer network): OR function Hint: in the next lecture we learn how to

Example (one-layer network): OR function Hint: in the next lecture we learn how to tune the weights optimally Source: ”Machine learning ” course, Andrew Ng 13

Example (multi-layer perceptron): XNOR function Source: ”Machine learning ” course, Andrew Ng 14

Example (multi-layer perceptron): XNOR function Source: ”Machine learning ” course, Andrew Ng 14

Multi Layer Perceptrons (MLP) - A network of interconnected Perceptrons in several layers -

Multi Layer Perceptrons (MLP) - A network of interconnected Perceptrons in several layers - First layer recives input, forwards to second layer etc. - Normally one hidden layer is sufficient to create good mappings Source: Automatic Learning techniques in Power Systems, L. Wehenkel 15

Multi-class (more than two) classification Source: ”Machine learning ” course, Andrew Ng 16

Multi-class (more than two) classification Source: ”Machine learning ” course, Andrew Ng 16

Example: One Machine Infinite Bus (OMIB) system - We randomly sample values for Pu

Example: One Machine Infinite Bus (OMIB) system - We randomly sample values for Pu and Qu creating a database with 5000 samples (objects) and for each object we have a set of attributes (Pu, Qu, V 1, P 1, Vinf, Xinf, CCT). Source: Automatic Learning techniques in Power Systems, L. Wehenkel 17

Design ANN for the OMIB problem - Perceptrons use linear combination of inputs and

Design ANN for the OMIB problem - Perceptrons use linear combination of inputs and tanh function - We want to calculate the clearing time (CCT), i. e. This is a Regression problem Source: Automatic Learning techniques in Power Systems, L. Wehenkel 18

ANN structure and tuned weights - After tuning the weights Source: Automatic Learning techniques

ANN structure and tuned weights - After tuning the weights Source: Automatic Learning techniques in Power Systems, L. Wehenkel 19

Summury - Perceptrons used for prediction and classification - Multi layer perceptrons (MLP) uses

Summury - Perceptrons used for prediction and classification - Multi layer perceptrons (MLP) uses hidden layers to solve more complicated problems - In the next lecture: - How to tune the weights? (e. g. , using back propagation) How to cope with over/under fitting? (e. g. , usingregularization) 20

Assignment 21

Assignment 21