SingleLayer Perceptrons PART 2 Neural Networks and Learning

  • Slides: 11
Download presentation
Single-Layer Perceptrons (PART 2) Neural Networks and Learning Machines, Third Edition Simon Haykin Copyright

Single-Layer Perceptrons (PART 2) Neural Networks and Learning Machines, Third Edition Simon Haykin Copyright © 2009 by Pearson Education, Inc. Upper Saddle River, New Jersey 07458 All rights reserved.

Chapter Organization • Linear Adaptive Filters and LMS algorithm – Adaptive Filtering Problem –

Chapter Organization • Linear Adaptive Filters and LMS algorithm – Adaptive Filtering Problem – Unconstrained Optimization Techniques • Method of Steepest Descent • Newton’s Method • Gauss-Newton Method – – Linear Least Squares Filter LMS Algorithm Learning Curves Learning-Rate Annealing Schedules (mostly omitted) • Rosenblatt’s Perceptron – Perceptron Convergence Theorem – Relation between Perceptron and Bayes Classifier for a Gaussian Environment (omitted) Neural Networks and Learning Machines, Third Edition Simon Haykin Copyright © 2009 by Pearson Education, Inc. Upper Saddle River, New Jersey 07458 All rights reserved.

Linear Least-Squares Filter • Neural Networks and Learning Machines, Third Edition Simon Haykin Copyright

Linear Least-Squares Filter • Neural Networks and Learning Machines, Third Edition Simon Haykin Copyright © 2009 by Pearson Education, Inc. Upper Saddle River, New Jersey 07458 All rights reserved.

Least-Mean-Square (LMS) Algorithm • • Neural Networks and Learning Machines, Third Edition Simon Haykin

Least-Mean-Square (LMS) Algorithm • • Neural Networks and Learning Machines, Third Edition Simon Haykin Copyright © 2009 by Pearson Education, Inc. Upper Saddle River, New Jersey 07458 All rights reserved.

Learning Curves Squared Error • Iterations Neural Networks and Learning Machines, Third Edition Simon

Learning Curves Squared Error • Iterations Neural Networks and Learning Machines, Third Edition Simon Haykin Copyright © 2009 by Pearson Education, Inc. Upper Saddle River, New Jersey 07458 All rights reserved.

Chapter Organization • Linear Adaptive Filters and LMS algorithm – Adaptive Filtering Problem –

Chapter Organization • Linear Adaptive Filters and LMS algorithm – Adaptive Filtering Problem – Unconstrained Optimization Techniques • Method of Steepest Descent • Newton’s Method • Gauss-Newton Method – – Linear Least Squares Filter LMS Algorithm Learning Curves Learning-Rate Annealing Schedules (mostly omitted) • Rosenblatt’s Perceptron – Perceptron Convergence Theorem – Relation between Perceptron and Bayes Classifier for a Gaussian Environment (omitted) Neural Networks and Learning Machines, Third Edition Simon Haykin Copyright © 2009 by Pearson Education, Inc. Upper Saddle River, New Jersey 07458 All rights reserved.

(Rosenblatt’s) Perceptron • LMS algorithm was described with a linear neuron but the perceptron

(Rosenblatt’s) Perceptron • LMS algorithm was described with a linear neuron but the perceptron is built around a nonlinear neuron • Output is +1 for positive and -1 for negative (can be thought as 2 classes) • The synaptic weights can be adapted by using perceptron convergence algorithm Neural Networks and Learning Machines, Third Edition Simon Haykin Copyright © 2009 by Pearson Education, Inc. Upper Saddle River, New Jersey 07458 All rights reserved.

Perceptron Convergence Theorem • Neural Networks and Learning Machines, Third Edition Simon Haykin Copyright

Perceptron Convergence Theorem • Neural Networks and Learning Machines, Third Edition Simon Haykin Copyright © 2009 by Pearson Education, Inc. Upper Saddle River, New Jersey 07458 All rights reserved.

Perceptron Convergence Theorem • Neural Networks and Learning Machines, Third Edition Simon Haykin Copyright

Perceptron Convergence Theorem • Neural Networks and Learning Machines, Third Edition Simon Haykin Copyright © 2009 by Pearson Education, Inc. Upper Saddle River, New Jersey 07458 All rights reserved.

Perceptron Convergence Theorem • Neural Networks and Learning Machines, Third Edition Simon Haykin Copyright

Perceptron Convergence Theorem • Neural Networks and Learning Machines, Third Edition Simon Haykin Copyright © 2009 by Pearson Education, Inc. Upper Saddle River, New Jersey 07458 All rights reserved.

Perceptron Convergence Theorem Neural Networks and Learning Machines, Third Edition Simon Haykin Copyright ©

Perceptron Convergence Theorem Neural Networks and Learning Machines, Third Edition Simon Haykin Copyright © 2009 by Pearson Education, Inc. Upper Saddle River, New Jersey 07458 All rights reserved.

SingleLayer Perceptrons PART 1 Neural Networks and LearningSingleLayer Perceptrons PART 1 Neural Networks and Learning
SingleLayer Perceptrons PART 2 Neural Networks and LearningSingleLayer Perceptrons PART 2 Neural Networks and Learning
Multilayer Perceptrons MLPs Singlelayer perceptrons can only representMultilayer Perceptrons MLPs Singlelayer perceptrons can only represent
Neural networks 1 Neural networks Neural networks areNeural networks 1 Neural networks Neural networks are
Neural networks 1 Neural networks Neural networks areNeural networks 1 Neural networks Neural networks are
Neural networks 1 Neural networks Neural networks areNeural networks 1 Neural networks Neural networks are
Outline Announcement Neural networks Perceptrons continued Multilayer neuralOutline Announcement Neural networks Perceptrons continued Multilayer neural
Multilayer Perceptrons PART 2 Neural Networks and LearningMultilayer Perceptrons PART 2 Neural Networks and Learning
Multilayer Perceptrons PART 1 Neural Networks and LearningMultilayer Perceptrons PART 1 Neural Networks and Learning
Multilayer Perceptrons Neural Networks n n n NetworksMultilayer Perceptrons Neural Networks n n n Networks