Neural Networks Teacher Elena Marchiori R 4 47

Neural Networks Teacher: Elena Marchiori R 4. 47 elena@cs. vu. nl Neural Networks Assistant: Kees Jong S 2. 22 cjong@cs. vu. nl NN 1 1

Course Outline Basics of neural network theory and practice for supervised and unsupervised learning. Most popular Neural Network models: • architectures • learning algorithms • applications Neural Networks NN 1 2

Course Outline Rules: - 4 s. p - Final mark is based on two assignments, which will be available at the end of the course. - one assignment is on theory (to do alone). - one assignment is on practice (to do in couples). - Programming in Matlab 5. 3. - Registration: send email to cjong@cs. vu. nl Neural Networks NN 1 3

Course Organization • There is no text book. • Course schedule, slides and exercises will be available at http: //www. cs. vu. nl/~elena/nn. html Neural Networks NN 1 4

Neural Networks • A NN is a machine learning approach inspired by the way in which the brain performs a particular learning task: – Knowledge about the learning task is given in the form of examples. – Inter neuron connection strengths (weights) are used to store the acquired information (the training examples). – During the learning process the weights are modified in order to model the particular learning task correctly on the training examples. Neural Networks NN 1 5

Learning • Supervised Learning – Recognizing hand-written digits, pattern recognition, regression. – Labeled examples (input , desired output) – Neural Network models: perceptron, feed-forward, radial basis function, support vector machine. • Unsupervised Learning – Find similar groups of documents in the web, content addressable memory, clustering. – Unlabeled examples (different realizations of the input alone) – Neural Network models: self organizing maps, Hopfield networks. Neural Networks NN 1 6

Network architectures • Three different classes of network architectures – single-layer feed-forward – multi-layer feed-forward – recurrent neurons are organized in acyclic layers • The architecture of a neural network is linked with the learning algorithm used to train Neural Networks NN 1 7

Single Layer Feed-forward Input layer of source nodes Neural Networks Output layer of neurons NN 1 8

Multi layer feed-forward 3 -4 -2 Network Output layer Input layer Hidden Layer Neural Networks NN 1 9

Recurrent network Recurrent Network with hidden neuron(s): unit delay operator z-1 implies dynamic system z-1 input hidden output z-1 Neural Networks NN 1 10

Neural Network Architectures Neural Networks NN 1 11

The Neuron • The neuron is the basic information processing unit of a NN. It consists of: 1 A set of synapses or connecting links, each link characterized by a weight: W 1, W 2, …, Wm 2 An adder function (linear combiner) which computes the weighted sum of the inputs: 3 Activation function (squashing function) limiting the amplitude of the output of the neuron. Neural Networks NN 1 for 12

The Neuron Bias x 1 b w 1 Local Field Input signal x 2 v w 2 Activation function Output y Summing function xm wm Synaptic weights Neural Networks NN 1 13

Bias of a Neuron • Bias b has the effect of applying an affine transformation to u v=u+b • v is the induced field of the neuron v u Neural Networks NN 1 14

Bias as extra input • Bias is an external parameter of the neuron. Can be modeled by adding an extra input. x 0 = +1 x 1 Input signal x 2 w 0 w 1 Local Field v w 2 Activation function Output y Summing function xm Neural Networks wm Synaptic weights NN 1 15

Dimensions of a Neural Network • Various types of neurons • Various network architectures • Various learning algorithms • Various applications Neural Networks NN 1 16

Face Recognition 90% accurate learning head pose, and recognizing 1 -of-20 faces Neural Networks NN 1 17

Handwritten digit recognition Neural Networks NN 1 18