Artificial Neural Networks ANNs and the Error Backpropagation

Artificial Neural Networks (ANNs) and the Error Backpropagation Procedure Prof. Carolina Ruiz Department of Computer Science Worcester Polytechnic Institute

A 2 -layer feedforward ANN Input hidden layer -1 output layer -1 -1

A B Out Error Backpropagation 1. Initialize the weights to small random values -1 A 0. 5 0. 1 C -0. 2 -0. 1 E 0. 05 B 0. 2 -1 0. 3 D 0. 5 -1 0 0 1 1 0

Error Backpropagation A B Out 2. For each of the examples: 2. 1. Present example to input layer 2. 2. Propagate the example forward 0 0 1 1 0 -1 0 A 0. 5 0. 1 0. 377 C -0. 2 -0. 1 E 0. 05 B 0 0. 2 -1 D 0. 377 0. 5 -1 0. 3 0. 5094

Error Backpropagation A B Out 2. For each of the examples: 2. 3. Compute node errors for output layer 2. 4. Compute node errors for hidden layer 0 0 1 1 0 -1 0 A 0. 5 0. 1 C 0. 025 0. 377 -0. 2 -0. 1 E 0. 05 B 0 0. 2 -1 0. 3 D 0. 5 -1 0. 377 -0. 0382 -0. 5094

Error Backpropagation A B Out 2. For each of the examples: 2. 5. Compute and record weight change for each connection 0 0 1 1 0 -1 0 0. 1 A A->C A->D B->C B B->D C->E 0 0. 5 C 0. 0000 -0. 0481 D->E -0. 0481 0. 025 0. 377 -0. 2 -0. 1 E 0. 05 0. 2 -1 0. 3 D 0. 5 -1 0. 377 -0. 0382 -0. 5094

Error Backpropagation A B Out 3. After processing all examples update weight 4. Repeat process until obtaining “good” weights 0 0 1 1 0 -1 0 0. 1 A A->C A->D B->C B B->D C->E 0 0. 5 C 0. 0001 -0. 0795 0. 0004 -0. 0863 0. 3853 D->E -0. 049 0. 025 0. 377 -0. 2 -0. 1 E 0. 05 0. 2 -1 0. 3 D 0. 5 -1 0. 377 -0. 0382 -0. 5094