Artificial Neural Network BackPropagation Neural Network Yusuf Hendrawan

Artificial Neural Network (Back-Propagation Neural Network) Yusuf Hendrawan, STP. , M. App. Life Sc. , Ph. D

Neurons Biological Artificial http: //research. yale. edu/ysm/images/78. 2/articles-neural-neuron. jpg http: //faculty. washington. edu/chudler/color/pic 1 an. gif

A typical AI agent

Neural Network Layers http: //smig. usgs. gov/SMIG/features_0902/tualatin_ann. fig 3. gif • Each layer receives its inputs from the previous layer and forwards its outputs to the next layer

Multilayer feed forward network It contains one or more hidden layers (hidden neurons). “Hidden” refers to the part of the neural network is not seen directly from either input or output of the network. The function of hidden neuron is to intervene between input and output. By adding one or more hidden layers, the network is able to extract higherorder statistics from input

Neural Network Learning Back-Propagation Algorithm: function BACK-PROP-LEARNING(examples, network) returns a neural network inputs: examples, a set of examples, each with input vector x and output vector y network, a multilayer network with L layers, weights Wj, i , activation function g repeat for each e in examples do for each node j in the input layer do aj ‰xj[e] for l = 2 to M do ini ‰ åj Wj, i aj ai ‰ g(ini) for each node i in the output layer do Dj ‰g’(inj) åi Wji Di for l = M – 1 to 1 do for each node j in layer l do Dj ‰g’(inj) åi Wj, i Di for each node i in layer l + 1 do Wj, i ‰Wj, i + a x aj x Di until some stopping criterion is satisfied return NEURAL-NET-HYPOTHESIS(network) [Russell, Norvig] Fig. 20. 25 Pg. 746

Back-Propagation Illustration ARTIFICIAL NEURAL NETWORKS Colin Fahey's Guide (Book CD)

Input (X) Hidden Output (Y) Z 1 X 1 Z 2 Y Z 3 X 2 Z 4 Vo Wo

Input (X) Output / Target (T) X 1 X 2 T 0. 3 0. 4 0. 1 0. 5 0. 6 0. 8 0. 2 0. 3 0. 4 0. 7 0. 5 Jumlah Neuron pada Input Layer 2 Jumlah Neuron pada Hidden Layer 4 Jumlah Neuron pada Output Layer 1 Learning rate (α) 0. 1 Momentum (m) 0. 9 Target Error 0. 01 Maximum Iteration 1000

Bobot Awal Input ke Hidden Bias ke Hidden V 11 = 0. 75 V 21 = 0. 35 Vo 11 = 0. 07 Vo 21 = 0. 12 V 12 = 0. 54 V 22 = 0. 64 Vo 12 = 0. 91 Vo 22 = 0. 23 V 13 = 0. 44 V 23 = 0. 05 Vo 13 = 0. 45 Vo 23 = 0. 85 V 14 = 0. 32 V 24 = 0. 81 Vo 14 = 0. 25 Vo 24 = 0. 09 Bobot Awal Hidden ke Output Bias ke Output W 1 = 0. 04 Wo 1 = 0. 66 W 2 = 0. 95 Wo 2 = 0. 56 W 3 = 0. 33 Wo 3 = 0. 73 W 4 = 0. 17 Wo 4 = 0. 01

Menghitung Zin & Z dari input ke hidden Zin(1) = (X 1 * V 11) + (X 2 * V 21) = (0. 3 * 0. 75) + (0. 4 * 0. 35) = 0. 302 Zin(2) = (X 1 * V 12) + (X 2 * V 22) = (0. 3 * 0. 54) + (0. 4 * 0. 64) = 0. 418 Zin(3) = (X 1 * V 13) + (X 2 * V 23) = (0. 3 * 0. 44) + (0. 4 * 0. 05) = 0. 152 Zin(4) = (X 1 * V 14) + (X 2 * V 24) = (0. 3 * 0. 32) + (0. 4 * 0. 81) = 0. 42

Menghitung Yin & Y dari hidden ke output Yin = (Z(1) * W 1) + (Z(2) * W 2) + (Z(3) * W 3) + (Z(4) * W 4) = (0. 57 * 0. 04) + (0. 603 * 0. 95) + (0. 538 * 0. 33) + (0. 603 * 0. 17) = 0. 876 Menghitung dev antara Y dengan output nyata dev = (T - Y) * Y * (1 - Y) = (0. 1 – 0. 706) * 0. 706 * (1 – 0. 706) = -0. 126 Menghitung selisih = T - Y= -0. 606

Back-Propagation Menghitung din dari output ke hidden din(1) = (dev * W 1) = (-0. 126 * 0. 04) = -0. 00504 din(2) = (dev * W 2) = (-0. 126 * 0. 95) = -0. 1197 din(3) = (dev * W 3) = (-0. 126 * 0. 33) = -0. 04158 din(4) = (dev * W 4) = (-0. 126 * 0. 17) = -0. 02142 Menghitung d d (1) = (din(1) * Z(1) * (1 - Z(1) ) = (-0. 00504 * 0. 575 * (1 – 0. 575) = -0. 00123 d (2) = (din(2) * Z(2) * (1 - Z(2) ) = (-0. 1197 * 0. 603 * (1 – 0. 603) = -0. 02865 d (3) = (din(3) * Z(3) * (1 - Z(3) ) = (-0. 04158 * 0. 538 * (1 – 0. 538) = -0. 01033 d (4) = (din(4) * Z(4) * (1 - Z(4) ) = (-0. 02142 * 0. 603 * (1 – 0. 603) = -0. 00512

Mengkoreksi bobot (W) dan bias (Wo) W 1 = W 1 + (α * dev * Z(1) ) + (m * Wo(1)) = 0. 04 + (0. 1 * -0. 126 * 0. 575) + (0. 9 * 0. 66) = 0. 627 W 2 = W 2 + (α * dev * Z(2) ) + (m * Wo(2)) = 0. 95 + (0. 1 * -0. 126 * 0. 603) + (0. 9 * 0. 56) = 1. 45 W 3 = W 3 + (α * dev * Z(3) ) + (m * Wo(3)) = 0. 33 + (0. 1 * -0. 126 * 0. 538) + (0. 9 * 0. 73) = 0. 98 W 4 = W 4 + (α * dev * Z(4) ) + (m * Wo(4)) = 0. 17 + (0. 1 * -0. 126 * 0. 603) + (0. 9 * 0. 01) = 0. 171 Wo 1 = (α * Z(1) ) + (m * Wo(1)) = (0. 1 * 0. 575) + (0. 9 * 0. 66) = 0. 65 Wo 2 = (α * Z(2) ) + (m * Wo(2)) = (0. 1 * 0. 603) + (0. 9 * 0. 56) = 0. 564 Wo 3 = (α * Z(3) ) + (m * Wo(3)) = (0. 1 * 0. 538) + (0. 9 * 0. 73) = 0. 71 Wo 4 = (α * Z(4) ) + (m * Wo(4)) = (0. 1 * 0. 603) + (0. 9 * 0. 01) = 0. 0693

Mengkoreksi bobot (V) dan bias (Vo) V 11 = V 11 + (α * d (1) * X 1 ) + (m * Vo(11)) = 0. 75 + (0. 1 * -0. 00123 * 0. 3) + (0. 9 * 0. 07) = 0. 8129 V 12 = V 12 + (α * d (2) * X 1 ) + (m * Vo(12)) = 0. 54 + (0. 1 * -0. 02865 * 0. 3) + (0. 9 * 0. 91) = 1. 3581 V 13 = V 13 + (α * d (3) * X 1 ) + (m * Vo(13)) = 0. 44 + (0. 1 * -0. 01033 * 0. 3) + (0. 9 * 0. 45) = 0. 8446 V 14 = V 14 + (α * d (4) * X 1 ) + (m * Vo(14)) = 0. 32 + (0. 1 * -0. 00512 * 0. 3) + (0. 9 * 0. 25) = 0. 5448 V 21 = V 21 + (α * d (1) * X 2 ) + (m * Vo(21)) = 0. 35 + (0. 1 * -0. 00123 * 0. 4) + (0. 9 * 0. 12) = 0. 4579 V 22 = V 22 + (α * d (2) * X 2 ) + (m * Vo(22)) = 0. 64 + (0. 1 * -0. 02865 * 0. 4) + (0. 9 * 0. 23) = 0. 8458 V 23 = V 23 + (α * d (3) * X 2 ) + (m * Vo(23)) = 0. 05 + (0. 1 * -0. 01033 * 0. 4) + (0. 9 * 0. 85) = 0. 8145 V 24 = V 24 + (α * d (4) * X 2 ) + (m * Vo(24)) = 0. 81 + (0. 1 * -0. 00512 * 0. 4) + (0. 9 * 0. 09) = 0. 8907

Mengkoreksi bobot (V) dan bias (Vo) Vo 11 = (α * d (1) * X 1 ) + (m * Vo 11) = (0. 1 * -0. 00123*0. 3)+(0. 9*0. 07) = 0. 0629 Vo 12 = (α * d (2) * X 1 ) + (m * Vo 12) = (0. 1 * -0. 02865*0. 3)+(0. 9*0. 91) = 0. 8181 Vo 13 = (α * d (3) * X 1 ) + (m * Vo 13) = (0. 1 * -0. 01033*0. 3)+(0. 9*0. 45) = 0. 4046 Vo 14 = (α * d (4) * X 1 ) + (m * Vo 14) = (0. 1 * -0. 00512*0. 3)+(0. 9*0. 25) = 0. 2248 Vo 21 = (α * d (1) * X 2 ) + (m * Vo 21) = (0. 1 * -0. 00123*0. 4)+(0. 9*0. 12) = 0. 1079 Vo 22 = (α * d (2) * X 2 ) + (m * Vo 22) = (0. 1 * -0. 02865*0. 4)+(0. 9*0. 23) = 0. 2058 Vo 23 = (α * d (3) * X 2 ) + (m * Vo 23) = (0. 1 * -0. 01033*0. 4)+(0. 9*0. 85) = 0. 7645 Vo 24 = (α * d (4) * X 2 ) + (m * Vo 24) = (0. 1 * -0. 00512*0. 4)+(0. 9*0. 09) = 0. 0807