Intro ANN Fuzzy Systems Lecture 9 MLP I

Intro. ANN & Fuzzy Systems Lecture 9 MLP (I): Feed-forward Model (C) 2001 by Yu Hen Hu

Intro. ANN & Fuzzy Systems Outline • • (C) 2001 by Yu Hen Hu Multi-Layer Perceptron Structure Feed Forward Model XOR Example MLP Applications 2

Intro. ANN & Fuzzy Systems Multi-Layer Perceptron Structure A Three Layer Feed-forward Multi-Layer Perceptron (C) 2001 by Yu Hen Hu 3

Intro. ANN & Fuzzy Systems Two Layer Perceptron XOR Gate Let x 1, x 2 {0, 1}, then y 1 = sgn(x 1 – x 2 – 0. 5) = x 1 AND y 2 = sgn(x 2 – x 1 – 0. 5) = x 2 AND z = sgn(y 1 + y 2 – 0. 5) = y 1 OR y 2 (C) 2001 by Yu Hen Hu 4

Intro. ANN & Fuzzy Systems Decision Boundaries of XOR • Linear Hyper-planes as decision boundaries x 1 – x 2 – 0. 5 = 0; and x 2 – x 1 – 0. 5 = 0 (C) 2001 by Yu Hen Hu 5

Intro. ANN & Fuzzy Systems MLP Nonlinear Mapping 1 1 x y (C) 2001 by Yu Hen Hu 2. 5 1 5 5 3 2 5 4 5 1 1 1 z 1 6

Intro. ANN & Fuzzy Systems MLP Feed-forward model Notation • (k) – Index of individual feature vectors, 1 k K. • ( ) -- Layer index, superscript, 0 L. = 0 input layer, = L output layer • i, j – ith and jth neuron in each layer, subscript Example: • zi( )(k): the output of ith neuron in the th layer corresponding to the kth feature vector. • wij( ): the value of the synaptic weight that connect the output of the jth neuron at 1 th layer to the jth neuron at the th layer. The value of the weight is updated once every epoch. (C) 2001 by Yu Hen Hu 7

Intro. ANN & Fuzzy Systems MLP Feed-forward model • Note that , and • The input layer usually consists of linear elements. Thus, a 2 -layer MLP will have two layers of non-linear neurons: the hidden layer, and the output layer. (C) 2001 by Yu Hen Hu 8

Intro. ANN & Fuzzy Systems Applications to Classification • Classification: Match output class to target class. MLP assigns each input feature vector to a membership of a particular class i. (C) 2001 by Yu Hen Hu 9

Intro. ANN & Fuzzy Systems Applications to Approximation • Approximation (regression, modeling) : Targets are real numbers instead of binary class membership. (C) 2001 by Yu Hen Hu 10