DESIGN OF A SELFORGANIZING LEARNING ARRAY SYSTEM IEEE

DESIGN OF A SELFORGANIZING LEARNING ARRAY SYSTEM IEEE International Symposium on Circuits and Systems Dr. Janusz Starzyk Tsun-Ho Liu May 25 -28 th, 2003 Ohio University November 8 th, 2002 School of Electrical Engineering and Computer Science

Outline § Introduction § Self-Organizing Learning Array Structure § Neuron Structure and Self-Organizing Principles § Data Preprocessing § Software Simulation Result § Conclusion and Future Work May 25 -28 th, 2003 School of Electrical Engineering and Computer Science 2

Introduction § Digital computers are good at: § Fast arithmetic calculation § Precise software execution § Artificial Neural Networks are good at: § § Software free Robust classification and pattern recognition Recommendation of an action Massive parallelism May 25 -28 th, 2003 School of Electrical Engineering and Computer Science 3

Introduction (Cont’d) § Research Objective: § § Less interconnection Self-organizing Local Learning Nonspecific classification May 25 -28 th, 2003 School of Electrical Engineering and Computer Science 4

Self-Organizing Learning Array Structure (Cont’d) § Feed forward organization and structure May 25 -28 th, 2003 School of Electrical Engineering and Computer Science 5

Self-Organizing Learning Array Structure (Cont’d) § Initial Wiring May 25 -28 th, 2003 School of Electrical Engineering and Computer Science 6

Neuron Structure and Self. Organizing Principles § Neuron Input - System clock May 25 -28 th, 2003 School of Electrical Engineering and Computer Science 7

Neuron Structure and Self. Organizing Principles (Cont’d) § Neuron Input - Data input May 25 -28 th, 2003 School of Electrical Engineering and Computer Science 8

Neuron Structure and Self. Organizing Principles (Cont’d) § Neuron Input - Threshold control input (TCI) May 25 -28 th, 2003 School of Electrical Engineering and Computer Science 9

Neuron Structure and Self. Organizing Principles (Cont’d) § Neuron Input - Input information deficiency § Indication of how much the input space (corresponding to this selected TCI) has been learned § [0 , 1] § 1 is set initially at the first input layer § 0 indicates this neuron has solved the problem 100% May 25 -28 th, 2003 School of Electrical Engineering and Computer Science 10

Neuron Structure and Self. Organizing Principles (Cont’d) § Neuron inside § Transformation functions § Linear and nonlinear § Single input or multiple inputs § Information index calculation May 25 -28 th, 2003 School of Electrical Engineering and Computer Science 11

Neuron Structure and Self. Organizing Principles (Cont’d) May 25 -28 th, 2003 School of Electrical Engineering and Computer Science 12

Neuron Structure and Self. Organizing Principles (Cont’d) May 25 -28 th, 2003 School of Electrical Engineering and Computer Science 13

Neuron Structure and Self. Organizing Principles (Cont’d) § Neuron output - System output May 25 -28 th, 2003 School of Electrical Engineering and Computer Science 14

Neuron Structure and Self. Organizing Principles (Cont’d) § Neuron output - Output Clock May 25 -28 th, 2003 School of Electrical Engineering and Computer Science 15

Neuron Structure and Self. Organizing Principles (Cont’d) § Neuron output - Output information deficiency § of TCO = Input information deficiency § of TCOT = Input information deficiency * local information deficiency (pass threshold) § of TCOTI = Input information deficiency * local information deficiency (does not pass threshold) May 25 -28 th, 2003 School of Electrical Engineering and Computer Science 16

Data Preprocessing § Missing data recovery § All features are independent § Some features are dependent § Ref: [Liu] & [Starzyk & Zhu] § Symbolic values assignment § Number of numerical feature = 1 § Number of numerical features > 1 May 25 -28 th, 2003 School of Electrical Engineering and Computer Science 17

Symbolic value – numerical feature =1 1) 3) 2) 4) May 25 -28 th, 2003 School of Electrical Engineering and Computer Science 18

Symbolic value – numerical feature =1 § Symbolic value – numerical feature =1 Xs = [1. 0 3. 5 8. 5 9. 0]T May 25 -28 th, 2003 School of Electrical Engineering and Computer Science 19

Data Preprocessing (Cont’d) 1) 4) 2) 5) 3) May 25 -28 th, 2003 School of Electrical Engineering and Computer Science 20

Data Preprocessing (Cont’d) § Symbolic value – numerical feature > 1 Xs = [1. 0 2. 85 3. 274 7. 241 7. 884]T May 25 -28 th, 2003 School of Electrical Engineering and Computer Science 21

Software Simulation Result May 25 -28 th, 2003 School of Electrical Engineering and Computer Science 22

Software Simulation Result (Cont’d) FSS Naïve Bayes NBTree C 4. 5 -auto IDTM (Decision table) HOODG / SOLAR C 4. 5 rules OC 1 C 4. 5 Voted ID 3 (0. 6) CN 2 Naïve-Bayes Voted ID 3 (0. 8) T 2 1 R Nearest-neighbor (3) Nearest-neighbor (1) Pebls May 25 -28 th, 2003 0. 1405 0. 1410 0. 1446 0. 1482 0. 1494 0. 1504 0. 1554 0. 1564 0. 1600 0. 1612 0. 1647 0. 1687 0. 1954 0. 2035 0. 2142 Crashed School of Electrical Engineering and Computer Science 23

Conclusion and Future Work § Conclusion § Local learning § Self-organizing § Data preprocessing § Future work § VHDL simulation § FPGA machine § VLSI design May 25 -28 th, 2003 School of Electrical Engineering and Computer Science 24

Reference § Information & Computer Science (ICS), University of California at Irvine (UCI). (1995, December), Machine Learning Repository, Available FTP: Hostname: ftp. ics. uci. edu Directory: /pub/machinelearning-databases/ § Liu T. H. (2002), Thesis, Future Hardware Realization of Self. Organizing Learning Array and Its Software Simulation. School of Electrical Engineering and Computer Science, Ohio University. § Starzyk A. J. and Zhu Z. (2002), Software Simulation of a Self. Organizing Learning Array. Int. Conf. on Artificial Intelligence and Soft Computing. May 25 -28 th, 2003 School of Electrical Engineering and Computer Science 25