Neural Network and its Applications Dr Priti Srinivas

Neural Network and its Applications Dr. Priti Srinivas Sajja Associate Professor G H Patel P G Department of Computer Science and Technology Sardar Patel University, Vallabh Vidyanagar 388 120, India September 15, 2010 Priti Srinivas Sajja 1

Introduction and Contact Information n Name: Dr. Priti Srinivas Sajja n Communication: n Email : priti_sajja@yahoo. com Mobile : +91 9824926020 n URL : http: //pritisajja. info n n Academic qualifications : Ph. D in Computer Science n Thesis title: Knowledge-Based Systems for Socio. Economic Development n Subject area of specialization : Artificial Intelligence n Publications : 84 in International/ National Conferences, Journals & Books September 15, 2010 Priti Srinivas Sajja 2

Lecture Plan Introduction and Background n n Artificial Intelligence and Data Pyramid Connectionist and Symbolic Systems Biological Neuron and Artificial Neuron Characteristics of Artificial Neural Network Architecture of ANN n n Hopfield Model and Parallel Relaxation Single Perceptron and Linearly Separable Models Multi Layer Perceptron and Back Propogation Supervised Learning and Training Data Example ANNs Hybrid Systems Conclusion and References September 15, 2010 Priti Srinivas Sajja 3

Artificial Intelligence “Artificial Intelligence(AI) is the study of how to make computers do things at which, at the moment, people are better” n September 15, 2010 Elaine Rich, Artificial Intelligence, Mc. Graw Hill Publications, 1986 Priti Srinivas Sajja 4

Knowledge-Based Systems (Symbolic Representation of Knowledge) Heuristics and models Wisdom Novelty Knowledge Rules K Experience Information Concepts Data Raw Data through fact finding Researching Understanding Absorbing Doing Interacting Reflecting Convergence from data to intelligence Knowledge-Based Systems (KBS) are Productive Artificial Intelligence Tools working in a narrow September 15, 2010 domain. Priti Srinivas Sajja 5

Structure of KBS Explanation/ Reasoning Knowledge Base Inference Engine Self Learning User Interface September 15, 2010 Priti Srinivas Sajja 6

Limitations of Symbolic Systems n Nature of knowledge n n Knowledge acquisition n n Limited knowledge structures support KBS development models n n Fact finding methods support only Tacit and higher level knowledge Multiple experts Knowledge representation n n Hard to characterize Voluminous Dynamic Only SAD/SE guidelines and a few quality metrics Large size of knowledge base September 15, 2010 Priti Srinivas Sajja 7

Biological and Artificial Neurons September 15, 2010 Priti Srinivas Sajja 8

Artificial Neural Network (towards Connectionist Representation of Knowledge) n Objective: Not to mimic brain functionality but to receive inspiration from the fact about working. n how brain is Characterized by: n n A large number of very simple neuron like processing elements. A large number of weighted connection between the elements. This weights encode the knowledge of a network. n Highly parallel and distributed control. n Emphasis on learning internal representation automatically. September 15, 2010 Priti Srinivas Sajja 9

Architectures of ANN n n n Hopfield network Perceptron Multi-layer Perceptron Self Organizing Network etc. September 15, 2010 Priti Srinivas Sajja 10

l e i f A m Si pl e d t e N w or k p o H n In a Hopfield network, all processing units/elements are in two states either n active or inactive. Units are connected to each other with weighted Connections. -1 Active -1 +1 +3 -1 +1 +2 n n +3 -2 +1 Inactive -1 A positively weighted connection indicates that the units tend to active each other. A negative connection allows an active unit to deactivate a neighbouring unit. September 15, 2010 Priti Srinivas Sajja 11

P a ar l ll e R a x a ti o n -1 Active -1 +1 +3 -1 +1 +2 +1 Inactive +3 -2 -1 • A random unit is chosen. • If any of its neighbours are active, the unit computes the sum of weights on the connections to those active neighbours. • If the sum is positive, the unit becomes active else new random unit is chosen. • This process will continue till the network become stable. That is no unit can change its status. This process is known as parallel relaxation. September 15, 2010 Priti Srinivas Sajja 12

Perceptron X 1 W 1 T ∑Wi Xi W 2 X 2 y: e? m Ar to B t o ot g i n or n o G e B o Mom T Importance to Mom 0. 6 ∑Wi Xi (0. 3) 0. 4 Dad (0. 5) September 15, 2010 Importance to Dad = 0. 3*0. 6 + 0. 5*0. 4 = 0. 18 +0. 20 = 0. 38 which is < 0. 6 Priti Srinivas Sajja 13

Logical Gate AND and OR X 1 0. 5 0. 6 ∑Wi Xi 0. 5 X 2 T Lo ND e A l abl a c T gi t ru h X 1 X 2 X 1 AND X 2 0 0 0*0. 5 + 0*0. 5 = 0 <0. 6 0 0 1 0*0. 5 + 1*0. 5 = 0 <0. 6 0 1*0. 5 + 0*0. 5 = 0. 5 <0. 6 0 1 1 1*0. 5 + 1*0. 5 = 1 1 (1, 1) September 15, 2010 >0. 6 Priti Srinivas Sajja 14

Logical Gate AND and OR (1, 1) September 15, 2010 Priti Srinivas Sajja 15

1 X 2 -1. 5 1. 0 ƒ -9. 0 1 X 2 a -0. 5 1. 0 ƒ 1. 0 K 10 September 15, 2010 A Pe Clas rceptio n sific atio Learni n Pr n oble g to So m lve wo. 41 w 2 -. 17 . 14 100 . 22 -. 14 . 11 300 -. 1 -. 008 . 07 635 -. 49 -. 1 . 14 Priti Srinivas Sajja 16

Output Layer O 1 O 2 Fu -fo lly c rw onn ar d n ecte etw d, m or ult ks il Oc tru aye ctu red W 2 ij re , fe e d Hidden Layer h 1 h 2 h 3 h. B w 1 ij Input Layer 1 x 2 x 3 x 4 x. A ……This network has three layers but there may be many. September 15, 2010 Priti Srinivas Sajja 17

Examples of Multilayer Perceptron d O 1 X 1 and X 2 are two one /two digited positive numbers X 1 H 1 X 2 H 2 e On o O 2 O 3 Tw ite g e d ee r Th ed it dig d te i g di Training Set Data 2, 3, 1, 0, 0 10, 0, 1, 0 90, 0, 0, 1 …… …. . … September 15, 2010 Priti Srinivas Sajja 18

r ye a l i t l c ian Mu of e l p m n a Ex eptro c Per l Re O 1 t e dg Bu X 1 X 2 r olo C H 2 X 3 H 3 X 4 H 4 ra me a C H 1 O 2 O 3 O 4 e se a B ia k No er igh H kia No r E ny So on s ics er b k ry c Bla O 5 io d Ra Training Set Data 2, 3, 0, 0, 1, 0, 0 6, 6, 7, 5, 0, 0, 1, 0, 0 8, 8, 0, 0, 1 …… …. . … September 15, 2010 Priti Srinivas Sajja 19

r ye a l i t l u M f le o p m n Exa eptro c Per O 1 X 1 y lar Sa t es er t In ce n a st Di H 1 X 2 H 2 X 3 H 3 X 4 H 4 O 2 b Jo A b Jo B b Jo C O 3 re tu Fu Design and Train the above structure using your own choices considering the following practical situations: A. Job At Bengaluru, salary Rs. 30, 000 per month of your field B. Job At USA, salary Rs. 80, 000 per month of other field C. Job At Anand, salary Rs. 25, 000 per month of your field September 15, 2010 Priti Srinivas Sajja 20

e n i rm rs e t De Use o N t de of N A itu t Ap September 15, 2010 Priti Srinivas Sajja 21

Am a g etho wa ener d wa ve ve ov ic de s de ry s v wi erto ign t elop de p e o Source: ran ping ol fo d to r ge p www. wldelft. nl/cons/area/wds/neural/index. html d of ischa estim rovid co ast rges atin e g al str for a uc tur es. September 15, 2010 Priti Srinivas Sajja 22

tor a l imu S N Sprayer Sensor and Nozzle Element (Zhang, Yang, & El. AN Faki, 1994) September 15, 2010 Priti Srinivas Sajja 23

ica l p Ap t s ion n n n n N N f. A o Optimization, function approximation, time series prediction, and modeling Classification, pattern matching and recognition (three-dimensional object recognition), novelty detection, and sequential decision-making Data processing (including filtering, clustering, blind source separation and compression), data mining, data compression (speech signal, image—for example, faces), and data validations System identification and control (vehicle control, process control) and signal processing Game playing and decision making (backgammon, chess, racing) Sequence recognition (gesture, handwritten text recognition) Medical diagnosis (for example, hepatitis or storing medical records based on case information) Financial applications (automated trading systems, time series analysis, stock market prediction) and customer research Cognitive science, neurobiology, and the study of models of how the brain works Biological neural networks, which communicate through pulses and use the timing of the pulses to transmit information and perform computations Integration of fuzzy logic and neural networks for applications in automotive engineering, screening applicants for jobs, controlling a crane, or monitoring a medical condition like glaucoma Robotics (navigation and vision recognition) Speech production and recognition Vision (face recognition, edge detection, visual search engines) New topologies and hardware implementations New learning algorithms In hybrid systems and soft computing—for example, rule extraction for fuzzy systems, self -evolving ANNs, and neuro-fuzzy systems September 15, 2010 Priti Srinivas Sajja 24

Hybrid Systems/Soft Computing Evolving neurofuzzy systems Neuro-fuzzy and fuzzy neural network Neural network Modular rough networks Genetic fuzzy Fuzzy logic Soft computing Genetic algorithms Probabilistic reasoning Genetic bayesian network Application Layer 1 Application Layer 2 Constituents of soft computing September 15, 2010 Priti Srinivas Sajja 25

Strength of a Hybrid Soft Computing System Field Strength Offered Fuzzy logic Approximate reasoning, impressions Artificial neural network Learning and implicit knowledge representation Genetic algorithm Natural evolution and optimization Probabilistic reasoning Uncertainty September 15, 2010 Priti Srinivas Sajja 26

Hybrid Applications Ne u rofuz Fuzzy Rules Fu zz y Int erf ac e Sy ste ms Output Data FIS ANN Fuzzy Sets (b) co-operative neuro-fuzzy model (a) fuzzy neural model Output Y 1 X 1 ANN FIS YN Xn (c) concurrent neuro-fuzzy model Y 2 X 2 Data zy (d) hybrid neuro-fuzzy model Approaches of neuro-fuzzy computing September 15, 2010 Priti Srinivas Sajja 27

Reference Knowledge-Based Systems Rajendra Akerkar and Priti Srinivas Sajja Jones & Bartlett Publishers Sudbury, MA, USA (2009) September 15, 2010 Priti Srinivas Sajja 28

mily a F TAR ia S I d M an arat, Ind V C e uj To th agar, G an Vidy September 15, 2010 Priti Srinivas Sajja 29