Artificial Neural Networks An Introduction What is a

Artificial Neural Networks An Introduction

What is a Neural Network? n n A human Brain A porpoise brain The brain in a living creature A computer program n n Simulates (at a very rudimentary level) a biological brain Limited connections

Artificial Neural Networks n n Artificial neural networks are information technology inspired by studies of the brain and nervous system ANNs are used to simulate the massively parallel processes that are effectively used in the brain for learning, and storing information and knowledge

Biological Neuron n n n Dendrites Axon Soma Membrane Synapse Neurotransmitter Spikes

Simple Neuron Configuration Inputs X 1 X 2 X 3 Weights W 1 W 2 W 3 W 4 X 4 Summation (weighted) Transfer Output (Y)

Threshold Logic Units n n Outputs are 0 or 1 If the activation (accumulated weighted input) is larger than threshold the unit generates a signal

Sigmoidal Transfer function Outputs are in the range from 0 to 1 y=1/(1+exp(-a)) Is differentiable

Neural Network Architecture n n n In feedforward NN, neurons are grouped into layers The neurons on each layer are the same type There are different types of layers n n n Input layer: receive input from external sources Output layer: communicate to user Hidden layer(s): neurons communicate only with other layers

Sample Network Configuration Input layer Hidden layer Output layer

Some Characteristics of ANN n n n n Tolerance to noise; Reliability; Two layer networks are restricted to linearly separable problems; Additional layers can solve more complicated problems; “Black Box”. Why? Non-linearity; Logic hidden in weights; Universal approximators.

Learning Methods n Supervised n n n Unsupervised n n n Error Backpropagation Counter-Propagation Hebb’s rule Competitive Learning Reinforcement

Error Backpropagation Algorithm n n n Generalized Delta Rule; Allowed training multi-layer ANN; Revived interest in ANN; Error terms are propagated back through the network; The weight coefficients are updated iteratively;

Error Backpropagation Algorithm: Drawbacks n n n Local Minima; Biologically implausible; Possibility of “network paralysis”; Slowness; Oscillations.

Problems solved by ANN n n n Classification Cluster Analysis Approximation Forecasting Association Data compression

Benefits of ANN n n n Parallelism Learning Generalization n n NN can learn the characteristics of a general category of objects on specific examples from that category Robustness (reliability) n n Tolerance to noise Performance does not degrade appreciably if some of its neurons or interconnections are lost (Distributed memory)

Limitations of ANN n n Two-layer NN limited to linearly separable problems Local minima & oscillations Number of hidden layers/units hard to determine Lack of transparency (perspicuity)

Sample of Applications n Business n n n n Credit scoring Bankruptcy prediction Bond rating Security trading Technological processes Robotics Consumer electronics