Neural Network Developed by Dr Eddie Ip Modified

Neural Network Developed by: Dr Eddie Ip Modified by: Dr Arif Ansari 1

Outline • • Where in business is NN used? How does it work? A function approximation method Case study 2

NN: Overview • • Wall Street’s “rocket science” Long history: 1950 Compete with expert system Several generations of ANN – Lab research – Hype : thinking machine – Practical apps in fast chips 3

NN: Overview • A handwritten recognition example • http: //members. aol. com/Trane 64/java/JRe c. html 4

NN: Overview • Learn by examples : mimic function of brain • Underlying technology: a network of “neurons” (nodes) connected by ‘nerves” (edges) – Each has a set of parameters – Each edge is weighted by the relative strength of that connection – Parameters & weights are given values after the network is “trained” 5

Where can NN be used? • Supervised learning – classification (directed DM) – prediction (directed DM) • Unsupervised learning – clustering (undirected DM) – Self Organizing Map (SOM) 6

Where can NN be used • real estate appraisal – NN learns by examples – Freddie Mac: Loan Prospector • Fraud detection – Master card • team up with Los Alamos Lab • use state-of-the-art technology that is deployed for military applications • report a saving of more than $50 m using the fraud detection system 7

Where can NN be used? • Direct mail – HNC’s Database Mining Marksman • for one bank, reduce cost by 50%, increase sales by 18% • system costs $48, 000 to set up (1995) • financial service customer relationship management (reader) • e-customer relationship management – e. g. HNC, Neo. Vista, Agnos’s Knowledge. STUDIO 8

How does NN work? • A set of inputs (nodes) • A set of outputs (one to a few) • Model biological neurons 9

How does NN work? • Inputs must be between 0 and 1 • Output is also between 0 and 1 • “Scaling” required for numerical values – (Value - min)/range 10

How does NN work? • Topology / Architecture – input – output – hidden layer – connection 11

Architecture of an artificial neural network Input 1 Input 2 Output Input 3 Input Layer Hidden Layer Output Layer 12

How does NN work? • Action is in hidden layer • Each node combines inputs to give an output 13

How does NN work? • How to combine? – combination function – transfer function 14

How does NN work? • Combination function • Put several values into one – weighted sum – maximum 15

How does NN work? • Transfer function – transfer value of combination function to output (of node) – sigmoid – turn the weighted sum back on 1 to 1 scale 16

How does NN work? • Training = let it learn from examples • Weights adjusted by procedure called backward propagation 17

How does NN work? • Training & testing sets • Overfitting problem 18

How does NN work? • Overfit problem – Getting “too close” to data – NN pays attention to noise rather than signal – Result: poor performance when applied to new data set – Remedy: use a separate data set (testing set) to stop training 19

How does NN work? • Evaluation set – Used for objectively evaluating performance of NN • Misclassification • Discrepancy between what it predicts and what is really observed – Performance metric based on learning sample too optimistic 20

How does NN work? • Steps in applying NN (p. 692) – Transform data – Select architecture – Train and test – Deploy 21

How does NN work? • • Devil is in the details Training set should cover full range Topology Choosing appropriate input variables – e. g. use output from Decision Tree 22

NN: function approximation • Relate the “inputs” and “outputs” • Mathematically a function approximation problem • NN combines “sigmoids” to produce nonlinear functions of any kind (almost) 23

Glossary • • NN: Neural Network ANN: Artificial Neural Network DM: Data Mining SOM: Self Organizing Maps 24