Neural Networks Chapter 8 8 1 FeedForward Neural

Neural Networks Chapter 8

8. 1 Feed-Forward Neural Networks

Figure 8. 1 A fully connected feedforward neural network

Neural Network Input Format Equation 8. 1

Neural Network Output Format

The Sigmoid Function Equation 8. 2

Figure 8. 2 The sigmoid function

8. 2 Neural Network Training: A Conceptual View

Supervised Learning with Feed-Forward Networks • Backpropagation Learning • Genetic Learning

Unsupervised Clustering with Self-Organizing Maps

Figure 8. 3 A 3 x 3 Kohonen network with two input layer nodes

8. 3 Neural Network Explanation • Sensitivity Analysis • Average Member Technique

8. 4 General Considerations • What input attributes will be used to build the network? • How will the network output be represented? • How many hidden layers should the network contain? • How many nodes should there be in each hidden layer? • What condition will terminate network training?

Neural Network Strengths • Work well with noisy data. • Can process numeric and categorical data. • Appropriate for applications requiring a time element. • Have performed well in several domains. • Appropriate for supervised learning and unsupervised clustering.

Weaknesses • Lack explanation capabilities. • May not provide optimal solutions to problems. • Overtraining can be a problem.

8. 5 Neural Network Training: A Detailed View

The Backpropagation Algorithm: An Example

Backpropagation Error Output Layer Equation 8. 3

Backpropagation Error Output Layer Equation 8. 4

Backpropagation Error Hidden Layer Equation 8. 5

The Delta Rule Equations 8. 6 and 8. 7

Root Mean Squared Error Equation 8. 8

Kohonen Self-Organizing Maps: An Example

Figure 8. 4 Connections for two output layer nodes

Classifying a New Instance Output Node = j Equation 8. 9

Adjusting the Weight Vectors Output Node = j Equation 8. 10

Building Neural Networks with i. DA Chapter 9

9. 1 A Four-Step Approach for Backpropagation Learning 1. 2. 3. 4. Prepare the data to be mined. Define the network architecture. Watch the network train. Read and interpret summary results.

Example 1: Modeling the Exclusive-OR Function

Figure 9. 1 A graph of the XOR function

Step 1: Prepare The Data To Be Mined

Figure 9. 2 XOR training data

Step 2: Define The Network Architecture

Figure 9. 3 Dialog box for supervised learning

Figure 9. 4 Training options for backpropagation learning

Step 3: Watch The Network Train

Figure 9. 5 Neural network execution window

Step 4: Read and Interpret Summary Results

Figure 9. 6 XOR output file for Experiment 1

Figure 9. 7 XOR output file for Experiment 2

Example 2: The Satellite Image Dataset

Step 1: Prepare The Data To Be Mined

Figure 9. 8 Satellite image data

Step 2: Define The Network Architecture

Figure 9. 9 Backpropagation learning parameters for the

Step 3: Watch The Network Train

Step 4: Read And Interpret Summary Results

Figure 9. 10 Statistics for the satellite image data

Figure 9. 11 Satellite image data: Actual and computed output

9. 2 A Four-Step Approach for Neural Network Clustering

Step 1: Prepare The Data To Be Mined The Deer Hunter Dataset

Step 2: Define The Network Architecture

Figure 9. 12 Learning parameters for unsupervised clustering

Step 3: Watch The Network Train

Figure 9. 13 Network execution window

Step 4: Read And Interpret Summary Results

Figure 9. 14 Deer hunter data: Unsupervised summary statistics

Figure 9. 15 Output clusters for the deer hunter dataset

9. 3 ESX for Neural Network Cluster Analysis