Review of Statistical Pattern Recognition WenHung Liao 1092007

Review Paper l l A. K. Jain, R. P. W. Duin and J. Mao,

Statistical Approach in PR l l l Each pattern is represented in terms of

Links Between Statistical and Neural Network Methods l l Linear Discriminant Function Principal Component

Model for Statistical Pattern Recognition Classification Preprocessing Feature Measurement Classification Training Preprocessing Feature Extraction

The Curse of Dimensionality l l The performance of a classifier depends on the

Class-Conditional Probability l l l Length d feature vector: x = (x 1, x

How Many Features are Enough? l l Question: More features, better classification? Answer: l

Dimensionality Reduction l Keep the number of features as small as possible (but not

Feature Extraction/Selection l l Feature Extraction: extract features from the sensed data Feature Selection:

Example: Chernoff Faces l l l Three classes of face Feature set: Nose length,

Slides: 13

Download presentation

Review of Statistical Pattern Recognition Wen-Hung Liao 10/9/2007

Review Paper l l A. K. Jain, R. P. W. Duin and J. Mao, “Statistical Pattern Recognition: A Review”, IEEE Transactions on Pattern Analysis and Machine Intelligence (PAMI), Vol. 22, No. 1, pp. 4 -37, Jan. 2000. More review papers: http: //www. ph. tn. tudelft. nl/PRInfo/revpapers. h tml

Statistical Approach in PR l l l Each pattern is represented in terms of d features and is viewed as a point in a ddimensional feature space. Goal: establish decision boundaries to separate patterns belonging to different classes. Need to specify/estimate the probability distributions of the patterns.

Various Approaches in Statistical PR

Links Between Statistical and Neural Network Methods l l Linear Discriminant Function Principal Component Analysis Nonlinear Discriminant Function Parzen Window Density -based Classifier l Perceptron l Auto-Associative Networks Multilayer Perceptron Radial Basis Function Network l l

Model for Statistical Pattern Recognition Classification Preprocessing Feature Measurement Classification Training Preprocessing Feature Extraction /Selection Learning

The Curse of Dimensionality l l The performance of a classifier depends on the relationship between sample sizes, number of features and classifier complexity. Number of training data points be an exponential function of the feature dimension space.

Class-Conditional Probability l l l Length d feature vector: x = (x 1, x 2, …, xd) C Classes (or categories): w 1, w 2, …, wc Class-conditional probability: The probability of x happening given that it belongs to class wi: p(x|wi)

How Many Features are Enough? l l Question: More features, better classification? Answer: l l Yes, if the class-conditional densities are completely known. No, if we need to estimate the classconditional densities.

Dimensionality Reduction l Keep the number of features as small as possible (but not too small) due to: l l l measurement cost classification accuracy Always some trade-off

Feature Extraction/Selection l l Feature Extraction: extract features from the sensed data Feature Selection: select (hopefully) the best subset of the input feature set. Feature extraction usually precedes selection Application-domain dependent

Example: Chernoff Faces l l l Three classes of face Feature set: Nose length, mouth curvature, eye size, face shape. 150 4 -d patterns, 50 patterns per class.

Chernoff Faces

Related presentations