Fuzzy Pattern Recognition Overview of Pattern Recognition Pattern
- Slides: 43
Fuzzy Pattern Recognition
Overview of Pattern Recognition • Pattern Recognition Procedure Unknown Speech /Image /Data Feature Extraction Feature Reduction Classification (supervised) Class Label Known Clustering (unsupervised or self-organizing) Performance Criteria Clusters Cluster Validity
Overview of Pattern Recognition • Supervised Learning for Classification – The class label is known for a set of samples. – Find the decision boundary from the given samples. – For unknown data set, do classification • Unsupervised Learning for Clustering – Set of data is given, find the group or grouping boundary • Reinforcement Learning (Reward/Penalty) – Unkind teacher is given – Trial and Error Scheme
Overview of Pattern Recognition • Classification and Clustering Problem: Which class to assign Problem: How to partition Class 1 How many clusters Class 2 ? Classification Clustering
Overview of Pattern Recognition • Pattern Recognition Algorithm – Based on statistical approach • Parametric Approach – Bay’es Classifier with Gaussian Density – Nonlinear Boundary or Decision Function • Nonparametric Approach for Density Estimation – Parzen window – K-nearest method – Based on Neural Networks • Classifier – Multilayer Perceptron, ART, Neocogntion, … • Clustering – SOM(Self-Organizing Map)
Fuzzy Pattern Recognition • Classification – Rule-Based Classifier – Fuzzy Perceptron – Fuzzy K-NN Algorithm • Clustering – – Fuzzy C-Mean Possibilistic C-Mean Fuzzy C-Shell Clustering Fuzzy Rough Clustering • Cluster Validity – Validity Measures Based on Fuzzy Set Theory
Fuzzy Pattern Recognition
Fuzzy Classification • Rule-Based Classifier – Idea: Nonlinear Partition of Feature Space – How to find the rule from sample data. • Project the labeled training data, and design membership functions • Fuzzy clustering and projection to obtain membership function
Fuzzy Classification • Fuzzy K-Nearest Neighbor Algorithm – Crisp K-NN Algorithm Class 1 Class 2 K=3 Class 1
Fuzzy Classification • Fuzzy K-Nearest Neighbor Algorithm – Fuzzy K-NN Algorithm Class 1 Class 2
Fuzzy Nearest Prototype Classification • Crisp and Fuzzy Nearest Prototype Classification Prototype of Class 1 Prototype of Class 2 Decision Boundary
• Crisp Version • Fuzzy Version
Fuzzy Perceptron • Crisp Single-Layer Perceptron (Two-class problem) Find the linear decision boundary of separable data Linear Decision Boundary
Fuzzy Perceptron • Fuzzy Perceptron
Fuzzy Perceptron • Fuzzy Perceptron • Advantage – Generalize the crisp algorithm – Elegant termination in non-separable case – Crisp case: Not terminate in finite time
Fuzzy Perceptron • Termination of FP – If misclassifications are all caused by very fuzzy data, then terminate the learning. • Note: FP can be combined with kernel-based method. (J. H. Chen & C. S. Chen, IEEE Trans. On NNs, 2002)
Fuzzy C-Mean • Clustering Objective – The aim of the iterative algorithm is to decrease the value of an objective function • Notations – Samples – Prototypes – L 2 -distance:
Fuzzy C-Mean • Crisp objective: • Fuzzy objective
Fuzzy C-Mean • Crisp C-Mean Algorithm – Initiate k seeds of prototypes p 1, p 2, …, pk – Grouping: Assign samples to their nearest prototypes Form non-overlapping clusters out of these samples – Centering: Centers of clusters become new prototypes – Repeat the grouping and centering steps, until convergence
Fuzzy C-Mean • Crisp C-Mean Algorithm – Grouping: Assigning samples to their nearest prototypes helps to decrease the objective – Centering: Also helps to decrease the above objective, because and equality holds only if
Fuzzy C-Mean • Membership matrix: Uc×n – Uij is the grade of membership of sample j with respect to prototype i • Crisp membership: • Fuzzy membership:
Fuzzy C-Mean • Objective function of FCM • Introducing the Lagrange multiplier λ with respect to the constraint the objective function as:
Fuzzy C-Mean • Setting the partial derivatives to zero, From the 2 nd equation, From this fact and the 1 st equation,
Fuzzy C-Mean • Therefore, updating rule is
Fuzzy C-Mean • Setting the derivative of J with respect to pi to zero,
Fuzzy C-Mean • Update rule of ci: • To summarize:
Fuzzy C-Mean K-means Fuzzy c-means
Fuzzy C-Mean
Fuzzy C-Mean • Gustafson-Kessel Algorithm
Cluster Validity to Determine Number of Clusters
Extraction of Rule Base from Fuzzy Cluster
Possibilistic C-Mean • Problem of FCM – Equal Evidence = Ignorance
Possibilistic C-Mean • Objective Function of Fuzzy C-Mean – Constraint from Ruspini: Sum of membership of a datum over all classes should be 1. – Too restrictive condition for noisy data • Objective Function of PCM – Minimize intra-cluster distance – Make membership as large as possible
Possibilistic C-Mean • Necessary Condition • Determination of – Average cluster distance – Based on alpha-cut
Possibilistic C-Mean • Membership according to
Possibilistic C-Mean • Cluster Centers • Inner Product – Gustafson-Kessel (See previous page) – Spherical shell cluster
Possibilistic C-Mean • 2 -Pass Algorithm: – Initialize PC Partition – DO Until (Change in PC Partition is Small) • Update Prototype • Update PC Partition using average cluster distances – Based on the resulted PC Partition – DO Until (Change in PC Partition is Small) • Update Prototype • Update PC Partition using alpha-cut distances
Possibilistic C-Mean • Advantage – Robust to noisy data – Possibly good to get the fuzzy rule base FCM-Based C-Shell PCM-Based C-Shell
Other Notion of Distance • Other Notion of Distance – Weights on features – Optimal Weights
Other Notion of Distance FCM with Euclidian Distance FCM with Adaptive Distance
- Fuzzy logic in pattern recognition
- Pattern recognition
- Fuzzy sets and fuzzy logic theory and applications
- Pattern recognition lab
- Cm bishop pattern recognition and machine learning
- Pattern recognition
- Pattern recognition
- Pattern recognition unibo
- Vyr737
- Pattern recognition
- Pattern recognition clinical reasoning
- Pattern recognition
- Flickr
- Bayesian decision theory in pattern recognition
- Design cycle of pattern recognition
- Isip ece 1111
- Contoh pattern recognition
- Pattern recognition
- Template matching pattern recognition
- Pattern recognition adalah
- Wax pattern in fpd
- Objectives of pattern recognition
- Pattern recognition
- Bayesian parameter estimation in pattern recognition
- Discriminant function in pattern recognition
- Statistical pattern recognition a review
- Bayesian estimation
- Pattern recognition slides
- Fuzzy logic controller
- Um expert
- Samsung wa85
- Contoh kasus logika fuzzy
- Fuzzy logic
- Fuzzy mole
- Pengertian himpunan fuzzy
- Fuzzification
- Fuzzy logic wiki
- "fuzzy math"
- Fuzzy matrix multiplication
- Timothy j ross
- Fuzzy duplicates acl
- Fuzzy clustering tutorial
- Crisp relations
- Modern cell theory