Fuzzy Clustering Algorithms SSIE 617 2 nd Presentation

Fuzzy Clustering Algorithms SSIE 617 2 nd Presentation Benjamin James Bush 05/02/2012

What is Clustering?

Crisp & Fuzzy Clustering Each point belongs to exactly one cluster. CRISP C-Means Clustering Cluster membership is a matter of degree. FUZZY Fuzzy C-Means Clustering (FCM) Fuzzy Min-Max Clustering Neural Network

C-Means Clustering Fixed number of clusters. One centroid per cluster. Each data point belongs to the cluster corresponding to the closest centroid. Figure Animation by Andrey A. Shabalin, Ph. D.

C-Means Clustering # of clusters distance between data point and cluster center cost function cost of the ith cluster data points belonging to the ith group

C-Means Clustering pick c centroids at random assign each data point to the cluster corresponding to the nearest centroid. move each centroid to the mean value of its cluster’s data points. Animation by Andrey A. Shabalin, Ph. D.

Fuzzy C-Means Clustering (FCM) Fixed number of clusters. One centroid per cluster. Clusters are fuzzy sets. Membership degree of a point can be any number between 0 and 1. Sum of all degrees for a point must add up to 1. Figure Animation by Matteo Matteucci, Ph. D.

Fuzzy C-Means Clustering (FCM) C-Means Fuzzy C-Means (FCM) summing over all data points fuzziness exponent membership degree

Fuzzy C-Means Clustering pick c centroids at random assign membership degrees according to: move each centroid to the following position: Note: formulas are result of the method of Lagrange multipliers as applied to aforementioned cost function. Proof left as exercise.

Crisp & Fuzzy Clustering Each point belongs to exactly one cluster. CRISP C-Means Clustering Cluster membership is a matter of degree. FUZZY Fuzzy C-Means Clustering (FCM) Fuzzy Min-Max Clustering Neural Network

How Many Clusters? ?

Fuzzy Min-Max Clustering NN Variable number of clusters. Each cluster has a Hyperbox Fuzzy Set. Degrees inside the box are 1. Degrees outside the hyperbox decrease linearly with distance from the box. Total degrees for a point need not add up to 1. Boxes may not overlap.

Hyperbox Fuzzy Sets Start Mathematica. . .

Hyperbox Fuzzy Sets Easy to implement as ANNs. Potential to take advantage of massive parallel processing.

Initialize population of 250 randomly chosen individuals, each with a random # of boxes. For each box, choose min point and max point at random. Create an child individual from each member of the population. When creating a child, add a Gaussean r. v. to each component of the min and max point, and change the # of boxes with probability 0. 5. Evaluate the fitness of each individual based on its Minimum Description Length (MDL) Penalty for # of clusters. goodness of fit Eliminate half of the individuals via round-robin tournament competition.

Applications

Applications of Fuzzy C-Means

Applications of Fuzzy C-Means

Applications of Min-Max Clustering NN

Applications of Min-Max Clustering NN

Bibliography Ch. 15 Ch. 1 Videolectures. net: MDL Tutorial http: //videolectures. net/icml 08_grunwald_mld/