A Clustering Framework for Unbalanced Partitioning and Outlier

A Clustering Framework for Unbalanced Partitioning and Outlier Filtering on High Dimensional Datasets 1 Turgay Tugay Bilgin and A. Yilmaz Camurcu 1 Department 2 of Computer Engineering, Maltepe University ttbilgin@maltepe. edu. tr 2 Department of Computer and Control Education, Marmara University camurcu@marmara. edu. tr

Outline n n n Introduction Relationship based clustering approach / framework Visualization using CLUSION (CLUSter visualizat. ION) Problems of the Framework Graclus partitioning system Our Proposed Framework Using Graclus: to create Micro-partition Space ¨ Outlier filtering on micro-partition space ¨ Using Graclus: to cluster ΔP Space ¨ Visualization of the results using CLUSION graphs ¨ n n Experiments Results

Introduction n n Mining high dimensional datasets are an important problem of Data Mining community Well-known problem: curse of dimensionality Graph based methods such as METIS and CHACO perform best on high dimensional space However, these methods have 2 major problems: ¨ can not perform outlier filtering ¨ Force clusters to be balanced

Relationship based Clustering Approach n n n Strehl A. and Ghosh J. proposed a better approach for mining high dimensional datasets [1]. They focus on similarity space rather than Feature space. A graph partitioning tool METIS is used to perform balanced clustering (OPOSSUM) They also provide a customized matrix visualization tool called CLUSION is fast, simple and it can operate on very high dimensional datasets.

Relationship based Clustering Framework Feature Extraction Data Sources Similarity OPOSSUM computation (Optimal partitioning of Similarity space using Metis) Feature Space Similarity Space Cluster Labels

Visualization using CLUSION n Clusters appear as symmetrical dark squares across the main diagonal Similarity Matrix Cluster Visualization CLUSION S is permuted with a nxn permutation matrix P λ index

Problems of the Framework n Produces balanced clusters only: It forces clusters to be of equal size. In some datasests this could be important, because it avoids trivial clusterings. But in most cases, can cause undesired results. n No outlier filtering : Outliers can reduce the quality and the validity of the clusters depending on the resolution and distribution of the dataset.

Graclus* partitioning system n n Graclus* is a fast kernel based multilevel algorithm which involves coarsening, initial partitioning and refinement phases. Unlike METIS, ¨ it does not force clusters to be nearly, equal size. ¨ Uses weighted form of kernel based k-means approach ¨ kernel k-means approach is extremely fast and gives high-quality partitions (*) * Dhillon, I. , Guan, Y. , Kulis, B. : A Fast Kernel-based Multilevel Algorithm for Graph Clustering, Proceedings of The 11 th ACM SIGKDD, Chicago, IL, August 21 - 24, (2005).

Our Proposed Framework n Three major improvements: ¨ An intermediate space (P): We call it “micro-partition space”. Graclus is used for creating unbalanced micro-partitions. ¨ Outlier filtering on the P space (results ΔP) : Graclus creates micro-partitions of different sizes. The singletons on the P space means the points that have not enough neighbors can be filtered or marked as outliers. ¨ Using Graclus for clustering ΔP space: Graclus has two important roles on our framework. The first role is creating the micro-partition space. The second role is unbalanced clustering of the filtered space ΔP which is denoted by Φ.

Our Proposed Framework Micro-partition space (P) Contains unbalanced tiny partitions creating micro-partitions (using Graclus) outlier filtering and (re)clustering (using Graclus) results ΔP Space ΔP

Using Graclus: to create Micro-partition Space n n Use Graclus in Similarity Space to create tiny partitions (micro-partitions) Notation: ¨n = number of samples, ¨ k = number of micro-partitions on P space n n relation between k and p should be: [1] Micro-partitions can contain up to 4 objects, therefore: [2]

Outlier filtering on micro-partition space n illustration

Outlier filtering on micro-partition space n Outliers in P space (Po) is: where To is Outlier threshold value n Then, ΔP space is:

Using Graclus: to cluster ΔP Space n Graclus needs the number of partitions k. ¨ In formula [1], k refers to the number of micro partitions. ¨ Here k refers to the number of clusters we desire. n n we denote the former one by k 1 and the latter one by k 2. Graclus performs clustering on the ΔP space and produces λ index which is defined as:

Visualization of the results using CLUSION graphs n CLUSION looks at the λ, ¨ reorders the ΔP space so that points with same cluster label are contiguous ¨ then visualize the resulting permuted ΔP′ n there are two λ indices produced during clustering process. ¨ λ 1 is created while forming micro-partitions ¨ λ 2 is created while clustering ΔP space n We use λ 2 for CLUSION, the first one is only used forming micro-partitions

Experiments: Datasets n We evaluated our proposed framework on two different real world datasets. 1. 2. 9636 terms from 2225 complete news articles from the BBC News web site. (2225 dimensional dataset, 5 natural clusters) Collection of news articles from Turkish newspaper “Milliyet”. Contains 6223 terms in Turkish from 1455 news articles. (1455 dimensional dataset, 3 natural clusters)

Experiments: Evaluated Frameworks n OPOSSUM: Strehl & Ghosh’s METIS based original framework n S&G(Graclus): We replaced METIS by Graclus on Strehl & Ghosh’s framework for testing the quality of the clusters produced by Graclus algorithm. n P space+Graclus: Our proposed framework.