GPGC Genetic Programming for Automatic Clustering using a

GPGC: Genetic Programming for Automatic Clustering using a Flexible Non-Hyper-Spherical Graph-Based Approach Andrew Lensen, Dr. Bing Xue, and Prof. Mengjie Zhang Victoria University of Wellington, New Zealand GECCO 2017

Cluster Shape • Many clustering algorithms make assumptions around a cluster’s shape. • k-means: hyper-sphericality. • Others (DBSCAN etc. ) assume clusters are regions of uniform density. 2

Graph-based Clustering • Graph-based clustering techniques exploit the natural graph structure formed by instances (nodes) – neighbouring instances share an edge. • Most techniques use a fixed threshold or n-nearest neighbours to decide which instances share an edge. • Inflexible, dataset-dependent, doesn’t allow varying density… 3

GP: Improving Graph-based Clustering • GP is good at creating function trees which map inputs to an output. • Why not use GP to design similarity functions that decide which instances to connect with an edge? • Learn which function is suitable for a given dataset. • Exploit power of graph representation. • Construct more powerful features than those used in standard similarity functions. 4

Goals (1) Propose a novel GP graph-based clustering algorithm (GPGC) to dynamically produce clusters of a range of shapes. 1. Develop a new GP representation to allow GP to build similarity functions. 2. Propose a new algorithm to use the output of a GP tree to directly perform graph-based clustering. 5

Goals (2) 3. Develop a new fitness function which can produce good quality clusters of varying shape. 4. Evaluate the performance of the proposed algorithm vs baselines across a wide range of clusters with varying shapes. 6

Overall Design 7

GP Program Design (Goal #1) • A similarity function takes two instances and outputs their similarity. • Terminal Set: Features from each instance, random double. • Function set: Normal arithmetic operators for FC, Conditional operators (if/max/min) to allow varying behaviour across dataset. 8

Clustering Alg. (Goal #2) • A GP tree takes two instances as input and outputs a measure of how similar they are. • The c most similar neighbours to an instance are given an edge. • Choose from l closest neighbours. • Each graph gives one cluster. 9

Fitness Function (Goal #3) • GPGC uses a fitness function designed to automatically discover a variety of cluster shapes (i. e. not only hyper-spherical clusters). • Balance three key measures of cluster quality: • Compactness: clusters should be tightly-packed in order to be specific. • Separability: clusters should be far apart to be distinct clusters. • Connectedness: close instances should be in the same cluster. • The number of clusters is not explicitly considered! 10

Fitness Function: Compactness • Find the most isolated instance in a given cluster. • The instance furthest away from its nearest-neighbour. • The red node is the most isolated in the top cluster. • The distance to its nearest-neighbour is the cluster’s sparsity. • More sparse � Less compact. • Minimise sparsity. 11

Fitness Function: Separation • Find the closest neighbouring cluster. • Minimum distance from any instance in the cluster to any other instance not in the same cluster. • The red connection shows the minimum separation between these two clusters. • Maximise separation. 12

Fitness Function: Connectedness • Find how well each instance lies in the same cluster as its nearest n neighbours. • For each of the n neighbours in the same cluster, find the inverse distance to that neighbour, and add to the total connectedness score. • Close neighbours are weighted higher. • Maximise connectedness. 13

Fitness Function: Combining the Measures • Find the ratio of cluster sparsity: separation (CS: S) for each cluster. • Sparsity and separation are conflicting. • Find the mean connectedness across all instances. • Maximise the overall fitness. 14

Experiments (Goal #4) • 15

Results (1) • Outperformed all baselines on hardest synthetic datasets, despite not pre-defining K. • Finds K much more accurately than OPTICS. • F-Measure compares to "gold standard" provided by dataset authors. • Much better F-Measure! 16

Results (2) • Best method in general across all handcrafted datasets. • GPGC shown to successfully adapt to range of cluster shapes. 17

Example of an Evolved Similarity Function 18

Final Remarks • First work using GP to evolve similarity functions for clustering. • Builds powerful high-level features within the GP tree. • Shown to be flexible across a range of datasets and dimensionalities. • Further room for improvement – EMO, multi-tree, other functions. 19

Thank you! 20