Parallel kmeans for Multiple SharedMemory Architectures Patrick Mackey

Parallel k-means++ for Multiple Shared-Memory Architectures Patrick Mackey Pacific Northwest National Laboratory Robert R. Lewis Washington State University ICPP 2016

This Paper �Describes the approaches for parallelizing k-means++ on three distinct hardware architectures. ◦ Open. MP: shared-memory multiple multicore processors. ◦ Cray XMT: massively multi-threaded architecture. ◦ high performance GPU.

k-means++ �A method that improves the quality of k-means clustering. ◦ Selecting a set of initial seeds that would on average provide better clustering than random selection. �Uses a probabilistic approach for selecting seeds. ◦ The probability is based on the distance of a data point from all previously selected seeds.

Pseudocode of Serial kmeans++

Pseudocode of Weighted_Rand_Index

Parallel k-means++ �Parallelizing the probabilistic selection is challenging. ◦ A dependence exists between each iteration in the while loop. ◦ Simple loop parallelism will not work. �Each thread is given a partition of data points, and make its own seed selection from its subset of weighted probabilities using the same basic algorithm. ◦ Produces a list of potential seed choices and their probabilities.

Parallel k-means++(Cont. ) �Performs another weighted probability selection on the list and decides the final chosen seed.

Proof of Correctness �Let x ∈ X be an arbitrary vector. ◦ ppar(x): probability of selecting x in the parallel algorithm. ◦ p(x): the true probability of selecting x ∈ X by weighted probability. �Theorem: Ppar(x) = P(x)

Proof �Let X’ be the set of vectors assigned to a thread containing the vector x. �Since p(X’|x) = 1. 0, ppar(x) = p(x).

k-means++ for Open. MP

K-means++ for Massively Multithreaded Architecture

Weighted_Rand() on Massively Multithreaded Architecture

K-means++ on GPU �Implemented with Nvidia’s Thrust library for C++.

Prob_Reduce()

Scaling Performance Results

Platform Performance Comparison �Conduct a series of experiments with varying size of n, m, and k on different platforms. �n: the number of data points. �m: the dimensional size of the data. �k: the number of clusters. �Platforms: �GPU (Nvidia Tesla C 1060) �Open. MP (8 cores) �Open. MP (4 cores) �Cray XMT (128 processors) �Cray XMT (64 processors) �Cray XMT (32 processors)

Linear Regression �Linear regression model: �Accuracy ◦ Root-mean-square-error(RMSE)

Comparison Visualization

Summaries �GPU dominated when the dimensionality of the data was small. �Cray XMT excelled when the dimensionality of the data was high or the number of data points became exceedingly large. �Shared-memory multiple multi-core processors outperform the others when the data was small, or the number of clusters desired was small.

Summaries(Cont. ) �“Using a number of threads equal to the number of processors will not always be the most efficient. ” ◦ A program could be implemented that selects a more optimal number of threads to run the algorithm with, with the added benefit of making more resources available for other processes.