Subspace Clustering Ali Sekmen and Ghan S Bhatt

Subspace Clustering Ali Sekmen and Ghan S. Bhatt Computer Science and Mathematical Sciences College of Engineering Tennessee State University 1 st Annual Workshop on Data Sciences

Part I Some Linear Algebra Spectral Analysis Singular Value Decomposition Presenter Dr. Ghan S. Bhatt

Definitions

Range and Null Spaces

Definitions

Eigenvalues - Eigenvectors

Symmetric Matrices

Projection on a Vector

Projection on a Subspace

Singular Value Decomposition

Important Lemma

Recall – Linear Mapping

General Matrix Norms

An Intuitive Matrix Norm This satisfies the general matrix norm properties Although it is useful, it is not suitable for large set of problems and we need another definition of matrix norms

Induced Matrix Norms

Matrix p-Norm

More on Matrix Norms

Part II Subspace Segmentation Problem Motion Segmentation Principal Component Analysis Dimensionality Reduction Spectral Clustering Presenter Dr. Ali Sekmen

Subspace Segmentation In many engineering and mathematics applications, data lives in a union of low dimensional subspaces Motion segmentation Facial images of a person with the same expression under different illumination approximately lie on the same subspace

Face Recognition

Problem Statement

What are we trying to solve?

Example – Motion Segmentation

Motion Segmentation Motion segmentation problem can simply be defined as identifying independently moving rigid objects in a video.

Motion Segmentation We will show that all trajectories lie in a 4 -dim subspace of

Motion Segmentation Z Z p z x Y Y X X y

Motion Segmentation Z p z x Y X y

Motion Segmentation

Motion Segmentation Y X

Motion Segmentation

Principal Component Analysis The goal is to reduce dimension of dataset with minimal loss of information We project a feature space onto a smaller subspace that represent data well Search for a subspace which maximizes the variance of projected points This is equivalent to linear least square fitting Minimize the sum of squared distances between points and subspace We find directions (components) that maximizes variance in dataset PCA can be done by Eigenvalue decomposition of a data covariance matrix Or SVD of a data matrix

Least Square Approximation

Principal Component Analysis

PCA with SVD Coordinates w. r. t. new basis

Principal Component Analysis inch cm

Principal Component Analysis inch cm 10 28 12 19 15 40 20 47 23 56 26 69

Solution with SVD

PCA: Pre-Processing 80 inch cm 10 28 12 19 15 40 20 47 20 23 56 10 26 69 70 60 Inch 50 40 30 0 0 Zero Mean, Unit Variance 30 20 0 -10 0 -20 -30 cm 5 10 inch 10 -5 20 cm Zero Mean -10 10 -1. 5 -1 2 1. 5 1 0. 5 0 -0. 5 0 -1 -1. 5 cm 0. 5 1 1. 5 30

PCA: Optimization

PCA: Reduce Dimensionality

General PCA

Spectral Clustering A very powerful clustering algorithm Easy to implement Outperforms traditional clustering algorithms Example: k-means It is not easy to understand why it works Given a set of data points and some similarity measure between all pairs of data points, we divide data into groups Points in the same group are similar Points in different groups are dissimilar