Segmentation via Graph Cuts Balakrishna Thiagarajan Computer Science

Segmentation via Graph Cuts Balakrishna Thiagarajan Computer Science and Engineering State University of New York at Buffalo

Introduction Gestalt approach (From) Normalized Cuts and Image Segmentation

Image Segmentation n Segmentation refers to the process of partitioning a digital image into multiple segments. n The goal of segmentation is to simplify and/or change the representation of an image into something that is more meaningful and easier to analyze. n Similarity (gray level difference), Proximity and continuity are important factors in segmentation.

Graph representation of Image n Set of points of the feature space represented as a weighted, undirected graph, G = (V, E). n The points of the feature space are the nodes of the graph. n Edge between every pair of nodes.

Graph representation of Image n Weight on each edge, w(i, j), is a function of the similarity between the nodes i and j. n Partition the set of vertices into disjoint sets where similarity within the sets is high and across the sets is low. n Therefore, Segmentation is equivalent to Graph partition.

Graph representation of Image Cut: n A cut in a graph is a set of edges whose removal disconnects the graph. n A minimum cut is a cut with a minimum number of edges. It is denoted by S. In the weighted graph, cut(A, B) = ∑ w(u, v) u E A, v E B where A & B are two partitioned disjoint sets

Segmentation by Min Cut method n By Min Cut method, the graph is partitioned into clusters. n Each cluster is considered as an image segment. n Min Cut method uses the HCS (Highly Connected Subgraphs) Algorithm to find the clusters.

Segmentation by Min Cut method n Edge Connectivity is the minimum number of edges whose removal results in a disconnected graph. It is denoted by k(G). n For a graph with vertices n > 1 to be highly connected if its edge-connectivity k(G) > n/2. n A highly connected subgraph (HCS) is an induced subgraph H in G such that H is highly connected.

Segmentation by Min Cut method Example: A B Not HCS! D No. of nodes = 5 C Edge Connectivity = 1 E

Segmentation by Min Cut method Example continued: A B HCS! D No. of nodes = 4 C Edge Connectivity = 3

HCS Algorithm - Min Cut method HCS(G(V, E)) begin (H, H’, C) MINCUT(G) if G is highly connected then return (G) else HCS(H) HCS(H’) end if end

HCS Algorithm - Min Cut method n The procedure MINCUT(G) returns H, H’ and C where C is the minimum cut which separates G into the subgraphs H and H’. n Procedure HCS returns a graph in case it identifies it as a cluster. n Single vertices are not considered clusters and are grouped into singletons set S.

HCS Algorithm - Min Cut method Example

HCS Algorithm - Min Cut method Example Continued

HCS Algorithm - Min Cut method Example Continued Cluster 2 Cluster 1 Cluster 3

HCS Algorithm - Min Cut method n The HCS Algorithm returns clusters which are nothing but image segments. n The problem with HCS Algorithm – Min Cut method is that there is a chance that many singletons may be separated. n To overcome this we use normalized-cut for image segmentation.

HCS Algorithm – Main Problem Example

HCS Algorithm – Main Problem Example – Another possible cut

Max Flow - Min Cut Method n Theorem In graph G, the maximum source-to-sink flow possible is equal to the capacity of the minimum cut in G. (L. R. Foulds, Graph Theory Applications, 1992 Springer-Verlag New York Inc. , 247 -248)

Max Flow - Min Cut Method n Min Cut can be made by using Ford- Fulkerson Algorithm to find the max-flow in the graph. n In the graph temporarily add Source ‘s’ and Sink ‘t’. n Consider the weight associated with each node as a flow capacity. n Starting from zero flow, increase the flow gradually by finding a path from s to t along which more flow can be sent, until a max-flow is achieved.

Max Flow - Min Cut Method

Max Flow - Min Cut Method n After the max-flow is found, the minimum cut is by S = {All vertices reachable from s} T=GS

Max Flow - Min Cut Method Illustration (From) Image Segmentation Using Minimal Graph Cuts by Anders P. Eriksson, Olof Barr and Kalle Astr om

Normalized Cut n Normalized Cut (Ncut) : Ncut(A, B) = cut(A, B) + cut(A, B) assoc(A, V) assoc(B, V) where, assoc(A, V) = ∑ w(u, t) u E A, t E V n The cut is made such that the normalized cut value between the two sets A & B is minimum.

Normalized Cut n Minimum cut criteria favors cutting small sets of isolated nodes in the graph (From) Normalized Cuts and Image Segmentation – Shi & Malik

Normalized Cut n Exact solution to minimizing normalized cut is an NP-complete problem. n However, approximate discrete solutions can be found efficiently. n Normalized cut criterion can be computed efficiently by solving a generalized eigenvalue problem.

Normalized Cut – Physical Interpretation n Think of the weighted graph as a spring mass system. n Graph nodes physical masses n Graph edges springs n Graph edge weight spring stiffness n Total incoming edge weights mass of the node

Normalized Cut – Physical Interpretation n Imagine giving a hard shake to this spring- mass system, forcing the nodes to oscillate in the direction perpendicular to the image plane. n Nodes that have stronger spring connections among them will likely oscillate together. n Eventually, the group will “pop” off from the image plane.

Normalized Cut (From) Normalized Cuts and Image Segmentation - Shi & Malik

Average Cut n Average Cut: Averagecut(A, B) = cut(A, B) |A| + cut(A, B) |B| n The average cut measure is proportional to the total link weight cut, normalized by the sizes of the partitions. n The problem with Average Cut is that one cannot ensure the two partitions computed will have tight within-group similarity.

Comparison n Different types of graph cuts - Min. Cut, Average Cut, and Normalized Cut were analysed by Padmanabhan Soundararajan and Sudeep Sarkar. n Totally 50 images were used. 5 different image classes with 10 images in each class.

Comparison n The 5 classes of images were 1. natural objects in indoor backgrounds 2. natural objects in outdoor backgrounds 3. man-made objects in indoor backgrounds 4. man-made objects in outdoor backgrounds 5. aerial images that contained man-made objects in natural surroundings.

Comparison n The best possible performances on each image for the three different cut measures were not significantly different from each other. n The performance with the three measures is worst for man made objects in indoor surroundings and for aerial images. n The normalized cut based partition took significantly larger time to compute than the average and the mincut.

References 1. J. Shi and J. Malik, “Normalized Cuts and Image Segmentation, ” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 22, no. 8, pp. 888 -905, Aug. 2000. 2. Erez Hartuv and Ron Shamir, “A Clustering Algorithm based on Graph Connectivity”, Mar. 1999. 3. Padmanabhan Soundararajan and Sudeep Sark, “Analysis of Min. Cut, Average Cut, and Normalized Cut Measures”, May 2000. 4. Anders P. Eriksson, Olof Barr and Kalle Astrom, ” Image Segmentation Using Minimal Graph Cuts”.

Thank You!!