Outline Perceptual organization grouping and segmentation Introduction Region

Outline • Perceptual organization, grouping, and segmentation – Introduction – Region growing – Split-and-merge 12/4/2020 File: week 13 -f. ppt Visual Perception Modeling 1

Introduction • Segmentation – Roughly speaking, segmentation is to partition the images into meaningful parts that are relatively homogenous in certain sense 12/4/2020 Visual Perception Modeling 2

Introduction – cont. • Why is segmentation important – Classification algorithms in general assume that the features are extracted only from the objects/regions that we are interested in 12/4/2020 Visual Perception Modeling 3

Introduction – cont. • Why is segmentation difficult – The first difficulty is a representation issue • There are many different kinds of objects, textures • Is there a representation that will apply to all the images 12/4/2020 Visual Perception Modeling 4

Introduction - continued • How can we characterize all these images perceptually? 12/4/2020 Visual Perception Modeling 5

Introduction – cont. • Why is segmentation difficult – The first difficulty is a representation issue • There are many different kinds of objects, textures • Is there a representation that will apply to all the images – The second difficulty is to identify first what types of objects are present in the given input image – The third difficulty is to localize the boundaries between regions 12/4/2020 Visual Perception Modeling 6

Introduction – cont. • But we can do the task effortless – How have we done so? 12/4/2020 Visual Perception Modeling 7

Introduction – cont. • Perceptual organization – Try to understand the principles behind perception by observing and building models for perceptual phenomena 12/4/2020 Visual Perception Modeling 8

Introduction – cont. 12/4/2020 Visual Perception Modeling 9

Introduction – cont. • Gestalt grouping principles – Proximity • Objects that are close to each other tend to be grouped together – Similarity • Objects that are more similar to one another tend to be grouped together – Closure • Objects that form closed units tend to be grouped together 12/4/2020 Visual Perception Modeling 10

Introduction – cont. • Gestalt grouping principles – continued – Good continuation – Common fate – Figure and ground – Subjective contour 12/4/2020 Visual Perception Modeling 11

Introduction – cont. • Problems with Gestalt principles – They are NOT computational models – In addition, those factors interfere each other in a given image 12/4/2020 Visual Perception Modeling 12

Introduction – cont. • Computational models/implementations – There are generally two kinds of computational models/implementations for segmentation • Based on homogeneity measure to group pixels with similar attributes together – Region growing/split-and-merge • Based on discontinuity of attributes to detect boundaries/contours of regions – Active contours 12/4/2020 Visual Perception Modeling 13

Region Growing • Region growing – Is a set of algorithms to group pixels with similar attributes together – The basic idea is to grow from a seed pixel • At a labeled pixel, check its neighbors – If the attributes of its neighbor is similar to the attributes of the labeled pixel, label the neighbor • Repeat until there is no pixel that can be labeled 12/4/2020 Visual Perception Modeling 14

Region Growing – cont. • A simple case – The attribute of a pixel is its pixel value – The similarity is given by the difference between the two pixel values • If the difference is smaller than a threshold, we say they are similar • Otherwise they are not 12/4/2020 Visual Perception Modeling 15

Region Growing – cont. • Recursive implementation – Given a seed point, call the following recursive function void Region. Growing(IMAGE animage, LABEL labelmap, int x, int y, int label) if (labelmap[x][y] != 0) return; labelmap[x][y] = label; for each neighbor nx, ny of pixel x, y if labelmap[nx][ny]==0 if diff(animage[x][y]-animage[nx][ny]) < threhold, Region. Growing(animage, labelmap, nx, ny, label) end if end for return 12/4/2020 Visual Perception Modeling 16

Region Growing – cont. • Efficient implementation – Based on scan-line algorithm in graphics – Each time we label a line instead of a pixel – This procedure is much more efficient than the recursive version 12/4/2020 Visual Perception Modeling 17

Split and Merge • There is also a different implementation to partition an input image into homogenous regions – Start with the entire image as one region – Then split a region into sub-regions if the variance is larger than a threshold and merge neighboring regions if they are similar 12/4/2020 Visual Perception Modeling 18

Split and Merge – cont. 1. Start with the entire image as a single region 2. Pick a region R. If H(R) is false, then split the region into four sub-regions 3. Consider any two or more neighboring subregions, R 1, R 2, . . . , Rn, in the image. If H(R 1 U R 2 U. . . U Rn ) is true, merge the n regions into a single region. 4. Repeat these steps until no further splits or merges take place 12/4/2020 Visual Perception Modeling 19

Split and Merge – cont. 12/4/2020 Visual Perception Modeling 20

Split and Merge – cont. 12/4/2020 Visual Perception Modeling 21

Split and Merge – cont. 12/4/2020 Visual Perception Modeling 22