PixelBased Processing ECE 847 Digital Image Processing Stan

Pixel-Based Processing ECE 847: Digital Image Processing Stan Birchfield Clemson University

Challenge ? original image detect objects, classify fruit, find banana stem Steps: 1. binarize / threshold 2. clean up binary image 3. find regions corresponding to foreground objects 4. compute properties of regions 5. classify regions based on properties S. Birchfield, Clemson Univ. , ECE 847, http: //www. ces. clemson. edu/~stb/ece 847

Outline • Pixel-based image transformations (thresholding, …) • Removing noise from binary image (morphological operators) • Labeling regions (floodfill, connected components) • Computing distance in an image (chamfer distance, …) • Region properties (area and boundary properties) S. Birchfield, Clemson Univ. , ECE 847, http: //www. ces. clemson. edu/~stb/ece 847

What is a digital image? 75 81 83 96 94 62 74 76 87 100 74 76 80 89 93 53 67 86 90 105 60 77 90 99 115 S. Birchfield, Clemson Univ. , ECE 847, http: //www. ces. clemson. edu/~stb/ece 847

Accessing pixels in an image C / C++ • Method 1: 1 D array val = p[ y * width + x]; • Method 2: 2 D array val = p[y][x]; • Method 3: pointers val = *p++; Matlab • Method 1: 2 D array val = im(y, x); • Method 2: parallelize im = im + constant; Timing Test (640 x 480 image, get/set each pixel using 2. 8 GHz P 4) 6. 6 ms 5. 6 ms 0. 5 ms (even faster with SIMD operations) 21. 4 ms 2. 5 ms S. Birchfield, Clemson Univ. , ECE 847, http: //www. ces. clemson. edu/~stb/ece 847

Outline • Pixel-based image transformations (thresholding, …) • Removing noise from binary image (morphological operators) • Labeling regions (floodfill, connected components) • Computing distance in an image (chamfer distance, …) • Region properties (area and boundary properties) S. Birchfield, Clemson Univ. , ECE 847, http: //www. ces. clemson. edu/~stb/ece 847

Types of image transformations • Graylevel transforms I’(x, y) f( I(x, y) ) (arithmetic, logical, thresholding, histogram equalization, …) • Geometric transforms I’(x, y) f( I(x’, y’) ) (flip, flop, rotate, scale, …) • Area-based transforms I’(x, y) f( I(x, y), I(x+1, y+1), … ) (morphological operators, convolution) • Global transforms I’(x, y) f( I(x’, y’), x’, y’ ) (Fourier transform, wavelet transform) A S. Birchfield, Clemson Univ. , ECE 847, http: //www. ces. clemson. edu/~stb/ece 847

Graylevel transforms identity inverse contrast enhancement threshold S. Birchfield, Clemson Univ. , ECE 847, http: //www. ces. clemson. edu/~stb/ece 847

Geometric transforms S. Birchfield, Clemson Univ. , ECE 847, http: //www. ces. clemson. edu/~stb/ece 847

Histograms image histogram Throws away all spatial information! S. Birchfield, Clemson Univ. , ECE 847, http: //www. ces. clemson. edu/~stb/ece 847

Histogram equalization S. Birchfield, Clemson Univ. , ECE 847, http: //www. ces. clemson. edu/~stb/ece 847

Running sum Step 1: Initialize 1 D array int hist[256]. . . Step 2: Compute running sum[0] = 0 for g = 1 to 255, sum[g] = sum[g-1] + hist[g] Example: [ 6 3 9 7 4 2] [6 9 18 25 29 31] S. Birchfield, Clemson Univ. , ECE 847, http: //www. ces. clemson. edu/~stb/ece 847

Histogram equalization pdf Steps: 1. Compute normalized histogram (PDF) 2. 3. Compute cumulative histogram (CDF) cum[0] = norm[0] for g = 1 to 255, cum[g] = cum[g-1] + norm[g] Transform for x, I(x) = 255*cum[ I(x) ] running sum cdf new gray level int hist[256] = 0… 0 float norm[256] for x, hist[ I(x) ]++ for g = 0 to 255, norm[g] = hist[g] / npixels equal area equal spacing old gray level S. Birchfield, Clemson Univ. , ECE 847, http: //www. ces. clemson. edu/~stb/ece 847

Histogram equalization * desired PDF p’ equal area a’ a’+d a new gray level k’ CDF d{ d{ a’ equal area new gray level k’ By definition, q equal spacing old gray level k where p is the PDF and q is the CDF S. Birchfield, Clemson Univ. , ECE 847, http: //www. ces. clemson. edu/~stb/ece 847

Histogram equalization * desired PDF p PDF equal area p’ 1/(L-1) a scaled CDF new gray level k’ L-1 d{ b L-1 old gray level k a’ a’+d L-1 new gray level k’ q·(L-1) a’ L-1 old gray level k S. Birchfield, Clemson Univ. , ECE 847, http: //www. ces. clemson. edu/~stb/ece 847

Histogram equalization * desired PDF p PDF equal area p’ 1/(L-1) a 1 b 1 a 2 b 2 L-1 new gray level k’ scaled CDF d. L-1{ d{ a 2 ’ a 1’+d a 2’+d L-1 new gray level k’ q·(L-1) a 1 ’ L-1 old gray level k S. Birchfield, Clemson Univ. , ECE 847, http: //www. ces. clemson. edu/~stb/ece 847

Integral image • Integral image is a 2 D running sum • S(x, y) = SS I(x, y) • To compute, S(x, y) = I(x, y) - S(x-1, y-1) + S(x-1, y) + S(x, y-1) • To use, V(l, t, r, b) = S(l, t) + S(r, b) - S(l, b) - S(r, t) Returns sum of values inside rectangle • Note: Sum of values in any rectangle can be computed in constant time! S. Birchfield, Clemson Univ. , ECE 847, http: //www. ces. clemson. edu/~stb/ece 847

Thresholding Valley separates light from dark How to find it? S. Birchfield, Clemson Univ. , ECE 847, http: //www. ces. clemson. edu/~stb/ece 847

A simple thresholding algorithm Repeat until convergence: T ½ (m 1 + m 2) T. Ridler and S. Calvard, Picture Thresholding Using an Iterative Selection Method, IEEE Transactions on Systems, Man and Cybernetics, 8(8): 630 --632, August 1978. S. Birchfield, Clemson Univ. , ECE 847, http: //www. ces. clemson. edu/~stb/ece 847

Ridler-Calvard algorithm S. Birchfield, Clemson Univ. , ECE 847, http: //www. ces. clemson. edu/~stb/ece 847

Otsu’s method • Threshold t splits image into two groups • Calculate within-group variance of each group • Search over all t to minimize total withingroup variance; moment calculations make search efficient percentage of pixels in first group variance of pixels in first group S. Birchfield, Clemson Univ. , ECE 847, http: //www. ces. clemson. edu/~stb/ece 847

Otsu’s method S. Birchfield, Clemson Univ. , ECE 847, http: //www. ces. clemson. edu/~stb/ece 847

Local Entropy Thresholding (LET) * • Compute co-occurrence matrix • Threshold s divides matrix into four quadrants • Pick s that maximizes HBB(s)+HFF(s) • Uses spatial information S. Birchfield, Clemson Univ. , ECE 847, http: //www. ces. clemson. edu/~stb/ece 847

Otsu vs. LET * image histogram Otsu LET S. Birchfield, Clemson Univ. , ECE 847, http: //www. ces. clemson. edu/~stb/ece 847

Adaptive thresholding background image histogram from Gonzalez and Woods, p. 597 S. Birchfield, Clemson Univ. , ECE 847, http: //www. ces. clemson. edu/~stb/ece 847

Double thresholding (hysteresis) graylevel threshold too high: misses part of object 2 object 1 threshold too low: captures noise Algorithm: 1. Threshold with high value 2. Threshold with low value, retaining only the pixels that are contiguous with existing ones pixel S. Birchfield, Clemson Univ. , ECE 847, http: //www. ces. clemson. edu/~stb/ece 847

Double thresholding example low threshold 1. Use high threshold to get seed pixel high threshold 3. Repeat 2. Floodfill using seed pixel S. Birchfield, Clemson Univ. , ECE 847, http: //www. ces. clemson. edu/~stb/ece 847

Double thresholding example image low threshold retains some background combined using floodfill on low threshold with seeds from high threshold S. Birchfield, Clemson Univ. , ECE 847, http: //www. ces. clemson. edu/~stb/ece 847 high threshold removes some foreground

Double thresholding We will explain Floodfill later S. Birchfield, Clemson Univ. , ECE 847, http: //www. ces. clemson. edu/~stb/ece 847

Outline • Pixel-based image transformations (thresholding, …) • Removing noise from binary image (morphological operators) • Labeling regions (floodfill, connected components) • Computing distance in an image (chamfer distance, …) • Region properties (area and boundary properties) S. Birchfield, Clemson Univ. , ECE 847, http: //www. ces. clemson. edu/~stb/ece 847

Binary image as a set = S. Birchfield, Clemson Univ. , ECE 847, http: //www. ces. clemson. edu/~stb/ece 847

Set operations Fundamental operators Other operators De. Morgan’s Laws S. Birchfield, Clemson Univ. , ECE 847, http: //www. ces. clemson. edu/~stb/ece 847

Minkowski operators Minkowski addition: Minkowski subtraction: S. Birchfield, Clemson Univ. , ECE 847, http: //www. ces. clemson. edu/~stb/ece 847

Minkowski addition Add each element of B to each element of A 1 S. Birchfield, Clemson Univ. , ECE 847, http: //www. ces. clemson. edu/~stb/ece 847

Minkowski subtraction Intersection of translating A 2 to every point in B Two examples: S. Birchfield, Clemson Univ. , ECE 847, http: //www. ces. clemson. edu/~stb/ece 847

Minkowski addition: “Center -out” approach (“center-out” is more intuitive) S. Birchfield, Clemson Univ. , ECE 847, http: //www. ces. clemson. edu/~stb/ece 847

Minkowski subtraction: “Center-out” approach (“center-out” is more intuitive) S. Birchfield, Clemson Univ. , ECE 847, http: //www. ces. clemson. edu/~stb/ece 847

Morphological operations • morphology – study of form or shape • Mathematical morphology – grew out of research at Fontainebleau in 1964 • We will deal only with binary images • Two fundamental operations: Dilation and Erosion B is the structuring element (SE) (same as Mink. add. ) (same as Mink. sub. after reflection) S. Birchfield, Clemson Univ. , ECE 847, http: //www. ces. clemson. edu/~stb/ece 847

Dilation • Algorithm (center-out): – Flip B horizontally and vertically (but often B is symmetric, so nothing is changed) – Move B around – Anywhere they intersect, turn center pixel on • Looks like nonlinear convolution S. Birchfield, Clemson Univ. , ECE 847, http: //www. ces. clemson. edu/~stb/ece 847

Erosion • Algorithm (center-out): – Move B around – If there is not complete overlap, turn center pixel off Duality b/w dilation and erosion: S. Birchfield, Clemson Univ. , ECE 847, http: //www. ces. clemson. edu/~stb/ece 847

Common structuring elements • Usually SE is either B 4 or B 8 • Greatly simplifies algorithm (and erosion is same as Min. sub. ) S. Birchfield, Clemson Univ. , ECE 847, http: //www. ces. clemson. edu/~stb/ece 847

Dilation example Let B be 3 x 3 with all 1 s. for y for x if ( in(x-1, y) || in(x-1, y-1) || in(x-1, y+1). . . || in(x+1, y+1) ) out(x, y) = 1 else out(x, y) = 0 B A S. Birchfield, Clemson Univ. , ECE 847, http: //www. ces. clemson. edu/~stb/ece 847

Erosion example Let B be 3 x 3 with all 1 s. for y for x if ( in(x-1, y) && in(x-1, y-1) && in(x-1, y+1). . . && in(x+1, y+1) ) out(x, y) = 1 else out(x, y) = 0 B A S. Birchfield, Clemson Univ. , ECE 847, http: //www. ces. clemson. edu/~stb/ece 847

Why does dilation require flipping the structuring element? Simple example: A Without flipping B With flipping Result is flipped! Fixed! S. Birchfield, Clemson Univ. , ECE 847, http: //www. ces. clemson. edu/~stb/ece 847

Closing and Opening • Dilation fills gaps. Erosion removes noise • Combine them: – First dilate, then erode (closing) – First erode, then dilate (opening) • Closing/opening retain object size • Repeated applications do nothing • Duality between closing and opening S. Birchfield, Clemson Univ. , ECE 847, http: //www. ces. clemson. edu/~stb/ece 847

Closing and Opening Closing: dilate, then erode Opening: erode, then dilate S. Birchfield, Clemson Univ. , ECE 847, http: //www. ces. clemson. edu/~stb/ece 847

Morphology Example thresholded difference open erode dilate close open then close S. Birchfield, Clemson Univ. , ECE 847, http: //www. ces. clemson. edu/~stb/ece 847

Accessing out-of-bounds pixels • If kernel is near boundary of image, some pixels of kernel will be out of bounds (OOB) • What to do? – Do not use OOB pixels (change kernel size near boundary) – Do not compute output for pixels near boundary – Extend image function: • • • zero pad – OOB pixels have value of zero replicate – OOB pixels have value of closest pixel reflect – pixel values are extended by mirror reflection at border wrap – pixel values are extended by repeating extrapolate – pixel values are extended by extrapolating function near boundary • No solution is best • Problem also occurs with convolution S. Birchfield, Clemson Univ. , ECE 847, http: //www. ces. clemson. edu/~stb/ece 847

Outline • Pixel-based image transformations (thresholding, …) • Removing noise from binary image (morphological operators) • Labeling regions (floodfill, connected components) • Computing distance in an image (chamfer distance, …) • Region properties (area and boundary properties) S. Birchfield, Clemson Univ. , ECE 847, http: //www. ces. clemson. edu/~stb/ece 847

Binary images • Each pixel is ON or OFF • We will say that foreground pixels are ON, background pixels are OFF • In computer, each pixel has value of 0 or 1. • We adopt convention that 0=OFF, 1=ON. • Usually 0=black and 1=white, but sometimes display is reversed S. Birchfield, Clemson Univ. , ECE 847, http: //www. ces. clemson. edu/~stb/ece 847

Neighbors 4 -neighbors 8 -neighbors diagonal neighbors N 4 N 8 ND S. Birchfield, Clemson Univ. , ECE 847, http: //www. ces. clemson. edu/~stb/ece 847

Adjacency • 4 -adjacency: p q • 8 -adjacency: p • m-adjacency: q or and Example: 48 m S. Birchfield, Clemson Univ. , ECE 847, http: //www. ces. clemson. edu/~stb/ece 847

Adjacency Use different connectedness foreground and background to avoid inconsistency foreground: D 8 background: D 4 dumbbell S. Birchfield, Clemson Univ. , ECE 847, http: //www. ces. clemson. edu/~stb/ece 847

Connectivity, Regions and Boundaries • path from p to q is sequence of adjacent pixels • p and q are connected in subset S if path exists between them containing only pixels in S • connected component of p is set of pixels in S that are connected to p • S is a region if it is a connected set • boundary of S is the set of pixel in S that have one or more neighbors not in S S. Birchfield, Clemson Univ. , ECE 847, http: //www. ces. clemson. edu/~stb/ece 847

Boundary definitions binary image outer region boundary inner region boundary outer hole boundary inner hole boundary outer complete boundary inner complete boundary S. Birchfield, Clemson Univ. , ECE 847, http: //www. ces. clemson. edu/~stb/ece 847

Floodfill • Fill a region with a new color – Start with a seed point – All pixels are colored if they have the same color as the seed and are connected to the seed via pixels that have the same color as the seed pixel S. Birchfield, Clemson Univ. , ECE 847, http: //www. ces. clemson. edu/~stb/ece 847

Floodfill * Repeated applications of dilation can fill a region image A. . . but terribly inefficient structuring element B complement Ac S. Birchfield, Clemson Univ. , ECE 847, http: //www. ces. clemson. edu/~stb/ece 847

Floodfill Efficient implementation uses a stack called the frontier: Function image = Floodfill(seed_point, image, new_color) old_color = image(seed_point) if (old_color == new_color) return frontier. Push(seed_point) image(seed_point) = newcolor while (NOT frontier. Is. Empty()) q = frontier. Pop(); for each neighbor r of q, if image(r) == old_color, frontier. Push(r) image(r) = new_color return image S. Birchfield, Clemson Univ. , ECE 847, http: //www. ces. clemson. edu/~stb/ece 847

Floodfill S. Birchfield, Clemson Univ. , ECE 847, http: //www. ces. clemson. edu/~stb/ece 847

Floodfill This code performs the same logic, but does not change the pixels of the input image S. Birchfield, Clemson Univ. , ECE 847, http: //www. ces. clemson. edu/~stb/ece 847

Revisit: Double thresholding S. Birchfield, Clemson Univ. , ECE 847, http: //www. ces. clemson. edu/~stb/ece 847

Connected components • Recall that connected component of p is set of pixels in S that are connected to p • Can solve the problem by repeated applications of Floodfill – assigning a new label to each new region • Slightly more efficient implementation involves two passes through the image (union-find) S. Birchfield, Clemson Univ. , ECE 847, http: //www. ces. clemson. edu/~stb/ece 847

Connected components using floodfill S. Birchfield, Clemson Univ. , ECE 847, http: //www. ces. clemson. edu/~stb/ece 847

Classical union-find connected component labeling algorithm scan left-to-right, top-to-bottom U L P I(P) == I(U) ? I(P) == I(L) ? 4 -neighbor algorithm: 4 -neighbor mask: YES NO YES C(P)=C(U) and set Equiv(U, L) C(P)=C(L) NO C(P)=C(U) C(P)=n++ 8 -neighbor mask: S. Birchfield, Clemson Univ. , ECE 847, http: //www. ces. clemson. edu/~stb/ece 847

Classical connected components: first pass 1 0 0 1 0 1 1 1 0 1 2 3 3 1 1 1 0 0 0 1 2 3 5 image 0 1 2 3 5 0 1 2 2 2 0 0 0 4 4 components 0 1 2 3 4 equivalence table 5 S. Birchfield, Clemson Univ. , ECE 847, http: //www. ces. clemson. edu/~stb/ece 847

Classical connected components: traverse table 1 0 0 1 0 1 1 1 0 1 2 3 3 1 1 1 0 0 0 1 2 3 5 image 0 1 2 3 5 0 1 2 2 2 0 0 0 4 4 components 0 1 2 3 4 equivalence table 5 S. Birchfield, Clemson Univ. , ECE 847, http: //www. ces. clemson. edu/~stb/ece 847

Classical connected components: second pass 1 0 0 1 0 1 1 1 0 3 3 1 1 1 0 0 0 1 0 3 0 image 0 1 0 3 0 0 1 0 0 0 4 4 components 0 1 2 3 4 equivalence table 5 S. Birchfield, Clemson Univ. , ECE 847, http: //www. ces. clemson. edu/~stb/ece 847

Another example from http: //www. sciencedirect. com/science/article/pii/S 0031320308004573

Connected components Function labels = Connected. Components(image) // first pass for y=0 to height-1, for x=0 to width-1, pix = image(x, y) if pix == image(x-1, y) && pix == image(x, y-1) labels(x, y) = labels(x, y-1) Set. Equivalence(labels(x-1, y), labels(x, y-1)) else if pix == image(x-1, y) labels(x, y) = labels(x-1, y) else if pix == image(x, y-1) labels(x, y) = labels(x, y-1) else labels(x, y) = next_label++ // second pass for y=0 to height-1, for x=0 to width-1, labels(x, y) = Get. Equivalent. Label( labels(x, y) ) S. Birchfield, Clemson Univ. , ECE 847, http: //www. ces. clemson. edu/~stb/ece 847

Connected components Function Set. Equivalence(a, b) aa = Get. Equivalent. Label(a) bb = Get. Equivalent. Label(b) if aa > bb equiv[aa] = bb else if aa < bb equiv[bb] = aa Function b = Get. Equivalent. Label(a) if a == equiv[a] return a else equiv[a] = Get. Equivalent. Label(equiv[a]) return equiv[a] S. Birchfield, Clemson Univ. , ECE 847, http: //www. ces. clemson. edu/~stb/ece 847

S. Birchfield, Clemson Univ. , ECE 847, http: //www. ces. clemson. edu/~stb/ece 847

Connected components quantized image components S. Birchfield, Clemson Univ. , ECE 847, http: //www. ces. clemson. edu/~stb/ece 847

Outline • Pixel-based image transformations (thresholding, …) • Removing noise from binary image (morphological operators) • Labeling regions (floodfill, connected components) • Computing distance in an image (chamfer distance, …) • Region properties (area and boundary properties) S. Birchfield, Clemson Univ. , ECE 847, http: //www. ces. clemson. edu/~stb/ece 847

Perimeter • Perimeter of binary region: – Dilate, then subtract – Erode, then subtract – (Which is better? ) • This only gives the pixels as binary image • To get ordered list of contiguous pixels, use Wall Follow algorithm S. Birchfield, Clemson Univ. , ECE 847, http: //www. ces. clemson. edu/~stb/ece 847

Wall following Algorithm: 1. Start at ON pixel 2. Move in any direction until end of region reached 3. Turn right 4. Repeat: 1. If pixel to the left is ON, Turn left and move forward 2. Else if pixel in front is OFF, Turn right 3. Else move forward - This traverses clockwise, using 4 -connectedness (other versions are possible) - Can be used to compute perimeter S. Birchfield, Clemson Univ. , ECE 847, http: //www. ces. clemson. edu/~stb/ece 847

Distance functions • D is a distance function (metric) if • Common distance functions: shape of each? S. Birchfield, Clemson Univ. , ECE 847, http: //www. ces. clemson. edu/~stb/ece 847

Other distances • Mahalanobis (weight differently along each axis) • Hausdorff (distance b/w point sets) A and B are subsets of S S. Birchfield, Clemson Univ. , ECE 847, http: //www. ces. clemson. edu/~stb/ece 847

Length along path # of diagonal moves Freeman formula c=0 (overestimates) Pythagorean theorem c=1 (underestimates) Kimura’s method c=½ (good compromise) Example: 3 methods compared: No=8 circle with radius = 10 * isothetic (blue) * diagonal (orange) Nd=6 # of isothetic moves Freeman: L = 16. 5 Pythagorean: L = 15. 2 Kimura: L = 15. 7 True arc length: L = 10*p/2 = 15. 7 S. Birchfield, Clemson Univ. , ECE 847, http: //www. ces. clemson. edu/~stb/ece 847

Chamfering • How to approximate “correct” distance (Euclidean)? • Along a path, – Let m be number of isothetic moves (horizontal or vertical) – Let n be number of diagonal moves • Define chamfer distance between p and q as dab = min {am+bn} over all possible paths • Chamfer distance is metric if a, b > 0 • If b > 2 a, then diagonal moves are ignored • Special cases: – – – a=1, b=infinity (Manhattan) a=1, b=1 (chessboard) a=1, b=sqrt(2) (another reasonable choice) a=1, b=1. 351 (best approximation to Euclidean) a=3, b=4 (best integer approximation, scaled by factor of 3) [G. Borgefors, Distance transformations in digital images, CVGIP, 1986] S. Birchfield, Clemson Univ. , ECE 847, http: //www. ces. clemson. edu/~stb/ece 847

Chamfering (cont. ) • Chamfering is process of computing distance to each foreground pixel • Two-pass algorithm – Traverse top-to-bottom, left-to-right – Traverse bottom-to-top, right-to-left In each case, compute minimum distance L: R: 4 -connectivity L: foreground pixels R: 8 -connectivity S. Birchfield, Clemson Univ. , ECE 847, http: //www. ces. clemson. edu/~stb/ece 847

Chamfering (cont. ) • Algorithm (for D 4 using Manhattan distance): Function labels = Chamfer(image) // first pass for y=0 to height-1, for x=0 to width-1, d(x, y) = image(x, y) ? 0 : min(infinity, 1+d(x-1, y), 1+d(x, y-1)) // second pass for y=height-1 to 0 step -1, for x=width-1 to 0 step -1, d(x, y) = image(x, y) ? 0 : min(d(x, y), 1+d(x+1, y), 1+d(x, y+1)) Note: “? : ” means “if-else” (C conditional expression operator) S. Birchfield, Clemson Univ. , ECE 847, http: //www. ces. clemson. edu/~stb/ece 847

Uses of chamfering • Find center of biggest part of shape (need to compute signed distance) centroid largest chamfer value • Shape matching http: //www. gavrila. net/Research/Chamfer_System/chamfer_system. html • Useful for watershed thresholding (we will see this later) S. Birchfield, Clemson Univ. , ECE 847, http: //www. ces. clemson. edu/~stb/ece 847

Outline • Pixel-based image transformations (thresholding, …) • Removing noise from binary image (morphological operators) • Labeling regions (floodfill, connected components) • Computing distance in an image (chamfer distance, …) • Region properties (area and boundary properties) S. Birchfield, Clemson Univ. , ECE 847, http: //www. ces. clemson. edu/~stb/ece 847

Descriptors • Region – – – Euler number convex hull skeleton moments compactness eccentricity (elongatedness) • Boundary – – – – } We will focus on regions chain code signature r-q plot s-y plot boundary segments shape number Fourier descriptors S. Birchfield, Clemson Univ. , ECE 847, http: //www. ces. clemson. edu/~stb/ece 847

Euler number E = C- H components 1 -1=0 1 -2=-1 holes 1 -0=1 2 -0=2 Measure of topology S. Birchfield, Clemson Univ. , ECE 847, http: //www. ces. clemson. edu/~stb/ece 847

Euler number • Euler number computed as (N 1 -N 2 -2 N 3)4, where – N 1 is the number of 2 x 2 patterns with 1 in either corner, 0 elsewhere (4 possible patterns) – N 2 is the number of 2 x 2 patterns with 0 in either corner , 0 elsewhere (4 possible patterns) – N 3 is the number of 2 x 2 checkerboard patterns (2 possible patterns) Explained in W. K. Pratt, Digital Image Processing, 1991 S. Birchfield, Clemson Univ. , ECE 847, http: //www. ces. clemson. edu/~stb/ece 847

Convex hull • Convex – any line connecting two points in R is also in R convex not convex • Convex hull – the minimal convex set containing R S. Birchfield, Clemson Univ. , ECE 847, http: //www. ces. clemson. edu/~stb/ece 847

Skeletons should skeleton touch boundary? • Point on skeleton is equidistant to >= 2 boundary points (without crossing) • Or, equivalently, – where wavefronts (grassfires) meet (shock graph) – centers of maximal balls – Blum’s Medial Axis Transform (MAT) • Skeleton is highly sensitive to noise S. Birchfield, Clemson Univ. , ECE 847, http: //www. ces. clemson. edu/~stb/ece 847

Skeleton examples S. Birchfield, Clemson Univ. , ECE 847, http: //www. ces. clemson. edu/~stb/ece 847

Skeletonization by thinning • First compute MAT • Then delete points such that – do not remove endpoints – do not break connectivity – do not excessively erode region S. Birchfield, Clemson Univ. , ECE 847, http: //www. ces. clemson. edu/~stb/ece 847

Moments in 1 D • The pth moment of a function f: • The pth central moment: • What are ? centroid (area, 0, variance, skew, kurtosis) m 2 is also moment of inertia about axis passing through centroid S. Birchfield, Clemson Univ. , ECE 847, http: //www. ces. clemson. edu/~stb/ece 847

Moments in 2 D • The pqth moment of a 2 D region: can be • grayscale or • binary (more typical) • The pqth central moment: (area) (variance in x) (variance in y) S. Birchfield, Clemson Univ. , ECE 847, http: //www. ces. clemson. edu/~stb/ece 847

Moments in 2 D (cont. ) • The pqth normalized central moment: • Hu moments (derived from these) are invariant to – – translation rotation scaling skew (at least one of them is) S. Birchfield, Clemson Univ. , ECE 847, http: //www. ces. clemson. edu/~stb/ece 847

Moments example • Simple 5 x 5 binary image: m 00 = m 10 = m 01 = m 11 = m 20 = m 02 = 12. 0 23. 0 21. 0 37. 0 65. 0 63. 0 m 00 = m 10 = m 01 = m 11 = m 20 = m 02 = 12. 0 0. 0 -3. 3 20. 9 26. 3 S. Birchfield, Clemson Univ. , ECE 847, http: //www. ces. clemson. edu/~stb/ece 847

Computing moments One pass through image to compute regular moments Central moments (and normalized / Hu) are computed without touching the pixels again! (Typically, f is a binarized image containing a single non-zero region) S. Birchfield, Clemson Univ. , ECE 847, http: //www. ces. clemson. edu/~stb/ece 847

Projections • Project image onto x, y, or diagonal • Projection offers compact representation of image • First-order moments of image are equal to first-order moments of projection • Second-order moments require diagonal projection S. Birchfield, Clemson Univ. , ECE 847, http: //www. ces. clemson. edu/~stb/ece 847

Compactness Related to eccentricity (elongatedness) • Define compactness as compactness = 1 (area = # of pixels, perimeter = length along boundary) compactness < 1 (Note: discretization effects of computing perimeter are severe) Alternative: Use S. Birchfield, Clemson Univ. , ECE 847, http: //www. ces. clemson. edu/~stb/ece 847

Eccentricity • Approximate region with ellipse • eccentricity = a / a = • Formula given by PCA S. Birchfield, Clemson Univ. , ECE 847, http: //www. ces. clemson. edu/~stb/ece 847

Principal components analysis (PCA) Given n points in d dimensional space Goals: 1. fit ellipsoid to data 2. reduce dimensionality Ellipsoid specified by • center (given by mean of data) • orientation of axes (given by eigenvectors of covariance matrix of data) • length of axes (given by square root of eigenvalues of covariance matrix of data) S. Birchfield, Clemson Univ. , ECE 847, http: //www. ces. clemson. edu/~stb/ece 847

PCA (cont. ) • n points in d dimensional space • eigenvalues and eigenvectors • matrix form • mean • covariance • factorize covariance matrix S. Birchfield, Clemson Univ. , ECE 847, http: //www. ces. clemson. edu/~stb/ece 847

PCA (cont. ) • linear transformation • reconstruct x • data uncorrelated! • but elements of y are decreasingly important • vectors ei form an orthonormal basis for space • project to k-dimensional space S. Birchfield, Clemson Univ. , ECE 847, http: //www. ces. clemson. edu/~stb/ece 847

PCA (cont. ) S. Birchfield, Clemson Univ. , ECE 847, http: //www. ces. clemson. edu/~stb/ece 847

PCA example • Simple 5 x 5 binary image: m = (2. 0, 2. 0) C = (1/9) * [ 12 -10] [-10 12] e 1 e 2 l 1 C = PLPT = [-0. 7] [2. 4 0 ] [-0. 7] [ 0 0. 2] [ 0. 7] l 2 major axis = e 1√l 1 = [1. 1 -1. 1]T T e √l = [-0. 3] 2 2 minor axis = S. Birchfield, Clemson Univ. , ECE 847, http: //www. ces. clemson. edu/~stb/ece 847

Best fitting ellipse The elliptical binary region in image bounded by the ellipse has the following second-order moments To be an ellipse, Note: (which is always true for a covariance matrix) S. Birchfield, Clemson Univ. , ECE 847, http: //www. ces. clemson. edu/~stb/ece 847

Eccentricity and Direction • Covariance matrix of region • Eigenvalues solve det(C-l. I)=0) • Eccentricity ranges from 0 (circle) to 1 (line) • Direction (clockwise angle from positive x-axis) Hint: Use atan 2(y, x), passing numerator and denominator as separate arguments • Eigenvectors S. Birchfield, Clemson Univ. , ECE 847, http: //www. ces. clemson. edu/~stb/ece 847

mean face } eigenvectors Eigenfaces N x N image is a point in an N 2 dimensional space N 2 x N 2 Covariance matrix captures relationships between face images in ensemble Turk and Pentland, 1991 S. Birchfield, Clemson Univ. , ECE 847, http: //www. ces. clemson. edu/~stb/ece 847

Fruit classification (high threshold) original image (low threshold) Note: double threshold and connected components can be computed simultaneously: • scan high threshold image • if pixel is 1, then floodfill using low threshold image (connected components) mij (double threshold) mij (moments) (classification) (banana stem) S. Birchfield, Clemson Univ. , ECE 847, http: //www. ces. clemson. edu/~stb/ece 847

Sample results fruit 1. pgm fruit 2. pgm S. Birchfield, Clemson Univ. , ECE 847, http: //www. ces. clemson. edu/~stb/ece 847