Computer and Robot Vision I Chapter 6 Neighborhood

Computer and Robot Vision I Chapter 6 Neighborhood Operators Presented by: 林德垣 R 00944051@. csie. ntu. edu. tw 指導教授: 傅楸善 博士 Digital Camera and Computer Vision Laboratory Department of Computer Science and Information Engineering National Taiwan University, Taipei, Taiwan, R. O. C.

6. 1 Introduction l neighborhood operator: workhorse of lowlevel vision l neighborhood operator: performs conditioning, labeling, grouping DC & CV Lab. CSIE NTU

6. 1 Introduction l The output of a neighborhood operator at a given pixel position is a function of the position, of the input pixel value at the position, of the values of the input pixels in some neighborhood around the given input position, and possibly of some values of previously generated output pixels DC & CV Lab. CSIE NTU

6. 1 Introduction l l numeric domain: arithmetic operations, +, -, min, max symbolic domain: Boolean operations, AND, OR, NOT, table-look-up nonrecursive neighborhood operators: output is function of input recursive neighborhood operators: output depends partly on previous output DC & CV Lab. CSIE NTU

6. 1 Introduction l neighborhood might be small and asymmetric or large DC & CV Lab. CSIE NTU

6. 1 Introduction l l : set of neighboring pixel positions around position general nonrecursive neighborhood operator : input , output DC & CV Lab. CSIE NTU

6. 1 Introduction l l linear operator: one common nonrecursive neighborhood operator output: possibly position-dependent linear combination of inputs DC & CV Lab. CSIE NTU

6. 1 Introduction l shift-invariant: position invariant: action same regardless of position l composition of shift-invariant operators: shiftinvariant [e. g. ] N satisfies a Shift invariance: (r’, c’) ∈ N(r, c) (r’- u, c’-v) ∈ N(r-u, c-v) , for all (u, v) DC & CV Lab. CSIE NTU

6. 1 Introduction l l l cross-correlation of with : weight function: kernel or mask of weights : domain of DC & CV Lab. CSIE NTU

6. 1 Introduction l common (a) box filter masks for noise cleaning, DC & CV Lab. CSIE NTU

6. 1 Introduction l common masked for noise cleaning DC & CV Lab. CSIE NTU

6. 1 Introduction l Different N x N filter effects comparison DC & CV Lab. CSIE NTU Original Image 3 x 3 5 x 5 9 x 9 15 x 15 35 x 35

6. 1 Introduction application of mask with weights to image l 7. 69 DC & CV Lab. CSIE NTU

6. 1 Introduction l convolution of l convolution: close relative to cross-correlation convolution: linear shift-invariant if mask symmetric, convolution, and correlation the same l l with DC & CV Lab. CSIE NTU

6. 2 Symbolic Neighborhood Operators l indexing of neighborhoods DC & CV Lab. CSIE NTU

6. 2. 1 Region-Growing Operator l l l : projection, outputs first or second argument : background pixel labeled first nonbackground label DC & CV Lab. CSIE NTU

6. 2. 1 Region-Growing Operator l 4 -connected: l output: DC & CV Lab. CSIE NTU

6. 2. 1 Region-Growing Operator l 8 -connected: l output: DC & CV Lab. CSIE NTU

6. 2. 1 Region-Growing Operator a 7 a 2 a 6 a 3 a 0 a 1 a 8 a 4 a 5 DC & CV Lab. CSIE NTU

6. 2. 1 Region-Growing Operator DC & CV Lab. CSIE NTU

6. 2. 1 Region-Growing Operator DC & CV Lab. CSIE NTU

6. 2. 1 Region-Growing Operator DC & CV Lab. CSIE NTU

6. 2. 1 Region-Growing Operator DC & CV Lab. CSIE NTU

6. 2. 2 Nearest Neighbor Sets and Influence Zones l l influence zones: nearest neighbor sets influence zones: iteratively region-growing DC & CV Lab. CSIE NTU

6. 2. 2 Nearest Neighbor Sets and Influence Zones l l l 4 -neighborhood for city-block distance e. g. ( i , j ), ( k , l ) : | ( k – i ) | + | ( l – j ) | 8 -neighborhood for max distance (of horizontal and vertical distances) e. g. ( i , j ), ( k , l ) : max( | k – i | , | l – j | ) alternate 4, 8 -neighborhood for Euclidean distance DC & CV Lab. CSIE NTU

Take a Break DC & CV Lab. CSIE NTU

6. 2. 3 Region-Shrinking Operator l l l region-shrinking: changes all border pixels to background region-shrinking: can change connectivity region-shrinking: can entirely delete region if repeatedly applied DC & CV Lab. CSIE NTU

6. 2. 3 Region-Shrinking Operator l l l : whether or not arguments identical : background border: has different neighbor and becomes background DC & CV Lab. CSIE NTU

6. 2. 3 Region-Shrinking Operator l 4 -connected: l output: DC & CV Lab. CSIE NTU

6. 2. 3 Region-Shrinking Operator l 8 -connected: l output: DC & CV Lab. CSIE NTU

6. 2. 3 Region-Shrinking Operator l region shrinking: related to binary erosion except on labeled region DC & CV Lab. CSIE NTU

6. 2. 3 Region-Shrinking Operator DC & CV Lab. CSIE NTU

6. 2. 3 Region-Shrinking Operator DC & CV Lab. CSIE NTU

6. 2. 3 Region-Shrinking Operator DC & CV Lab. CSIE NTU

6. 2. 3 Region-Shrinking Operator DC & CV Lab. CSIE NTU

6. 2. 4 Mark-Interior/Border-Pixel Operator l mark-interior/border-pixel operator marks all interior pixels with the label and all border pixels with the label DC & CV Lab. CSIE NTU

6. 2. 4 Mark-Interior/Border-Pixel Operator l l : whether or not arguments identical : recognizes whether or not its argument is symbol DC & CV Lab. CSIE NTU

6. 2. 4 Mark-Interior/Border-Pixel Operator l 4 -connected: l output: DC & CV Lab. CSIE NTU

6. 2. 4 Mark-Interior/Border-Pixel Operator l 8 -connected: l output: DC & CV Lab. CSIE NTU

6. 2. 5 Connectivity Number Operator l l connectivity number: nonrecursive and symbolic data domain connectivity number: classify the way pixel connected to neighbors six values of connectivity: five for border, one for interior border: isolated, edge, connected, branching, crossing DC & CV Lab. CSIE NTU

6. 2. 5 Connectivity Number Operator DC & CV Lab. CSIE NTU 4 -connectivity

6. 2. 5 Connectivity Number Operator l corner neighborhood DC & CV Lab. CSIE NTU

6. 2. 5 Connectivity Number Operator DC & CV Lab. CSIE NTU

Yokoi Connectivity Number l l 4 -connectivity : corner transition : corner all , no transition : center , neighbor , nothing will happen DC & CV Lab. CSIE NTU

Yokoi Connectivity Number l l 5: no transition all 8 neighbors 1, thus interior : 1 transition generates one connected component if center removed DC & CV Lab. CSIE NTU

Yokoi Connectivity Number l 4 - Connectivity number DC & CV Lab. CSIE NTU

Yokoi Connectivity Number e d b c DC & CV Lab. CSIE NTU

Yokoi Connectivity Number e d b c DC & CV Lab. CSIE NTU

Yokoi Connectivity Number l lena. 64*64 DC & CV Lab. CSIE NTU

Yokoi Connectivity Number l lena. yokoi DC & CV Lab. CSIE NTU

Yokoi Connectivity Number l l 8 -connectivity, only slightly different : center and corner transition : corner no transition, all : itself and two 4 -neighbors , corner DC & CV Lab. CSIE NTU

Yokoi Connectivity Number DC & CV Lab. CSIE NTU

Rutovitz Connectivity Number l l Rutovitz connectivity: number of transitions from one symbol to another Rutovitz connectivity number: sometimes called crossing number h(a, b, c) = { 1, if (a=b and a <> c) or (a<>b and a =c) 0, otherwise } a 1 = h( X 0, X 1, X 6) a 2 = h( X 0, X 6, X 2) a 3 = h( X 0, X 2, X 7) a 4 = h( X 0, X 7, X 3) a 5 = h( X 0, X 3, X 8) a 6 = h( X 0, X 8, X 4) a 7 = h( X 0, X 4, X 5) a 8 = h( X 0, X 5, X 1) DC & CV Lab. CSIE NTU

Take a Break DC & CV Lab. CSIE NTU

6. 2. 6 Connected Shrink Operator l l connected shrink: recursive operator, symbolic data domain connected shrink: deletes border pixels without disconnecting regions DC & CV Lab. CSIE NTU

6. 2. 6 Connected Shrink Operator l Top-down, left-right scan: deletes edge pixels not right-boundary DC & CV Lab. CSIE NTU

DC & CV Lab. CSIE NTU

6. 2. 6 Connected Shrink Operator l : determines whether corner connected DC & CV Lab. CSIE NTU

6. 2. 6 Connected Shrink Operator l l : background output symbol l DC & CV Lab. CSIE NTU

6. 2. 6 Connected Shrink Operator DC & CV Lab. CSIE NTU

6. 2. 6 Connected Shrink Operator 1 h (b, c, d, e) = { 1, if c= b and (d <> b or e<>b) 0, otherwise } Output symbol y = f (1 , 0 , 0 , 1) = g DC & CV Lab. CSIE NTU

6. 2. 6 Connected Shrink Operator g g 1 1 1 DC & CV Lab. CSIE NTU

6. 2. 7 Pair Relationship Operator l l pair relationship: nonrecursive operator, symbolic data domain : determines whether first argument equals label DC & CV Lab. CSIE NTU

6. 2. 7 Pair Relationship Operator l 4 -connected mode, output l : not deletable if Yokoi number or no neighbor : possibly deletable if Yokoi number and some neighbor l DC & CV Lab. CSIE NTU

6. 2. 8 Thinning Operator l thinning operator is composition of three operators: l l l Mark-Interior/Border-Pixel The pair relationship Connected shrink DC & CV Lab. CSIE NTU

6. 2. 8 Thinning Operator DC & CV Lab. CSIE NTU

6. 2. 8 Thinning Operator l lena. thin DC & CV Lab. CSIE NTU

6. 2. 8 Thinning Operator l lena. thin DC & CV Lab. CSIE NTU

Take a Break DC & CV Lab. CSIE NTU

6. 2. 9 Distance Transformation Operator l distance transformation: produces distance to closest border pixel DC & CV Lab. CSIE NTU

6. 2. 9 Distance Transformation Operator l l nonrecursive, iterative, start with : background : border, because distance to border is : interior pixels DC & CV Lab. CSIE NTU

6. 2. 9 Distance Transformation Operator l iteration: label with DC & CV Lab. CSIE NTU all with neighbor

6. 2. 9 Distance Transformation Operator l equivalent algorithm: nonrecursive iterative l : if no neighbor labeled, still interior, leave it alone : self interior but neighbor labeled : already labeled, won’t change value since later route farther 4 -connected: output l l l DC & CV Lab. CSIE NTU

6. 2. 9 Distance Transformation Operator l l distance transformation: produces distance to closest background recursive, two-pass DC & CV Lab. CSIE NTU

6. 2. 9 Distance Transformation Operator l first pass: left-right, top-bottom l l : if background still background, since min: min of upper and left neighbors and add l 4 -connected output DC & CV Lab. CSIE NTU

6. 2. 9 Distance Transformation Operator l second pass: right-left, bottom-up l 4 -connected output DC & CV Lab. CSIE NTU

6. 2. 9 Distance Transformation Operator DC & CV Lab. CSIE NTU

6. 2. 9 Distance Transformation Operator l after pass 1 DC & CV Lab. CSIE NTU

6. 2. 9 Distance Transformation Operator pass 1 DC & CV Lab. CSIE NTU

Take a Break DC & CV Lab. CSIE NTU

6. 2. 10 Radius of Fusion DC & CV Lab. CSIE NTU

6. 2. 10 Radius of Fusion DC & CV Lab. CSIE NTU

6. 2. 11 Number of Shortest Paths l number of shortest paths for each 0 -pixel: to binary-1 pixel set given binary image l : can be 4 -neighborhood or 8 -neighborhood : nonzero, stays unchanged since shortest paths counted : if zero then sum of neighboring shortest paths l l DC & CV Lab. CSIE NTU

6. 2. 11 Number of Shortest Paths DC & CV Lab. CSIE NTU

6. 2. 11 Number of Shortest Paths DC & CV Lab. CSIE NTU

6. 2. 11 Number of Shortest Paths DC & CV Lab. CSIE NTU

Take a Break DC & CV Lab. CSIE NTU

6. 3 Extremum-Related Neighborhood Operators DC & CV Lab. CSIE NTU

6. 3. 1 Non-Minima-Maxima Operator l non-minima-maxima: nonrecursive operator, symbolic output DC & CV Lab. CSIE NTU

6. 3. 1 Non-Minima-Maxima Operator l a pixel can be neighborhood maximum not relative maximum DC & CV Lab. CSIE NTU

6. 3. 1 Non-Minima-Maxima Operator DC & CV Lab. CSIE NTU

6. 3. 1 Non-Minima-Maxima Operator l 4 -connected: l output pixel DC & CV Lab. CSIE NTU

6. 3. 2 Relative Extrema Operator l relative extrema operators l l relative extrema l l relative maximum and minimum operators recursive operator, numeric data domain relative extrema l input not changed；output modified l l l top-down, left-right scan then bottom-up, right-left scan until no change DC & CV Lab. CSIE NTU

6. 3. 2 Relative Extrema Operator l l output value: of highest extrema reachable by monotonic path relative extrema pixels: output same as input pixels DC & CV Lab. CSIE NTU

6. 3. 2 Relative Extrema Operator l pixel designations for the normal and reverse scans DC & CV Lab. CSIE NTU

6. 3. 2 Relative Extrema Operator DC & CV Lab. CSIE NTU

6. 3. 2 Relative Extrema Operator l two primitive functions l : use new maximum value when ascending : keep original maximum when descending l DC & CV Lab. CSIE NTU and

6. 3. 2 Relative Extrema Operator : top-down , left-right : bottom-up , right-left l 4 -connected, output DC & CV Lab. CSIE NTU

6. 3. 2 Relative Extrema Operator a 0 = l 0 a 1 = h (x 0, x 1, a 0, l 1) a 2 = h (x 0, x 2, a 1, l 2) x 2 , l 2 x 1 , l 1 a 0 l 0, x 0 x 2 , l 2 3 3, 3, 3, 3 ) x 1 , l 1 a 0 l 0, x 0 DC & CV Lab. CSIE NTU

6. 3. 2 Relative Extremaa Operator =l 0 0 a 1 = h (x 0, x 1, a 0, l 1) a 2 = h (x 0, x 2, a 1, l 2) 5 x 2 , l 2 x 1 , l 5, 5, 5, 5) 1 a 0 l 0, x 0 x 2 , l 2 x 1 , l 1 a 0 l 0, x 0 DC & CV Lab. CSIE NTU

6. 3. 2 Relative Extrema Operator DC & CV Lab. CSIE NTU

6. 3. 2 Relative Extrema Operator DC & CV Lab. CSIE NTU

6. 3. 2 Relative Extrema Operator DC & CV Lab. CSIE NTU

6. 3. 2 Relative Extrema Operator DC & CV Lab. CSIE NTU

6. 3. 2 Relative Extrema Operator 4 DC & CV Lab. CSIE NTU

6. 3. 2 Relative Extrema Operator a 0 = l 0 a 1 = h (x 0, x 1, a 0, l 1) a 2 = h (x 0, x 2, a 1, l 2) 6 x 2 1 9 x 1 a 0 l 0 x 0 x 2 l 2 2 x 1 l 2 DC & CV Lab. CSIE NTU a 0 l 1

6. 3. 2 Relative Extrema Operator 6 DC & CV Lab. CSIE NTU

6. 3. 3 Reachability Operator l successively propagating labels that can reach by monotonic paths descending reachability operator employs DC & CV Lab. CSIE NTU

6. 3. 3 Reachability Operator l l l g: pixels that are not relative extrema labeled background a: unchanged when neighbor is g or the same with neighbor b: become first propagated label if originally g c: common region more than one extremum can reach it by monotonic path c: already labeled and neighbor has different label, then two extrema reach a: propagate from extremum DC & CV Lab. CSIE NTU

6. 3. 3 Reachability Operator l 4 -connected, output a 0 = l 0 a 1 = h (a 0 , l 1 , x 0 , x 1) a 2 = h (a 1 , l 2 , x 0 , x 2) DC & CV Lab. CSIE NTU

6. 3. 3 Reachability Operator DC & CV Lab. CSIE NTU

6. 3. 3 Reachability Operator DC & CV Lab. CSIE NTU

6. 3. 3 Reachability Operator DC & CV Lab. CSIE NTU

Take a Break DC & CV Lab. CSIE NTU

6. 4 Linear Shift-Invariant Neighborhood Operators l convolution: commutative, associative, distributor over sums, homogeneous DC & CV Lab. CSIE NTU

6. 4. 1 Convolution and Correlation l convolution of an image f with kernel w DC & CV Lab. CSIE NTU

6. 4. 1 Convolution and Correlation DC & CV Lab. CSIE NTU

6. 4. 1 Convolution and Correlation DC & CV Lab. CSIE NTU

6. 4. 1 Convolution and Correlation DC & CV Lab. CSIE NTU

6. 4. 1 Convolution and Correlation DC & CV Lab. CSIE NTU

6. 4. 1 Convolution and Correlation DC & CV Lab. CSIE NTU

6. 4. 1 Convolution and Correlation DC & CV Lab. CSIE NTU

6. 4. 1 Convolution and Correlation DC & CV Lab. CSIE NTU

6. 4. 1 Convolution and Correlation DC & CV Lab. CSIE NTU

6. 4. 2 Separability l l straightforward computation: multiplications, additions decomposition: multiplications, additions DC & CV Lab. CSIE NTU

6. 4. 2 Separability DC & CV Lab. CSIE NTU

Project due Nov. 6 l l l Write a program to generate Yokoi connectivity number You can down sample lena. bmp from 512*512 to 64*64 first. Sample pixels positions at (r, c) = (0, 0), (0, 8) ……, so everyone will get the same answer. DC & CV Lab. CSIE NTU

Project due Nov. 20 l l l Write a program to generate thinned image. You can down sample lena. bmp from 512*512 to 64*64 first. Sample pixels positions at (r, c) = (0, 0), (0, 8) ……, so everyone will get the same answer. DC & CV Lab. CSIE NTU

Midterm Nov. 13 l extent: whatever covered up to Nov. 6 (inclusive) DC & CV Lab. CSIE NTU