Computer and Robot Vision I Chapter 10 Image
Computer and Robot Vision I Chapter 10 Image Segmentation Presented by: 傅楸善 & 施登富 0910429501 g 104018004@gmail. com 指導教授: 傅楸善 博士 Digital Camera and Computer Vision Laboratory 1
10. 1 Introduction l l image segmentation: partition of image into set of non-overlapping regions image segmentation: union of segmented regions is the entire image segmentation purpose: to decompose image into meaningful parts to application segmentation based on valleys in gray level histogram into regions Digital Camera and Computer Vision Laboratory 2
10. 1 Introduction (cont’) Digital Camera and Computer Vision Laboratory 3
Digital Camera and Computer Vision Laboratory 4
10. 1 Introduction (cont’) rules for general segmentation procedures l 1. 2. 3. 4. region uniform, homogeneous w. r. t. characteristic e. g. gray level, texture region interiors simple and without many small holes adjacent regions with significantly different values on characteristic boundaries simple not ragged spatially accurate Digital Camera and Computer Vision Laboratory 5
10. 1 Introduction (cont’) l l Clustering: process of partitioning set of pattern vectors into clusters set of points in Euclidean measurement space separated into 3 clusters Digital Camera and Computer Vision Laboratory 6
10. 1 Introduction (cont’) Digital Camera and Computer Vision Laboratory 7
10. 1 Introduction (cont’) l l l no full theory of clustering no full theory of image segmentation techniques: ad hoc, different in emphasis and compromise Digital Camera and Computer Vision Laboratory 8
l Joke Digital Camera and Computer Vision Laboratory 9
10. 2 Measurement-Space-Guided Spatial Clustering l The technique of measurement-space-guided spatial clustering for image segmentation uses the measurement-space-clustering process to define a partition in measurement space. Digital Camera and Computer Vision Laboratory 10
10. 2 Measurement-Space-Guided Spatial Clustering (cont’) l l l histogram mode seeking: a measurementspace-clustering process histogram mode seeking: homogeneous objects as clusters in histogram mode seeking: one pass, the least computation time Digital Camera and Computer Vision Laboratory 11
10. 2 Measurement-Space-Guided Spatial Clustering (cont’) l enlarged image of a polished mineral ore section Digital Camera and Computer Vision Laboratory 12
10. 2 Measurement-Space-Guided Spatial Clustering (cont’) l 3 nonoverlapping modes: black holes, pyrorhotite, pyrite Digital Camera and Computer Vision Laboratory 13
10. 2 Measurement-Space-Guided Spatial Clustering (cont’) Digital Camera and Computer Vision Laboratory 14
10. 2 Measurement-Space-Guided Spatial Clustering (cont’) l 2 valleys in histogram is a virtually perfect (meaningful) segmentation Digital Camera and Computer Vision Laboratory 15
10. 2 Measurement-Space-Guided Spatial Clustering (cont’) l example image not ideal for measurementspace-clustering image segmentation Digital Camera and Computer Vision Laboratory 16
l 10. 2 Measurement-Space-Guided Spatial Clustering (cont’) histogram with three modes and two valleys Digital Camera and Computer Vision Laboratory 17
10. 2 Measurement-Space-Guided Spatial Clustering (cont’) l Undesirable: many border regions show up as dark segments Digital Camera and Computer Vision Laboratory 18
10. 2 Measurement-Space-Guided Spatial Clustering (cont’) l segmentation into homogeneous regions: not necessarily good solution Digital Camera and Computer Vision Laboratory 19
l joke Digital Camera and Computer Vision Laboratory 20
10. 2 Measurement-Space-Guided Spatial Clustering (cont’) ldiagram of an F-15 bulkhead(隔艙板) Digital Camera and Computer Vision Laboratory 21
10. 2 Measurement-Space-Guided Spatial Clustering (cont’) l image of a section of the F-15 bulkhead Digital Camera and Computer Vision Laboratory 22
10. 2 Measurement-Space-Guided Spatial Clustering (cont’) l Histogram of the image Digital Camera and Computer Vision Laboratory 23
10. 2 Measurement-Space-Guided Spatial Clustering (cont’) l Five clusters: bad spatial continuation, boundaries noisy and busy Digital Camera and Computer Vision Laboratory 24
10. 2 Measurement-Space-Guided Spatial Clustering (cont’) l three clusters: less boundary noise, but much less detail Digital Camera and Computer Vision Laboratory 25
10. 2 Measurement-Space-Guided Spatial Clustering (cont’) l recursive histogram-directed spatial clustering Digital Camera and Computer Vision Laboratory 26
Digital Camera and Computer Vision Laboratory 27
10. 2 Measurement-Space-Guided Spatial Clustering (cont’) l applied to the bulkhead image Digital Camera and Computer Vision Laboratory 28
10. 2 Measurement-Space-Guided Spatial Clustering (cont’) l performing morphological opening with 3 x 3 square structuring element Digital Camera and Computer Vision Laboratory 29
10. 2 Measurement-Space-Guided Spatial Clustering (cont’) l tiny regions removed, but several long, thin regions lost Digital Camera and Computer Vision Laboratory 30
10. 2 Measurement-Space-Guided Spatial Clustering (cont’) l a color image Digital Camera and Computer Vision Laboratory 31
10. 2 Measurement-Space-Guided Spatial Clustering (cont’) l recursive histogram-directed spatial clustering using R, G, B bands and other Digital Camera and Computer Vision Laboratory 32
l joke Digital Camera and Computer Vision Laboratory 33
10. 2. 1 Thresholding l Kohler denotes the set E(T) of edges detected by a threshold T to be the set of all pairs of neighboring pixels one of whose gray level intensities is less than or equal to T and one of whose gray level intensities is greater than T Digital Camera and Computer Vision Laboratory 34
10. 2. 1 Thresholding (cont’) l l 1. pixels 2. min and are neighbors max Digital Camera and Computer Vision Laboratory
10. 2. 1 Thresholding (cont’) l The total contrast C(T) of edges detected by threshold T is given by Digital Camera and Computer Vision Laboratory
10. 2. 1 Thresholding (cont’) l The average contrast of all edges detected by threshold T: l The best threshold Tb is determined by the value that maximizes Digital Camera and Computer Vision Laboratory 37
TEST!!! 10. 2. 1 Thresholding (cont’) l l l l Example T = 50 [45, 110] [33, 88] [15, 65] [45, 115] [0, 225] DC & CV Lab. NTU CSIE
10. 2. 1 Thresholding (cont’) l l l l Example T = 50 [45, 110] [33, 88] [15, 65] [45, 115] [0, 225] -> [5, 60] -> [17, 38] -> [35, 15] -> [5, 65] -> [50, 175] DC & CV Lab. NTU CSIE
10. 2. 1 Thresholding (cont’) l l l l Example T = 50 [5, 60] [17, 38] [35, 15] [5, 65] [50, 175] The average contrast of all edges (5+17+15+5+50) / 5 = 18. 4 DC & CV Lab. NTU CSIE
10. 2. 1 Thresholding (cont’) l l l l Example T = 60 [45, 110] -> [15, 50] [33, 88] -> [27, 28] [15, 65] -> [45, 5] [45, 115] -> [15, 55] [0, 225] -> [60, 165] DC & CV Lab. NTU CSIE
10. 2. 1 Thresholding (cont’) l l l l Example T = 60 [15, 50] [27, 28] [45, 5] [15, 55] [60, 165] The average contrast of all edges (15+27+5+15+60) / 5 = 24. 4 The best threshold Tb is determined by the maximizes DC & CV Lab. NTU CSIE
10. 2. 1 Thresholding (cont’) l l approach for segmenting white blob against dark background pixel with small gradient: not likely to be an edge Digital Camera and Computer Vision Laboratory 43
10. 2. 1 Thresholding (cont’) l l if not an edge, then either dark background pixel or bright blob pixel histogram of small gradient pixels: bimodal Digital Camera and Computer Vision Laboratory 44
10. 2. 1 Thresholding (cont’) l pixels with small gradients: valley between two modes: threshold point Digital Camera and Computer Vision Laboratory 45
10. 2. 1 Thresholding (cont’) l FLIR (Forward Looking Infra-Red) image from NATO (North Atlantic Treaty Organization) database 紅外線偵測戰車引擎位置 Digital Camera and Computer Vision Laboratory 46
10. 2. 1 Thresholding (cont’) l thresholded at gray level intensity 159 and 190 真正引擎的位置 引擎附近的熱氣 Digital Camera and Computer Vision Laboratory 47
10. 2. 1 Thresholding (cont’) l pixels having large gradient magnitude 戰車引擎位置 Digital Camera and Computer Vision Laboratory 48
10. 2. 1 Thresholding (cont’) l 2 D gray level intensity-gradient space Digital Camera and Computer Vision Laboratory 49
Digital Camera and Computer Vision Laboratory 50
10. 2. 1 Thresholding (cont’) l resulting segmentation: bright object with slightly darker appendage on top 戰車引擎位置 Digital Camera and Computer Vision Laboratory 51
l joke Digital Camera and Computer Vision Laboratory 52
10. 2. 2 Multidimensional Measurement-Space Clustering LANDSAT image: consists of seven separate images called bands Constraints of reality l l 1. 2. high correlation between band-to-band pixel values large amount of spatial redundancy in image data Digital Camera and Computer Vision Laboratory 53
10. 2. 2 Multidimensional Measurement-Space Clustering l l l l l Spectral sensitivity of LANDSAT 7 Bands. Band Number Wavelength Interval Spectral Response. 1. 0. 45 -0. 52 µm Blue-Green 2. 0. 52 -0. 60 µm Green 3. 0. 63 -0. 69 µm Red 4. 0. 76 -0. 90 µm Near IR 5. 1. 55 -1. 75 µm Mid-IR 6. 10. 40 -12. 50 µm Thermal IR 7. 2. 08 -2. 35 µm Mid-IR 54 Digital Camera and Computer Vision Laboratory
10. 2. 2 Multidimensional Measurement-Space Clustering Digital Camera and Computer Vision Laboratory 55
Digital Camera and Computer Vision Laboratory 56
Band 1: c 1, c 2, c 3 三個clusters Band 2: c 4, c 5 兩個clusters Band 1 + Band 2 的cluster 就可能是 {(c 1, c 4), (c 1, c 5) , (c 2, c 4) , (c 2, c 5) , (c 3, c 4) , (c 3, c 5)} Digital Camera and Computer Vision Laboratory 57
l joke Digital Camera and Computer Vision Laboratory 58
10. 3 Region Growing 10. 3. 1 Single-Linkage Region Growing l l Single-linkage region-growing schemes: regard each pixel as node in graph neighboring pixels with similar enough properties: joined by an arc image segments: maximal sets of pixels belonging to same connected component simple image and the corresponding graph Digital Camera and Computer Vision Laboratory 59
10. 3. 1 Single-Linkage Region Growing (cont’) l two pixels connected by edge: if 4 -neighbor and values differ 5 Digital Camera and Computer Vision Laboratory 60
10. 3. 1 (cont’) Single-Linkage Region Growing Digital Camera and Computer Vision Laboratory 61
10. 3. 2 Hybrid-Linkage Region Growing l l hybrid single-linkage techniques: more powerful than simple single-linkage hybrid techniques: assign property vector to each pixel property vector: depends on K x K neighborhood of the pixels similar: because neighborhoods similar in some special sense Digital Camera and Computer Vision Laboratory 62
10. 3. 2 Hybrid-Linkage Region Growing (cont’) l l region cannot be declared segment unless completely surrounded by edge pixels edge image with gaps in the edges can cause problems in segmentation Digital Camera and Computer Vision Laboratory 63
10. 3. 2 Hybrid-Linkage Region Growing (cont’) l edges from second directional derivative zero -crossing l l l Edge detection Edge Non-edge Digital Camera and Computer Vision Laboratory 64
10. 3. 2 Hybrid-Linkage Region Growing (cont’) l after region-filling l Some edges have been assigned to neighbor region l Solve edge gap l More regions Digital Camera and Computer Vision Laboratory 65
10. 3. 2 Hybrid-Linkage Region Growing (cont’) l Pong et al. suggest an approach to segmentation based on the facet model one iteration of Pong algorithm Digital Camera and Computer Vision Laboratory 66
Digital Camera and Computer Vision Laboratory 67
10. 3. 2 Hybrid-Linkage Region Growing (cont’) l second iteration of the Pong algorithm Digital Camera and Computer Vision Laboratory 68
10. 3. 2 Hybrid-Linkage Region Growing (cont’) l third iteration of the Pong algorithm Digital Camera and Computer Vision Laboratory 69
10. 3. 2 Hybrid-Linkage Region Growing (cont’) l after removing regions smaller than size 25 Digital Camera and Computer Vision Laboratory 70
l joke Digital Camera and Computer Vision Laboratory 71
10. 3. 3 Centroid-Linkage Region Growing l In centroid-linkage region growing, the image is scanned in some predetermined manner, such as left right, top-bottom. A pixel’s value is compared with the mean of an already existing but not necessarily completed neighboring segment. If its value and the segment’s mean value are close enough, then the pixel is added to the segment and the segment’s mean is updated. Digital Camera and Computer Vision Laboratory 72
10. 3. 3 Centroid-Linkage Region Growing (cont’) l If more than one region is close enough, then it is added to the closest region. However, if the means of the two competing regions are close enough, the two regions are merged and the pixel is added to the merged region. If no neighboring region has a close-enough mean, then a new segment is established having the given pixel’s value as it's first member. Digital Camera and Computer Vision Laboratory 73
10. 3. 3 Centroid-Linkage Region Growing (cont’) l caption of Fig 10. 33 explains and the figure illustrates the geometry Digital Camera and Computer Vision Laboratory 74
10. 3. 3 Centroid-Linkage Region Growing (cont’) l second image of the F-15 bulkhead Digital Camera and Computer Vision Laboratory 75
10. 3. 3 Centroid-Linkage Region Growing (cont’) l One-pass centroid-linkage segmentation Digital Camera and Computer Vision Laboratory 76
10. 3. 3 Centroid-Linkage Region Growing (cont’) l Two-pass centroid segmentation of the bulkhead image Digital Camera and Computer Vision Laboratory 77
10. 4 Hybrid-Linkage Combinations l l l centroid linkage, hybrid linkage: can be combined to use relative strengths single linkage strength: boundaries are spatially accurate single linkage weakness: edge gaps result in excessive merging centroid linkage strength: to place boundaries in weak gradient area One-pass combined centroid and hybrid-linkage segmentation of bulkhead Digital Camera and Computer Vision Laboratory 78
10. 4 Hybrid-Linkage Combinations (cont’) Digital Camera and Computer Vision Laboratory 79
10. 4 Hybrid-Linkage Combinations (cont’) l Two-pass combined centroid and hybridlinkage segmentation Digital Camera and Computer Vision Laboratory 80
l joke Digital Camera and Computer Vision Laboratory 81
10. 5 Spatial Clustering l l Spatial-clustering: combining clustering with spatial region growing Spatial-clustering: combine histogram-modeseeking with region growing or spatial-linkage technique Digital Camera and Computer Vision Laboratory 82
10. 6 Split and Merge l A splitting method for segmentation begins with the entire image as the initial segment. Then the method successively splits each of its current segments into quarters if the segment is not homogeneous enough; that is, if the difference between the largest and smallest gray level intensities is large. A merging method starts with an initial segmentation and successively merges regions that are similar enough. Digital Camera and Computer Vision Laboratory 83
10. 6 Split and Merge l Splitting method Digital Camera and Computer Vision Laboratory 84
10. 6 Split and Merge l Merging method Digital Camera and Computer Vision Laboratory 85
10. 6 Split and Merge (cont’) l Split-and-merge segmentation of the bulkhead image Digital Camera and Computer Vision Laboratory 86
10. 7 Rule-Based Segmentation l l Nazif and Levine rule-based segmentation Advantage : easier to try different concepts without reprogramming The knowledge in the system is not application domain specific, but is generalpurpose, scene-independent knowledge about images, grouping criteria Different types of data entries allowed Digital Camera and Computer Vision Laboratory 87
10. 7 Rule-Based Segmentation (cont’) Digital Camera and Computer Vision Laboratory 88
10. 7 Rule-Based Segmentation (cont’) l numerical descriptive features that can be associated with condition Digital Camera and Computer Vision Laboratory 89
Digital Camera and Computer Vision Laboratory 90
10. 7 Rule-Based Segmentation (cont’) l numerical spatial features that can be associated with condition Digital Camera and Computer Vision Laboratory 91
10. 7 Rule-Based Segmentation (cont’) l logical features that can be associated with condition Digital Camera and Computer Vision Laboratory 92
Digital Camera and Computer Vision Laboratory 93
10. 7 Rule-Based Segmentation (cont’) l Area, region, and line analyzer actions Digital Camera and Computer Vision Laboratory 94
Digital Camera and Computer Vision Laboratory 95
10. 7 Rule-Based Segmentation (cont’) l Focus-of-attention and supervisor actions Digital Camera and Computer Vision Laboratory 96
Digital Camera and Computer Vision Laboratory 97
10. 7 Rule-Based Segmentation (cont’) l examples of rules from the Nazif and Levine system Digital Camera and Computer Vision Laboratory 98
Digital Camera and Computer Vision Laboratory 99
l joke Digital Camera and Computer Vision Laboratory 100
10. 8 Motion-Based Segmentation l l In time-varying image analysis the data are a sequence of images instead of a single image. One paradigm under which such a sequence can arise is with a stationary camera viewing a scene containing moving objects. In each frame of the sequence after the first frame the moving objects appear in different positions of the image from those in the previous frame. Thus the motion of the objects creates a change in the images that can be used to help 101 locate the moving objects. Digital Camera and Computer Vision Laboratory
10. 8 Motion-Based Segmentation (cont’) l l l (a) image ti (b) image tj (c) difference image Gonzalez, Digital Image Processing, First Edition Fig 7. 40 Digital Camera and Computer Vision Laboratory 102
Digital Camera and Computer Vision Laboratory 103
10. 9 Summary l l Single-linkage region-growing schemes: simplest and most prone to errors Split-and-merge: large memory usage, excessively blocky region boundaries Digital Camera and Computer Vision Laboratory 104
l Joke Digital Camera and Computer Vision Laboratory 105
- Slides: 105