Introduction to Computer and Human Vision Shimon Ullman

  • Slides: 43
Download presentation
Introduction to Computer and Human Vision Shimon Ullman, Ronen Basri, Michal Irani Assistants: Tal

Introduction to Computer and Human Vision Shimon Ullman, Ronen Basri, Michal Irani Assistants: Tal Hassner <hassner@wisdom. weizmann. ac. il> Eli Shechtman <elishe @wisdom. weizmann. ac. il>

Misc. . . · Course website: www. wisdom. weizmann. ac. il/~hassner/cv 0203 · To

Misc. . . · Course website: www. wisdom. weizmann. ac. il/~hassner/cv 0203 · To be added to course mailing-list: send email to <hassner@wisdom…. > · Other recommended courses (for credit): - Basic Topics - Statistical Machine Learning · Vision & Robotics Seminar (not for credit): Thursdays at 11: 00 -12: 00 (Ziskind 1) send email <leah@wisdom…> ask to be added to “seminar 13” mailing list

Applications: - Manufacturing and inspection; QA · Hownavigation is an image formed? - Robot

Applications: - Manufacturing and inspection; QA · Hownavigation is an image formed? - Robot (geometry and photometry) - Autonomous vehicles - Guiding tools for blind · What kind operations - Security and of monitoring can we apply to images? - Object/face recognition; OCR. - Medical Applications · What do images - Visualization; NVStell us aboutcommunication the world? - Visual (analysis & interpretation) - Digital libraries and video search - Video manipulation and editing

Tentative Schedule Lessons 1 -3 (Michal): Basic Image Processing Lessons 4 -6 (Ronen): Stereo

Tentative Schedule Lessons 1 -3 (Michal): Basic Image Processing Lessons 4 -6 (Ronen): Stereo and Structure from Motion Lessons 7 -9 (Michal): Motion and video analysis Lesson 10 (Ronen): Image Segmentation Lesson 11 (Ronen): Photometry Lesson 12 (Shimon): Lessons 13 -14 (Shimon): Object recognition Human Vision · 3 programming exercises (MATLAB) -· 3 -4 theoretical exercises -· EXAM CAN SUBMIT IN PAIRS MUST SUBMIT INDIVIDUALLY

Digital Images World Camera Digitizer Digital Image Formation: (i) What determines where the image

Digital Images World Camera Digitizer Digital Image Formation: (i) What determines where the image of a 3 D point appears on the 2 D image? (ii) What determines how bright that image point is? (iii) How is a digital image represented? (iv) Some simple operations on 2 D images? today

Digital Images World Camera Digital Image Digitizer 0 10 10 15 0 0 100

Digital Images World Camera Digital Image Digitizer 0 10 10 15 0 0 100 125 130 0 35 100 150 80 50 0 15 70 10 20 20 0 15 70 15 50 120 110 130 110 5 10 20 0 50 0 70 80 0 50 50 20 250 PIXEL (picture element) Typically: 0 = black 255 = white

64 65 65 66 66 59 60 62 63 60 62 66 66 64

64 65 65 66 66 59 60 62 63 60 62 66 66 64 63 62 62 64 69 100 149 151 176 182 179 68 97 145 148 175 183 181 70 95 142 146 176 185 184 68 90 135 140 172 184 64 84 129 134 168 181 182 62 88 130 128 166 185 180 60 85 127 125 163 183 178 58 81 122 120 160 181 176 58 78 117 159 180 176

Grayscale Image x = 58 59 y = 41 42 43 44 45 46

Grayscale Image x = 58 59 y = 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 210 206 201 216 221 209 204 214 209 208 207 208 204 200 205 209 196 207 206 214 212 215 209 210 205 206 203 210 60 61 62 63 64 65 66 67 68 69 70 71 72 204 203 192 211 224 213 215 214 205 211 209 203 199 202 197 201 193 194 199 208 207 205 203 199 209 236 203 197 195 198 202 196 194 191 208 204 202 217 195 188 199 247 210 213 207 193 190 180 196 186 194 203 197 143 207 156 208 220 204 191 172 187 174 183 188 183 196 71 56 69 57 56 173 214 188 196 185 177 185 190 181 64 63 65 69 63 64 60 69 86 149 209 187 183 173 80 84 58 53 57 55 60 59 62 66 72 55 62 66 71 63 90 62 239 58 221 75 196 122 186 105 54 53 52 77 46 51 76 49 87 55 64 68 61 63 62 54 61 53 49 97 62 51 56 57 55 52 61 58 58 57 57 58 62 51 60 50 62 61 58 106 56 48 49 55 52 56 60 48 45 56 93 52 51 56 60 60 64 66 64 63

Three types of images: – Gray-scale images I(x, y) [0. . 255] – Binary

Three types of images: – Gray-scale images I(x, y) [0. . 255] – Binary images I(x, y) {0 , 1} – Color images IR(x, y) IG(x, y) IB(x, y)

Color Image

Color Image

Effects of down-sampling (reducing number of pixels) 128 x 128 64 x 64 32

Effects of down-sampling (reducing number of pixels) 128 x 128 64 x 64 32 x 32 16 x 16 8 x 8 4 x 4

Effects of reducing number of gray levels 256 gray levels (8 bits/pixel) 4 gray

Effects of reducing number of gray levels 256 gray levels (8 bits/pixel) 4 gray levels (2 bits/pixel) 16 gray levels (4 bits/pixel) 8 gray levels (3 bits/pixel) 2 gray levels (1 bit/pixel) BINARY IMAGE

The Image Histogram Occurrence (# of pixels) Histogram = The gray-level distribution: H(k) =

The Image Histogram Occurrence (# of pixels) Histogram = The gray-level distribution: H(k) = #pixels with gray-level k Normalized histogram: Hnorm(k)=H(k)/N (N = # pixels in the image) Continuous probability density function: Gray Level

The Image Histogram (Cont. ) PI(k) 1 k PI(k) 1 0. 5 k PI(k)

The Image Histogram (Cont. ) PI(k) 1 k PI(k) 1 0. 5 k PI(k) 0. 1 k

Histogram Stretching PI(k) 0. 1 k 0. 5 k

Histogram Stretching PI(k) 0. 1 k 0. 5 k

Histogram Equalization k k

Histogram Equalization k k

Histogram Equalization Original Equalized

Histogram Equalization Original Equalized

Histogram Equalization 3000 2500 2000 1500 1000 500 0 0 50 100 150 Original

Histogram Equalization 3000 2500 2000 1500 1000 500 0 0 50 100 150 Original 200 250 0 0 50 100 150 Equalized 200 250

Histogram Specification Transforms an image so that its histogram matches that of another image

Histogram Specification Transforms an image so that its histogram matches that of another image Aa Ab k (e. g. , for comparing two images of the same scene acquired under different lighting condition) k

Image Enhancement 1) Gray value (histogram) Domain - Histogram stretching - Histogram equalization -

Image Enhancement 1) Gray value (histogram) Domain - Histogram stretching - Histogram equalization - Histogram specification - Gamma correction etc. . . 2) Spatial Domain 3) Frequency Domain noisy image (salt & pepper noise)

Spatial Operations Replace center pixel with average/median level: (averaging mask; weighted mask; median filter…)

Spatial Operations Replace center pixel with average/median level: (averaging mask; weighted mask; median filter…) Examples of neighborhoods S: 3 x 3 5 x 5 S = neighborhood of pixel (x, y) M = number of pixels in neighborhood S e. g. , g(x, y) = 1/M f(n, m) S (n, m) in S

Noise Cleaning Salt & Pepper Noise 3 X 3 Average 5 X 5 Average

Noise Cleaning Salt & Pepper Noise 3 X 3 Average 5 X 5 Average 7 X 7 Average Median

Noise Cleaning Salt & Pepper Noise 3 X 3 Average 5 X 5 Average

Noise Cleaning Salt & Pepper Noise 3 X 3 Average 5 X 5 Average 7 X 7 Average Median

Other spatial filters Are strong brightness variations always noise…?

Other spatial filters Are strong brightness variations always noise…?

Edge Detection

Edge Detection

Line Edge gray value Edge Types x Step Edge edge gray value edge x

Line Edge gray value Edge Types x Step Edge edge gray value edge x edge

1 D image f(x) gray value Edge Detection by Differentiation x 1 st derivative

1 D image f(x) gray value Edge Detection by Differentiation x 1 st derivative f'(x) |f'(x)| Edge Pixels: |f'(x)| > Threshold threshold

Original image x derivative y derivative Gradient magnitude

Original image x derivative y derivative Gradient magnitude

Edge Detection Image Vertical edges Horizontal edges

Edge Detection Image Vertical edges Horizontal edges

Edge Detection Image

Edge Detection Image

Image Sharpening Blurry Image Laplacian Sharpened Image Also Laplacian; Zero-crossings; Edge sharpening; etc….

Image Sharpening Blurry Image Laplacian Sharpened Image Also Laplacian; Zero-crossings; Edge sharpening; etc….

The End. . . · Exercise#1: Noise Cleaning -- on course website (+ Matlab

The End. . . · Exercise#1: Noise Cleaning -- on course website (+ Matlab tutorial) DUE: Nov. 10 (in 2 weeks) · Course mailing list: Send email to <hassner@wisdom…. > · Vision & Robotics Seminar: send email <leah@wisdom…> ask to be added to “seminar 13” mailing list

Panoramic Mosaic Image Original video clip Generated Mosaic image

Panoramic Mosaic Image Original video clip Generated Mosaic image

Video Removal Original Outliers Synthesized

Video Removal Original Outliers Synthesized

Image Segmentation

Image Segmentation

Image Segmentation

Image Segmentation

Photometric Stereo

Photometric Stereo

Photometric Stereo

Photometric Stereo