EE 4780 Introduction to Computer Vision Introduction Bahadir
EE 4780: Introduction to Computer Vision Introduction Bahadir K. Gunturk
EE 4780 n n n Instructor: Bahadir K. Gunturk Office: EE 225 Email: bahadir@ece. lsu. edu Tel: 8 -5621 Office Hours: MW 10: 00 – 12: 00 Bahadir K. Gunturk 2
EE 4780 n n We will learn the fundamentals of digital image processing and computer vision. Lecture slides, problems sets, solutions, study materials, etc. will be posted on the class website. [www. ece. lsu. edu/gunturk/EE 4780] Textbook is not required. References: q Gonzalez/Woods, Digital Image Processing, Prentice-Hall, 2/e. Forsyth/Ponce, Computer Vision: A Modern Approach, Prentice-Hall. Duda, Hart, and Stork, “Pattern Classification, ” John Wiley&Sons, 2001. Shapiro/Stockman, Computer Vision, Prentice-Hall. q Horn, “Robot Vision, ” MIT Press, 1986. q q q Bahadir K. Gunturk 3
Grading Policy n Your grade will be based on q q q n Problem Sets: 30% Midterm: 30% Final: 40% Problem Sets q q q Mini projects: Theoretical problems and MATLAB assignments 4 -5 Problem Sets Individually or in two-person teams Bahadir K. Gunturk 4
Digital Image Acquisition Sensor array n n n Bahadir K. Gunturk When photons strike, electron-hole pairs are generated on sensor sites. Electrons generated are collected over a certain period of time. The number of electrons are converted to pixel values. (Pixel is short for picture element. ) 5
Digital Image Acquisition Two types of quantization: 1. There are finite number of pixels. (Spatial resolution) 2. The amplitude of pixel is represented by a finite number of bits. (Gray-scale resolution) Bahadir K. Gunturk 6
Digital Image Acquisition Take a look at this cross section Bahadir K. Gunturk 7
Digital Image Acquisition • • • Bahadir K. Gunturk 256 x 256 - Found on very cheap cameras, this resolution is so low that the picture quality is almost always unacceptable. This is 65, 000 total pixels. 640 x 480 - This is the low end on most "real" cameras. This resolution is ideal for e-mailing pictures or posting pictures on a Web site. 1216 x 912 - This is a "megapixel" image size -- 1, 109, 000 total pixels -- good for printing pictures. 1600 x 1200 - With almost 2 million total pixels, this is "high resolution. " You can print a 4 x 5 inch print taken at this resolution with the same quality that you would get from a photo lab. 2240 x 1680 - Found on 4 megapixel cameras -- the current standard -- this allows even larger printed photos, with good quality for prints up to 16 x 20 inches. 4064 x 2704 - A top-of-the-line digital camera with 11. 1 megapixels takes pictures at this resolution. At this setting, you can create 13. 5 x 9 inch prints with no loss of picture quality. 8
Image Resolution Don’t confuse image size and resolution. Bahadir K. Gunturk 9
Bit Depth – Grayscale Resolution 8 bits 7 bits 6 bits 5 bits Bahadir K. Gunturk 10
Bit Depth – Grayscale Resolution 4 bits 2 bits Bahadir K. Gunturk 3 bits 1 bit 11
Matrix Representation of Images n A digital image can be written as a matrix Bahadir K. Gunturk 12
Digital Color Images n Bahadir K. Gunturk 13
Color Displays CRT LCD Polarize to control the amount of light passed. Bahadir K. Gunturk 14
Video n = vertical position = horizontal position = frame number ~24 frames per second. Bahadir K. Gunturk 15
Why do we process images? n n n To facilitate their storage and transmission To prepare them for display or printing To enhance or restore them To extract information from them To hide information in them Bahadir K. Gunturk 16
Image Processing Example n Image Restoration Original image Bahadir K. Gunturk Blurred Restored by Wiener filter 17
Image Processing Example n Noise Removal Noisy image Bahadir K. Gunturk Denoised by Median filter 18
Image Processing Example n Image Enhancement Histogram equalization Bahadir K. Gunturk 19
Image Processing Example n Artifact Reduction in Digital Cameras Original scene Bahadir K. Gunturk Captured by a digital camera Processed to reduce artifacts 20
Image Processing Example n Image Compression Original image 64 KB Bahadir K. Gunturk JPEG compressed 15 KB JPEG compressed 9 KB 21
Image Processing Example n Object Segmentation “Rice” image Bahadir K. Gunturk Edges detected using Canny filter 22
Image Processing Example n Resolution Enhancement Bahadir K. Gunturk 23
Image Processing Example n Watermarking Original image Watermarked image Generate watermark Hidden message Secret key Bahadir K. Gunturk 24
Image Processing Example n Face Recognition Search in the database Surveillance video Bahadir K. Gunturk 25
Image Processing Example n Fingerprint Matching Bahadir K. Gunturk 26
Image Processing Example n Segmentation Bahadir K. Gunturk 27
Image Processing Example n Texture Analysis and Synthesis Photo Pattern repeated Bahadir K. Gunturk Computer generated 28
Image Processing Example n Face detection and tracking http: //www-2. cs. cmu. edu/~har/faces. html Bahadir K. Gunturk 29
Image Processing Example n Face Tracking Bahadir K. Gunturk 30
Image Processing Example n Object Tracking Bahadir K. Gunturk 31
Image Processing Example n Virtual Controls Bahadir K. Gunturk 32
Image Processing Example n Visually Guided Surgery Bahadir K. Gunturk 33
Cameras n n First camera was invented in 16 th century. It used a pinhole to focus light rays onto a wall or translucent plate. n n Bahadir K. Gunturk Take a box, prick a small hole in one of its sides with a pin, and then replace the opposite side with a translucent plate. Place a candle on the pinhole side, you will see an inverted image of the candle on the translucent plate. 34
Perspective Projection n Perspective projection equations Bahadir K. Gunturk 35
Pinhole Camera Model n n If the pinhole were really reduced to a point, exactly one light ray would pass through each point in the image plane. In reality, each point in the image place collects light from a cone of rays. Bahadir K. Gunturk 36
Pinhole Cameras Pinhole too big many directions are averaged, blurring the image Pinhole too small diffraction effects blur the image Bahadir K. Gunturk 37
Cameras With Lenses n n Most cameras are equipped with lenses. There are two main reasons for this: q q To gather light. For an ideal pinhole, a single light ray would reach point the image plane. Real pinholes have a finite size, so each point in the image plane is illuminated by a cone of light rays. The larger the hole, the wider the cone and the brighter the image => blurry pictures. Shrinking the pinhole produces sharper images, but reduces the amount of light and may introduce diffraction effects. To keep the picture in sharp focus while gathering light from a large area. Bahadir K. Gunturk 38
Compound Lens Systems Bahadir K. Gunturk 39
Real Lenses n Rays may not focus at a single point. Spherical aberration can be eliminated completely by designing aspherical lenses. Bahadir K. Gunturk 40
Real Lenses n n The index of refraction is a function of wavelength. Light at different wavelengths follow different paths. Chromatic aberration Bahadir K. Gunturk 41
Real Lenses Chromatic Aberration Bahadir K. Gunturk 42
Real Lenses n Special lens systems using two or more pieces of glass with different refractive indeces can reduce or eliminate this problem. However, not even these lens systems are completely perfect and still can lead to visible chromatic aberrations. Bahadir K. Gunturk 43
Real Lenses n Barrel Distortion & Pincushion Distortion Causes of distortion Stop (Aperture) Chief ray (normal) Bahadir K. Gunturk 44
Real Lenses n Barrel Distortion & Pincushion Distorted Corrected http: //www. vanwalree. com/optics/distortion. html http: //www. dpreview. com/learn/? /Image_Techniques/Barrel_Distortion_Correction_01. htm Bahadir K. Gunturk 45
Real Lenses Vignetting effect in a two-lens system. The shaded part of the beam never reaches the second lens. The brightness drop in the image perimeter. Bahadir K. Gunturk 46
Real Lenses Optical vignetting example. Left: f/1. 4. Right: f/5. 6. f-number focal length to diameter ratio Bahadir K. Gunturk 47
Real Lenses Long exposure time Short exposure time Bahadir K. Gunturk 48
Real Lenses Flare Hood may prevent flares Bahadir K. Gunturk 49
Real Lenses Flare Bahadir K. Gunturk 50
Compound Lens Systems http: //www. dpreview. com/learn/? /glossary/ http: //www. cartage. org. lb/en/themes/Sciences/Physics/Optical/Lens. htm http: //www. vanwalree. com/optics. html Bahadir K. Gunturk 51
Digital Camera Pipeline (from Katz and Gentile) Auto-exposure algorithms measure brightness over discrete scene regions to compensate for overexposed or underexposed areas by manipulating shutter speed and/or aperture size. The net goals here are to maintain relative contrast between different regions in the image and to achieve a good overall quality. Bahadir K. Gunturk 52
Digital Camera Pipeline Auto-focus algorithms divide into two categories. Active methods use infrared or ultrasonic emitters/receivers to estimate the distance between the camera and the object being photographed. Passive methods, on the other hand, make focusing decisions based on the received image in the camera. Bahadir K. Gunturk 53
Digital Camera Pipeline Lens distortion correction This set of algorithms accounts for the physical properties of lenses that warp the output image compared to the actual scene the user is viewing. Different lenses can cause different distortions; for instance, wide-angle lenses create a "barrel distortion", while telephoto lenses create a "pincushion distortion“. Bahadir K. Gunturk 54
Digital Camera Pipeline Vignetting (shading distortion) reduces image brightness in the area around the lens. Chromatic aberration causes color fringes around an image. The media processor needs to mathematically transform the image in order to correct for these distortions. Bahadir K. Gunturk 55
Digital Camera Pipeline Sensor's output needs to be gamma-corrected to account for eventual display, as well as to compensate for nonlinearities in the sensor's capture response. Bahadir K. Gunturk 56
Digital Camera Pipeline Image stability compensation, or hand-shaking correction is another area of preprocessing. Here, the processor adjusts for the translational motion of the received image, often with the help of external transducers that relate the realtime motion profile of the sensor. Bahadir K. Gunturk 57
Digital Camera Pipeline White balance is another important stage of preprocessing. When we look at a scene, regardless of lighting conditions, our eyes tend to normalize everything to the same set of natural colors. For instance, an apple looks deep red to us whether we're indoors under fluorescent lighting, or outside in sunny weather. However, an image sensor's "perception" of color depends largely on lighting conditions, so it needs to map its acquired image to appear natural in its final output. This mapping can be done either manually or automatically. Bahadir K. Gunturk 58
Digital Camera Pipeline Demosaicking (Bayer interpolation) estimates missing color samples in singlechip cameras. Bahadir K. Gunturk 59
Digital Camera Pipeline In this stage, the interpolated RGB image is transformed to the targeted output color space (if not already in the right space). For compression or display to a television, this will usually involve an RGB YCb. Cr matrix transformation, often with another gamma correction stage to accommodate the target display. The YCb. Cr outputs may also be chroma subsampled at this stage to the standard 4: 2: 2 format for color bandwidth reduction with little visual impact. Bahadir K. Gunturk 60
Digital Camera Pipeline Postprocessing In this phase, the image is perfected via a variety of filtering operations before being sent to the display and/or storage media. For instance, edge enhancement, pixel thresholding for noise reduction, and color-artifact removal are all common at this stage. Bahadir K. Gunturk 61
Digital Camera Pipeline Display / Compress / Store Once the image itself is ready for viewing, the image pipe branches off in two different directions. In the first, the postprocessed image is output to the target display, usually an integrated LCD screen (but sometimes an NTSC/PAL television monitor, in certain camera modes). In the second, the image is sent to the media processor's compression algorithm, where industry-standard compression techniques (JPEG, for instance) are applied before the picture is stored locally in some storage medium (e. g. , Flash memory card). Bahadir K. Gunturk 62
- Slides: 62
