Artificial Intelligence Bai Xiao Lecture Two Camera and
Artificial Intelligence Bai Xiao
Lecture Two Camera and Image Representation • Road map: – Digital Equipment, camera, video recorder – Digital Images – From 2 D to 3 D
Camera • The pinhole projection model – Qualitative properties – Perspective projection matrix • Cameras with lenses – Depth of focus – Field of view – Lens aberrations • Digital cameras – Types of sensors – Color
Pinhole camera • Add a barrier to block off most of the rays – This reduces blurring – The opening is known as the aperture Slide by Steve Seitz
Pinhole camera model • Pinhole model: – Captures pencil of rays – all rays through a single point – The point is called Center of Projection (focal point) – The image is formed on the Image Plane Slide by Steve Seitz
Dimensionality Reduction Machine (3 D to 2 D) 3 D world 2 D image What have we lost? • Angles • Distances (lengths) Slide by A. Efros Figures © Stephen E. Palmer, 2002
Projection properties • • • Points project to points Lines project to lines Planes project to the whole or half image Angles are not preserved Degenerate cases – Line through focal point projects to a point. – Plane through focal point projects to line
Vanishing points • Each direction in space has its own vanishing point – All lines going in that direction converge at that point – Exception: directions parallel to the image plane • All directions in the same plane have vanishing points on the same line
Pinhole camera in 2 D X’ = (f’ / Z) X
Distant objects are smaller Size is inversely proportional to distance .
The equation of perspective projection
Weak perspective Assume object points are all at same depth -z 0
Why not use pinhole cameras? If pinhole is too big many directions are averaged, blurring the image Pinhole too smalldiffraction effects blur the image Generally, pinhole cameras are dark, because a very small set of rays from a
Pinhole model with a single lens A lens follows the pinhole model for objects that are in focus.
An out-of-focus lens An image plane at the wrong distance means that rays from different parts of the lens create a blurred region (the “point spread function”).
Lens systems • A good camera lens may contain 15 elements and cost a thousand dollars • The best modern lenses may contain aspherical elements
Building a real camera
Adding a lens • A lens focuses light onto the film – Rays passing through the center are not deviated Slide by Steve Seitz
Adding a lens focal point f • A lens focuses light onto the film – Rays passing through the center are not deviated – All parallel rays converge to one point on a plane located at the focal length f Slide by Steve Seitz
Adding a lens “circle of confusion” • A lens focuses light onto the film – There is a specific distance at which objects are “in focus” • other points project to a “circle of confusion” in Slide by Steve Seitz the image
Thin lens formula D’ D f Frédo Durand’s slide
Thin lens formula Similar triangles everywhere! D’ D f Frédo Durand’s slide
Thin lens formula Similar triangles everywhere! D’ y’/y = D’/D D f y y’ Frédo Durand’s slide
Thin lens formula Similar triangles everywhere! D’ D y’/y = D’/D y’/y = (D’-f)/f f y y’ Frédo Durand’s slide
Thin lens formula 1 +1 =1 D’ D f D’ D Any point satisfying the thin lens equation is in focus. f Frédo Durand’s slide
Depth of Field http: //www. cambridgeincolour. com/tutorials/depth-of-field. htm Slide by A. Efros
How can we control the depth of field? • Changing the aperture size affects depth of field – A smaller aperture increases the range in which the object is approximately in focus – But small aperture reduces amount of light – need to increase exposure Slide by A. Efros
Field of View Slide by A. Efros
Field of View Slide by A. Efros
Field of View f f FOV depends on focal length and size of the camera retina Smaller FOV = larger Focal Length Slide by A. Efros
Field of View / Focal Length Large FOV, small f Camera close to car Small FOV, large f Camera far from the car Sources: A. Efros, F. Durand
Same effect for faces wide-angle standard telephoto Source: F. Durand
Digital camera • A digital camera replaces film with a sensor array – Each cell in the array is light-sensitive diode that converts photons to electrons – Two common types • Charge Coupled Device (CCD) • Complementary metal oxide semiconductor (CMOS) – http: //electronics. howstuffworks. com/digital-camera. htm Slide by Steve Seitz
CCD vs. CMOS • CCD: transports the charge across the chip and reads it at one corner of the array. An analog-to-digital converter (ADC) then turns each pixel's value into a digital value by measuring the amount of charge at each photosite and converting that measurement to binary form • CMOS: uses several transistors at each pixel to amplify and move the charge using more traditional wires. The CMOS signal is digital, so it needs no ADC. http: //electronics. howstuffworks. com/digital-camera. htm http: //www. dalsa. com/shared/content/pdfs/CCD_vs_CMOS_Litwiller_2005. pdf
Color sensing in camera: Color filter array Bayer grid Estimate missing components from neighboring values (demosaicing) Why more green? Human Luminance Sensitivity Function Source: Steve Seitz
Assignment 1: Demosaicing
• Digital camera artifacts Noise • low light is where you most notice noise • light sensitivity (ISO) / noise tradeoff • stuck pixels • In-camera processing • oversharpening can produce halos • Compression • JPEG artifacts, blocking • Blooming • charge overflowing into neighboring pixels • Color artifacts • purple fringing from microlenses, • white balance Slide by Steve Seitz
Historic milestones • Pinhole model: Mozi (470 -390 BCE), Aristotle (384 -322 BCE) • Principles of optics (including lenses): Alhacen (965 -1039 CE) Camera obscura: Leonardo da Vinci (1452 -1519), Johann Zahn (1631 -1707) First photo: Joseph Nicephore Niepce (1822) Daguerréotypes (1839) Photographic film (Eastman, 1889) Cinema (Lumière Brothers, 1895) Color Photography (Lumière Brothers, 1908) Television (Baird, Farnsworth, Zworykin, 1920 s) First consumer camera with CCD: Sony Mavica (1981) First fully digital camera: Kodak DCS 100 (1990) • • • Alhacen’s notes Niepce, “La Table Servie, ” 1822 CCD chip
Image Acquisition and Representation • Digital Equipment – – To capture digital images, Camera, video recorder, scanner, mobile … Digital Camera – share similar function with 35 mm film camera Sensing Light
Image Acquisition and Representation • Digital Camera – The digital camera is very much like a film camera, except on the image plane, instead of chemical film reacting to light, tiny solid state cells convert light energy into electrical charge.
Image Acquisition and Representation • Digital Camera – The geometry of image formation can be conceptualized as the projection of each point of the 3 D scene through the center of projection or lens center onto the image plane. The intensity at the image point is related to the intensity radiating from the 3 D surface point.
Image Acquisition and Representation • Digital Video Recorder – Video cameras creating imagery for human consumption record sequences of images at a rate of 30 per second, enabling a representation of object motion over time in addition to the spatial features represented in the single images or frames. – Frames of video sequence are separated by markers and some image compression scheme is usually used to reduce the amount of data. – rmvb, avi, mpeg, …
Image Acquisition and Representation • Image Representation – digital image – Image representation – Matrix N×N or 3×N×N – Some major reasons : picture (rectangular), computer (computation), display (monitor). – Picture function (图像函数) : The picture function is a mathematical model that is often used in analysis where it is fruitful to consider the image as a function of two variables. All of functional analysis is then available for analyzing images. – 图像函数是一个数学模型,用于分析图像,一般把图像看成是双变 量函数。这样,分析图像就可以用所有的函数分析方法。
Image Acquisition and Representation • Image Representation – digital image – The digital image is merely a 2 D rectangular array of discrete values. Both image space and intensity range are quantized into a discrete set of values, permitting the image to be stored in a 2 D computer memory structure. It is common to record intensity as an 8 -bit(1 -byte) number which allows values of 0 to 255. – 图像空间位置和强度值都被量化程离散的数值,这样图像就能够存 储在 2 D计算机存储器中。一般像素强度用 8位(1字节)来表示,取 值范围 0到 255。
Image Acquisition and Representation • Image Representation – digital image
Image Acquisition and Representation • Image Representation – digital image and analog image – Digital image: A digital image is a 2 D image I[r, c] represented by a discrete 2 D array of intensity samples, each of which is represented using a limited precision. Discrete – Analog image: An analog image is a 2 D image F(x, y) which has infinite precision in spatial parameters x and y and infinite precision in intensity at each spatial point (x, y). Continuous
Image Acquisition and Representation • Image Representation – some common types of digital images – A grey scale image is a monochrome digital image I[r, c] with one intensity value per pixel. – A multispectral image is a 2 D image M[x, y] which has a vector of values at each spatial point or pixel. If the image is actually a color image, then the vector has 3 elements. – A binary image is a digital image with all pixel values 0 or 1. – A labeled image is a digital image L[r, c] whose pixel values are symbols from a finite alphabet. The symbol value of a pixel denotes the outcome of some decision made for that pixel. Related concept are thematic image and pseudo-colored image. (主题图像和伪色彩图像)
Image Acquisition and Representation • Image Representation – some common types of digital images – A grey scale image is a monochrome digital image I[r, c] with one intensity value per pixel. – A multispectral image is a 2 D image M[x, y] which has a vector of values at each spatial point or pixel. If the image is actually a color image, then the vector has 3 elements. – A binary image is a digital image with all pixel values 0 or 1. – A labeled image is a digital image L[r, c] whose pixel values are symbols from a finite alphabet. The symbol value of a pixel denotes the outcome of some decision made for that pixel. Related concept are thematic image and pseudo-colored image. (主题图像和伪色彩图像)
Image Acquisition and Representation • Image Representation – some common types of digital images
Image Acquisition and Representation • Digital Image Formats – Digital images are quite popular however, there are different formats for digital images now. – Image file header + Image data – An image compression method is lossless if a decomposition method exists to precisely recover the original image representation. Otherwise, the compression method is lossy. – PGM, GIF, TIFF, JPEG, …
Image Acquisition and Representation • Problems with digital image representation – Computer Vision try to understand high level information contained in the images. – The complex real world environment cause the images to contain too richness information. – There also noise problems exist – hardware, light, scale, rotation…
Image Acquisition and Representation • From 2 D to 3 D – The human vision system perceives the structure of the 3 D world by integrating several different cues. However, 2 D image is in 2 dimensional space. How can we get 3 D information from images? – Even for 2 D images, there are many 3 D cues inside the 2 D images.
Image Acquisition and Representation • 3 D cues – Interposition: objects that are closer occlude parts of objects that are farther away, recognition of occlusions gives relative depth. – Relative : The image of a car 20 meters away will be much smaller than the image of a same car 10 meters away. – Texture gradient: The texture of surfaces changes with both the distance from the viewer and the surface orientation.
Image Acquisition and Representation • Use more images to extract 3 D information (stereo vision)
- Slides: 54