Image formation ECE 847 Digital Image Processing Stan
- Slides: 78
Image formation ECE 847: Digital Image Processing Stan Birchfield Clemson University
Cameras • First photograph due to Niepce • Basic abstraction is the pinhole camera – lenses required to ensure image is not too dark – various other abstractions can be applied F. Dellaert, http: //www. cc. gatech. edu/~dellaert/vision/html/materials. html
Image formation overview Image formation involves • geometry – path traveled by light • radiometry – optical energy flow • photometry – effectiveness of light to produce “brightness” sensation in human visual system • colorimetry – physical specifications of light stimuli that produce given color sensation • sensors – converting photons to digital form
Pinhole camera D. Forsyth, http: //luthuli. cs. uiuc. edu/~daf/bookpages/slides. html
Parallel lines meet: vanishing point • each set of parallel lines (=direction) meets at a different point – The vanishing point for this direction • Sets of parallel lines on the same plane lead to collinear vanishing points. – The line is called the horizon for that plane
Perspective projection Properties of projection: • Points go to points • Lines go to lines • Planes go to whole image • Polygons go to polygons • Degenerate cases k P – line through focal point to point – plane through focal point to line C f O j p q i Q F. Dellaert, http: //www. cc. gatech. edu/~dellaert/vision/html/materials. html
Perspective projection (cont. ) F. Dellaert, http: //www. cc. gatech. edu/~dellaert/vision/html/materials. html
Weak perspective projection • perspective effects, but not over the scale of individual objects • collect points into a group at about the same depth, then divide each point by the depth of its group D. Forsyth, http: //luthuli. cs. uiuc. edu/~daf/bookpages/slides. html
Weak perspective (cont. ) F. Dellaert, http: //www. cc. gatech. edu/~dellaert/vision/html/materials. html
Orthographic projection Let Z 0=1: F. Dellaert, http: //www. cc. gatech. edu/~dellaert/vision/html/materials. html
Pushbroom cameras
Pinhole size Pinhole too big many directions are averaged, blurring the image Pinhole too smalldiffraction effects blur the image Generally, pinhole cameras are dark, because a very small set of rays from a particular point hits the screen. D. Forsyth, http: //luthuli. cs. uiuc. edu/~daf/bookpages/slides. html
The reason for lenses D. Forsyth, http: //luthuli. cs. uiuc. edu/~daf/bookpages/slides. html
The thin lens focal points D. Forsyth, http: //luthuli. cs. uiuc. edu/~daf/bookpages/slides. html
Focusing http: //www. theimagingsource. com
Thick lens • thick lens has 6 cardinal points: – two focal points (F 1 and F 2) – two principal points (H 1 and H 2) – two nodal points (N 1 and N 2) • complex lens is formed by combining individual concave and convex lenses http: //physics. tamuk. edu/~suson/html/4323/thick. html D. Forsyth, http: //luthuli. cs. uiuc. edu/~daf/bookpages/slides. html
Complex lens All but the simplest cameras contain lenses which are actually composed of several lens elements http: //www. cambridgeincolour. com/tutorials/camera-lenses. htm
Choosing a lens • How to select focal length: – x=f. X/Z – f=x. Z/X • Lens format should be >= CCD format to avoid optical flaws at the rim of the lens http: //www. theimagingsource. com/en/resources/whitepapers/download/choosinglenswp. en. pdf
Lenses – Practical issues • standardized lens mount has two varieties: – C mount – CS mount • CS mount lenses cannot be used with C mount cameras http: //www. theimagingsource. com/en/resources/whitepapers/download/choosinglenswp. en. pdf
Spherical aberration perfect lens actual lens On a real lens, even parallel rays are not focused perfectly http: //en. wikipedia. org/wiki/Spherical_aberration
Chromatic aberration On a real lens, different wavelengths are not focused the same http: //en. wikipedia. org/wiki/Chromatic_aberration
Radial distortion straight lines are curved: uncorrected
Radial distortion (cont. ) Two types: barrel distortion pincushion distortion (more common) barrel pincushion http: //en. wikipedia. org/wiki/Image_distortion http: //foto. hut. fi/opetus/260/luennot/11/atkinson_6 -11_radial_distortion_zoom_lenses. jpg
Vignetting vignetting – reduction of brightness at periphery of image D. Forsyth, http: //luthuli. cs. uiuc. edu/~daf/bookpages/slides. html
Normalized Image coordinates 1 O u=X/Z = dimensionless ! P F. Dellaert, http: //www. cc. gatech. edu/~dellaert/vision/html/materials. html
Pixel units Pixels are on a grid of a certain dimension f O u=k f X/Z = in pixels ! P [f] = m (in meters) [k] = pixels/m F. Dellaert, http: //www. cc. gatech. edu/~dellaert/vision/html/materials. html
Pixel coordinates We put the pixel coordinate origin on topleft f O u=u 0 + k f X/Z P F. Dellaert, http: //www. cc. gatech. edu/~dellaert/vision/html/materials. html
Pixel coordinates in 2 D (0. 5, 0. 5) 480 640 (u 0, v 0) i (640. 5, 480. 5) j F. Dellaert, http: //www. cc. gatech. edu/~dellaert/vision/html/materials. html
Summary: Intrinsic Calibration skew 5 Degrees of Freedom ! F. Dellaert, http: //www. cc. gatech. edu/~dellaert/vision/html/materials. html
Camera Pose In order to apply the camera model, objects in the scene must be expressed in camera coordinates. Camera Coordinates World Coordinates Calibration target looks tilted from camera viewpoint. This can be explained as a difference in coordinate systems. F. Dellaert, http: //www. cc. gatech. edu/~dellaert/vision/html/materials. html
Rigid Body Transformations • Need a way to specify the six degrees-of-freedom of a rigid body. • Why are there 6 DOF? A rigid body is a collection of points whose positions relative to each other can’t change Fix one point, three DOF Fix second point, two more DOF (must maintain distance constraint) Third point adds one more DOF, for rotation around line F. Dellaert, http: //www. cc. gatech. edu/~dellaert/vision/html/materials. html
Notations • Superscript references coordinate frame • AP is coordinates of P in frame A • BP is coordinates of P in frame B • Example: F. Dellaert, http: //www. cc. gatech. edu/~dellaert/vision/html/materials. html
Translation F. Dellaert, http: //www. cc. gatech. edu/~dellaert/vision/html/materials. html
Translation • Using homogeneous coordinates, translation can be expressed as a matrix multiplication. • Translation is commutative F. Dellaert, http: //www. cc. gatech. edu/~dellaert/vision/html/materials. html
Rotation means describing frame A in The coordinate system of frame B F. Dellaert, http: //www. cc. gatech. edu/~dellaert/vision/html/materials. html
Rotation Orthogonal matrix! F. Dellaert, http: //www. cc. gatech. edu/~dellaert/vision/html/materials. html
Example: Rotation about z axis What is the rotation matrix? F. Dellaert, http: //www. cc. gatech. edu/~dellaert/vision/html/materials. html
Rotation in homogeneous coordinates • Using homogeneous coordinates, rotation can be expressed as a matrix multiplication. • Rotation is not commutative F. Dellaert, http: //www. cc. gatech. edu/~dellaert/vision/html/materials. html
Rigid transformations F. Dellaert, http: //www. cc. gatech. edu/~dellaert/vision/html/materials. html
Rigid transformations (con’t) • Unified treatment using homogeneous coordinates. F. Dellaert, http: //www. cc. gatech. edu/~dellaert/vision/html/materials. html
Projective Camera Matrix 5+6 DOF = 11 ! F. Dellaert, http: //www. cc. gatech. edu/~dellaert/vision/html/materials. html
Projective Camera Matrix 5+6 DOF = 11 ! F. Dellaert, http: //www. cc. gatech. edu/~dellaert/vision/html/materials. html
Columns & Rows of M m 2 P=0 O F. Dellaert, http: //www. cc. gatech. edu/~dellaert/vision/html/materials. html
Effect of Illumination (Subject 8 from the Yale face database due to P. Belhumeur et. al. ) Light source strength and direction has a dramatic impact on distribution of brightness in the image (e. g. shadows, highlights, etc. ) F. Dellaert, http: //www. cc. gatech. edu/~dellaert/vision/html/materials. html
Image formation • Light source emits photons • Absorbed, transmitted, scattered • fluorescence source Camera F. Dellaert, http: //www. cc. gatech. edu/~dellaert/vision/html/materials. html
Surfaces receives and emits • Incident light from lightfield • Act as a light source • How much light ? F. Dellaert, http: //www. cc. gatech. edu/~dellaert/vision/html/materials. html
Irradiance • Irradiance – amount of light falling on a surface patch • symbol=E, units = W/m 2 d. A F. Dellaert, http: //www. cc. gatech. edu/~dellaert/vision/html/materials. html
Radiosity • power leaving a point per area • symbol=B, units = W/m 2 d. A F. Dellaert, http: //www. cc. gatech. edu/~dellaert/vision/html/materials. html
Light = Directional • Light emitted varies w. direction F. Dellaert, http: //www. cc. gatech. edu/~dellaert/vision/html/materials. html
Steradians (Solid Angle) • 3 D analogue of 2 D angle A R F. Dellaert, http: //www. cc. gatech. edu/~dellaert/vision/html/materials. html
Steradians (cont’d) F. Dellaert, http: //www. cc. gatech. edu/~dellaert/vision/html/materials. html
Polar Coordinates F. Dellaert, http: //www. cc. gatech. edu/~dellaert/vision/html/materials. html
Intensity • Intensity – amount of light emitted from a point per steradian • symbol=I, units = W/sr F. Dellaert, http: //www. cc. gatech. edu/~dellaert/vision/html/materials. html
Irradiance and Intensity d. A F. Dellaert, http: //www. cc. gatech. edu/~dellaert/vision/html/materials. html
Radiance • Radiance – amount of light passing through an area d. A and • symbol=L, units = W x m-2 x sr-1 F. Dellaert, http: //www. cc. gatech. edu/~dellaert/vision/html/materials. html
Radiance is important • Response of camera/eye is proportional to radiance • Pixel values • Constant along a ray F. Dellaert, http: //www. cc. gatech. edu/~dellaert/vision/html/materials. html
Lightfield = Gibson optic array ! • 5 DOF: Position = 3 DOF, 2 DOF for direction F. Dellaert, http: //www. cc. gatech. edu/~dellaert/vision/html/materials. html
Lightfield Sampler F. Dellaert, http: //www. cc. gatech. edu/~dellaert/vision/html/materials. html
Lightfield Sample F. Dellaert, http: //www. cc. gatech. edu/~dellaert/vision/html/materials. html
Lambertian Emitters • • Lambertian = constant radiance More photons emitted straight up Oblique: see fewer photons, but area looks smaller Same brightness ! Total power is proportional to wedge area “Cosine law” Sun approximates Lambertian: Different angle, same brightness • Moon should be less bright at edges, as gets less light from sun. • Reflects more light at grazing angles than a Lambertian reflector F. Dellaert, http: //www. cc. gatech. edu/~dellaert/vision/html/materials. html
Radiance Emitted/Reflected • Radiance – amount of light emitted from a surface patch per steradian per area • foreshortened ! d. A F. Dellaert, http: //www. cc. gatech. edu/~dellaert/vision/html/materials. html
Calculating Radiosity If reflected light is not dependent on angle, then can integrate over angle: radiosity is an approximate radiometric unit F. Dellaert, http: //www. cc. gatech. edu/~dellaert/vision/html/materials. html
Example: Sun • • Power= 3. 91 1026 W Surface Area: 6. 07 1018 m 2 Power = Radiance. Area. L = 2. 05 107 W/m 2. sr Example from P. Dutre SIGGRAPH tutorial F. Dellaert, http: //www. cc. gatech. edu/~dellaert/vision/html/materials. html
Irradiance (again) • Integrate incoming radiance over hemisphere F. Dellaert, http: //www. cc. gatech. edu/~dellaert/vision/html/materials. html
Example: Sun F. Dellaert, http: //www. cc. gatech. edu/~dellaert/vision/html/materials. html
BRDF E L Symmetric in incoming and outgoing directions – this is the Helmholtz reciprocity principle F. Dellaert, http: //www. cc. gatech. edu/~dellaert/vision/html/materials. html
BRDF Example F. Dellaert, http: //www. cc. gatech. edu/~dellaert/vision/html/materials. html
Lambertian surfaces and albedo • For some surfaces, the DHR is independent of illumination direction too – cotton cloth, carpets, matte paper, matte paints, etc. • For such surfaces, radiance leaving the surface is independent of angle • Called Lambertian surfaces (same Lambert) or ideal diffuse surfaces • Use radiosity as a unit to describe light leaving the surface • DHR is often called diffuse reflectance, or albedo • for a Lambertian surface, BRDF is independent of angle, too. • Useful fact: F. Dellaert, http: //www. cc. gatech. edu/~dellaert/vision/html/materials. html
Specular surfaces • Another important class of surfaces is specular, or mirror-like. – radiation arriving along a direction leaves along the specular direction – reflect about normal – some fraction is absorbed, some reflected – on real surfaces, energy usually goes into a lobe of directions – can write a BRDF, but requires the use of funny functions F. Dellaert, http: //www. cc. gatech. edu/~dellaert/vision/html/materials. html
Lambertian + specular • Widespread model – all surfaces are Lambertian plus specular component • Advantages – easy to manipulate – very often quite close true • Disadvantages – some surfaces are not • e. g. underside of CD’s, feathers of many birds, blue spots on many marine crustaceans and fish, most rough surfaces, oil films (skin!), wet surfaces – Generally, very little advantage in modelling behaviour of light at a surface in more detail -- it is quite difficult to understand behaviour of L+S surfaces F. Dellaert, http: //www. cc. gatech. edu/~dellaert/vision/html/materials. html
Radiometry vs. Photometry http: //www. optics. arizona. edu/Palmer/rpfaq. htm F. Dellaert, http: //www. cc. gatech. edu/~dellaert/vision/html/materials. html
Sensors • CCD vs. CMOS • Types of CCDs: linear, interline, fullframe, frame-transfer • Bayer filters • progressive scan vs. interlacing • NTSC vs. PAL vs. SECAM • framegrabbers • blooming F. Dellaert, http: //www. cc. gatech. edu/~dellaert/vision/html/materials. html
Bayer color filter http: //en. wikipedia. org/wiki/Bayer_filter
Adobe’s plenoptic lens captures multiple views of the scene from slightly different viewpoints David Salesin and Todor Georgiev F. Dellaert, http: //www. cc. gatech. edu/~dellaert/vision/html/materials. html
Raw image from plenoptic system http: //livesmooth. istreamplanet. com/nvidia 100921/
Reconstructed image
Change the focus
Lenticular display http: //en. wikipedia. org/wiki/Lenticular_printing
- Zooming and shrinking in digital image processing
- Histogram processing in digital image processing
- Neighborhood processing
- Laplacian filter
- Point processing in image processing
- Point processing in digital image processing
- Gonzalez
- Image processing and analysis birchfield pdf
- 8476851447
- Translate
- Optimum notch filter in digital image processing
- Image compression models in digital image processing
- Key stage in digital image processing
- Lossless image compression matlab source code
- Image sharpening and restoration
- Geometric transformation in digital image processing
- Image transforms in digital image processing
- Maketform matlab
- Image restoration in digital image processing
- Perspective projection
- Explain various boundary descriptors
- Representation and description in digital image processing
- Image thresholding matlab
- Segmentation in digital image processing
- Pixel relationship in digital image processing
- Intensity transformation and spatial filtering
- Zooming and shrinking in digital image processing
- Imadjust
- Mixed adjacency in image processing
- Coordinate conventions in digital image processing
- Dam construction in digital image processing
- Digital image processing java
- Thresholding in digital image processing
- Segmentation in digital image processing
- Spatial filtering in digital image processing
- Representation and description in digital image processing
- Optimum global thresholding using otsu's method
- What is boundary descriptors in digital image processing
- Color slicing in image processing
- Let h and i be the size and intensity of the image
- Intensity transformation and spatial filtering
- Fourier transform in image processing
- Digital image processing
- Digital image processing
- Kl transform in digital image processing
- Digital image processing
- Jpeg in digital image processing
- Digital image processing
- Mach band effect in digital image processing
- Filetype:ppt
- Digital image processing
- Color fundamentals in digital image processing
- Image processing
- Digital path in image processing
- Digital path in image processing
- Lossless compression in digital image processing
- Intensity level slicing matlab
- Intensity transformation in digital image processing
- Digital image processing
- Houghpeaks matlab
- Fluorocein
- For a chain code 10103322
- Fundamental steps in digital image processing
- Digital image processing
- Full colour image processing
- Spatial resolution in digital image processing
- Digital image processing
- Image processing
- Color transformation in digital image processing
- Origins of digital image processing
- Pattern and pattern classes in image processing
- Raquel anido
- Digital image processing
- Hadamard transform in digital image processing
- Aliasing in image processing
- Shape numbers in digital image processing
- Digital processing
- Image processing
- Coordinate conventions in digital image processing