Image Formation Image Formation occurs when a sensor

  • Slides: 44
Download presentation
Image Formation • • Image Formation occurs when a sensor registers radiation. Mathematical models

Image Formation • • Image Formation occurs when a sensor registers radiation. Mathematical models of image formation: 1. 2. 3. 4. 5. 6. Image function model Geometrical model Radiometrical model Color model Spatial Frequency model Digitizing model E. G. M. Petrakis Image Formation 1

E. G. M. Petrakis Image Formation 2

E. G. M. Petrakis Image Formation 2

1. Image Function • Mathematical representation of a (digital) image. – Relates to digitization:

1. Image Function • Mathematical representation of a (digital) image. – Relates to digitization: conversion from continuous signal to discrete function • Black & White image • Color image • Multispectral image f = (f 1, f 2, …, fn) E. G. M. Petrakis Image Formation 3

2. Geometrical Model • Determines where in the image plane the projection of a

2. Geometrical Model • Determines where in the image plane the projection of a point will be located. – the projected image is inverted – (x, y, z) is projected on (x’, y’) – f: focal length E. G. M. Petrakis Image Formation 4

 • Avoid inversion by assuming that the image plane is in front of

• Avoid inversion by assuming that the image plane is in front of the center of projection – done automatically by cameras or by the human brain • Apply Euclidean geometry – x’ = x f /z and y’ = y f/z – depth z is lost ! E. G. M. Petrakis Image Formation 5

Depth Computation • Acquire a pair of images of the same scene using two

Depth Computation • Acquire a pair of images of the same scene using two cameras (or two images by a moving camera) • Two identical cameras separated in the x direction by a baseline distance b • The image planes are coplanar E. G. M. Petrakis Image Formation 6

 • A point is projected at two different positions on the two camera

• A point is projected at two different positions on the two camera planes – their displacement is called “disparity” E. G. M. Petrakis Image Formation 7

 • In certain systems (human eyes) the optical axes of the cameras intersect

• In certain systems (human eyes) the optical axes of the cameras intersect in space – for any angle there is a surface in space corresponding to d = 0. – the disparities may be d = 0, d < 0 or d > 0. E. G. M. Petrakis Image Formation 8

 • Epipolar constraint: even if the cameras are in arbitrary positions and orientation

• Epipolar constraint: even if the cameras are in arbitrary positions and orientation the projections lie on the intersection of camera - epipolar planes E. G. M. Petrakis Image Formation 9

 • Correspondence problem: detection of conjugate pairs in stereo images: – for each

• Correspondence problem: detection of conjugate pairs in stereo images: – for each point in the left image find the corresponding point in the right image – measure the similarity between points – the points to be matched should be distinctly different from their surrounding points – both region and edge features can be used in stereo matching – the epipolar constraint limits the search space for finding conjugate pairs. E. G. M. Petrakis Image Formation 10

3. Radiometrical Model • Measures the intensity of the reflected light at a point

3. Radiometrical Model • Measures the intensity of the reflected light at a point (x’, y’) of the image plane – it is determined by the physics of imaging • The proper term of image intensity is image irradiance but – intensity, brightness, gray value are also used • Image irradiance is the power per unit area of radiant energy falling into the image plane – Irradiance is incoming energy – Radiance is outgoing energy (from reflecting surface) E. G. M. Petrakis Image Formation 11

 • • The irradiance at point (x’, y’) of the image plane depends

• • The irradiance at point (x’, y’) of the image plane depends on the amount of energy radiated by points (x, y, z) in the scene Two factors determine the radiance emitted by a patch of scene surface: 1. The illumination falling on a surface (depends in its position relative to the distribution of light) 2. The fraction of incident illumination reflected by the surface (depends on surface properties e. g. , dull, flat, mirror-like etc. ) • E. G. M. Petrakis The reflectance of a surface is given by the Bi-directional Reflectance Distribution Function (BRDF) Image Formation 12

 • Scene Radiance – – Φ: light energy flux Α: area of source

• Scene Radiance – – Φ: light energy flux Α: area of source θ: angle (surface normal & direction of emission) dω: incremental solid angle E. G. M. Petrakis Image Formation 13

 • Image irradiance • Ideally, an imaging device should be calibrated so that

• Image irradiance • Ideally, an imaging device should be calibrated so that the variation in sensitivity as a function of a is removed. E. G. M. Petrakis Image Formation 14

4. Color Model • Visible light is an electromagnetic wave in the 400 nm

4. Color Model • Visible light is an electromagnetic wave in the 400 nm – 700 nm range • The light we see is combination of many wavelengths – spectra: the profile below E. G. M. Petrakis Image Formation 15

 • Each neuron on the retina is either a “rod” or a “cone”

• Each neuron on the retina is either a “rod” or a “cone” (rods are not sensitive to color). • Cons come in 3 types: red, green, blue – each responds differently to various frequencies of light. • Spectral response functions of cones: E. G. M. Petrakis Image Formation 16

 • The color signal to the brain is obtained by adding the responses

• The color signal to the brain is obtained by adding the responses of the 3 cones – the color signal consists of 3 numbers. – R, G, B sensors filter the scene radiance E(λ). – each sensor has a different spectral response S(λ). E. G. M. Petrakis Image Formation 17

 • CIE primaries: this figure shows the amounts of the 3 primaries needed

• CIE primaries: this figure shows the amounts of the 3 primaries needed to match all the wavelengths of the visible spectrum – the negative value indicates that some colors cannot be exactly produced by adding up the 3 primaries. E. G. M. Petrakis Image Formation 18

CIE XYZ • Based on the CIE primaries – negative values are transformed to

CIE XYZ • Based on the CIE primaries – negative values are transformed to positive – chromaticity values x=X/(X+Y+Z), y=Y/(X+Y+Z), z=Z/(X+Y+Z) – x+y+z=1: two values represent all colors E. G. M. Petrakis Image Formation 19

Chromaticity Diagram • Visible colors: points in the bell • Non-visible colors outside the

Chromaticity Diagram • Visible colors: points in the bell • Non-visible colors outside the bell • Primaries at edges • A white point at centre • Saturated colors along the radii from edge E. G. M. Petrakis Image Formation 20

Color Representation • Several methods – Hardware-oriented: defined to properties of devices (TV, printers)

Color Representation • Several methods – Hardware-oriented: defined to properties of devices (TV, printers) that reproduce colors (RGB, CMY etc. ) – User-Oriented: based on human perception of colors (HIS, L*u*v etc. ) – Colorimetric (CIE), Physiological (CIE XYZ, RGB), Psychological (HIS, L*u*v etc) E. G. M. Petrakis Image Formation 21

RGB Color Space • The most popular hardware oriented scheme • The colors form

RGB Color Space • The most popular hardware oriented scheme • The colors form a unit cube – r = R/(R+G+B) – g = G/(R+G+B) – b = B/(R+G+B) • RGB is good for acquisition and display but not for the perception of colors E. G. M. Petrakis Image Formation 22

CMY Color Space • Cyan, Magenta, Yellow are complements of Red, Green, Blue •

CMY Color Space • Cyan, Magenta, Yellow are complements of Red, Green, Blue • Obtained by subtracting light from white • For color printing • Conversion from RGB to CMY –R=1–C –G=1–M –B=1–Y E. G. M. Petrakis Image Formation 23

Munsell Color Space • Represented in cylindrical coordinates based on – Brightness: vertical axis

Munsell Color Space • Represented in cylindrical coordinates based on – Brightness: vertical axis – Hue: angular displacement – Saturation: cylindrical radius E. G. M. Petrakis Image Formation 24

Color Definitions • Brightness: intensity of color, average intensity over all wavelengths • Hue:

Color Definitions • Brightness: intensity of color, average intensity over all wavelengths • Hue: is roughly proportional to the average wavelength of the color percept • Saturation: amount of white light in color – highly saturated colors have no white – deep red has S=1, pinks have S 0 • P = SH+(1–S)W: Think of a color P as an additive mixture W and H where S controls the proportions of W and H E. G. M. Petrakis Image Formation 25

HIS Color Space • Represented as a double cone – Intensity: the main axis

HIS Color Space • Represented as a double cone – Intensity: the main axis (white at the top, black at the bottom) – Hue: angle around the axis – Saturation: distance from axis – Saturated colors on maximal circles E. G. M. Petrakis Image Formation 26

HSV Color Space • Similar to HIS – Value – Hue – Saturation –

HSV Color Space • Similar to HIS – Value – Hue – Saturation – H = undefined for S = 0 – H = 360 – H if B/V > G/V E. G. M. Petrakis Image Formation 27

Color Models for Video (YIQ) • YIQ is used for color TV broadcasting E.

Color Models for Video (YIQ) • YIQ is used for color TV broadcasting E. G. M. Petrakis Image Formation 28

4. Spatial Frequency Model • Describe spatial variations in the frequency domain of the

4. Spatial Frequency Model • Describe spatial variations in the frequency domain of the Fourier Transform: E. G. M. Petrakis Image Formation 29

 • f(m, n): linear combination of periodic waveforms – – exp{j 2π(ux +

• f(m, n): linear combination of periodic waveforms – – exp{j 2π(ux + vy)} F(u, v): weight factor of frequency u, v High u, v image detail (edges, points etc. ) Low u, v no detail, smooth areas E. G. M. Petrakis Image Formation 30

Fourier Transform Pairs E. G. M. Petrakis Image Formation 31

Fourier Transform Pairs E. G. M. Petrakis Image Formation 31

Fourier Transform Pairs (2) E. G. M. Petrakis Image Formation 32

Fourier Transform Pairs (2) E. G. M. Petrakis Image Formation 32

Sinc(x, y) E. G. M. Petrakis Image Formation 33

Sinc(x, y) E. G. M. Petrakis Image Formation 33

(a) Original image (b) Edge Enhanced image (c) Smoothed image Fourier Transform of (a)

(a) Original image (b) Edge Enhanced image (c) Smoothed image Fourier Transform of (a) Fourier Transform of (b) Fourier Transform of (c) E. G. M. Petrakis Image Formation 34

Convolution • Convolution of f and g: • Invert g by 1800, pass g

Convolution • Convolution of f and g: • Invert g by 1800, pass g over f and compute h on each point of f • Theorem: E. G. M. Petrakis Image Formation 35

6. Digitizing Model • Digitization: Conversion of continuous signals to discrete. – f(x, y)

6. Digitizing Model • Digitization: Conversion of continuous signals to discrete. – f(x, y) f(m, n) , 0<= m <= M-1, 0<=n<=N-1 – f(m, n) = k (intensity value) 0 <= k <= K-1. – f(m, n): samples taken at equal intervals. • Perfect sampling: It is possible to reconstruct f(x, y) from f(m, n) – K, m, n must be large enough E. G. M. Petrakis Image Formation 36

Image Sampling • Multiply f(x, y) by Sampling function E. G. M. Petrakis Image

Image Sampling • Multiply f(x, y) by Sampling function E. G. M. Petrakis Image Formation 37

 • One way to reconstruct an image from its samples f(k. T) would

• One way to reconstruct an image from its samples f(k. T) would be to interpolate suitably between the samples • Consider one dimensional signals – in the frequency domain – g(x-k. T) interpolation function – T: sampling period E. G. M. Petrakis Image Formation 38

F(u) for 1 D band limited function fc: max frequency Non-overlapping copies of F(u)

F(u) for 1 D band limited function fc: max frequency Non-overlapping copies of F(u) Overlapping copies of F(u) E. G. M. Petrakis Image Formation 39

F(u, v) 2 D band-limited function Non-overlapping copies of F(u, v) E. G. M.

F(u, v) 2 D band-limited function Non-overlapping copies of F(u, v) E. G. M. Petrakis Image Formation 40

 • Select G(w) that isolates F(w) from its samples • Whittaker-Kotelnikov-Shannon theorem: f(x)

• Select G(w) that isolates F(w) from its samples • Whittaker-Kotelnikov-Shannon theorem: f(x) can be reconstructed if the time distance between the samples is at least 1/2 f – 2 fc: sampling rate – If the signal is not band-limited we have aliasing (interference from high frequencies) – Smooth before sampling E. G. M. Petrakis Image Formation 41

E. G. M. Petrakis Image Formation 42

E. G. M. Petrakis Image Formation 42

E. G. M. Petrakis Image Formation 43

E. G. M. Petrakis Image Formation 43

 K E. G. M. Petrakis Image Formation 44

K E. G. M. Petrakis Image Formation 44