Light and Color Jehee Lee Seoul National University

Light and Color Jehee Lee Seoul National University With a lot of slides stolen from Alexei Efros, Stephen Palmer, Fredo Durand others

Programming Assignment #4 • Shading opaque and translucent surfaces (20%) – You are required to render two overlapping surfaces • The sweep polygonal surface from the previous assignment should be opaque and shaded smoothly • A translucent cube should be created and placed such that it intersects the opaque sweep surface • Material (20%) – Render the cube by six polygons of different material – Select material properties for the surfaces, such as metal, plastic, rubber, and so on.

Programming Assignment #4 • Depth ordering (20%) – The opaque and translucent polygons should be sorted in a depth order such that they can be rendered appropriately – It is important to render translucent polygons properly when they intersect opaque polygons • Viewing and Lighting (20%) – Use more than one light sources to illuminate the surfaces appropriately when they are viewed from an arbitrary viewpoint – The user should be able to change the viewpoint

Programming Assignment #4 • Report (20%) – Submit a report of at most 5 pages – Explain material properties of each surface • What material do you want to simulate? • Describe all coefficients of the illumination model – Explain your depth ordering algorithm briefly • How do you handle intersecting polygons? – Explain the lighting configuration • How many light sources do you use? • Where do you put the light sources? • What is the type of each light source?

Programming Assignment #4 • Optional requirements – You may render complex opaque and translucent objects – You may implement BSP trees for depth ordering

Image Formation Digital Camera Film The Eye

Digital camera • A digital camera replaces film with a sensor array – Each cell in the array is light-sensitive diode that converts photons to electrons – Two common types • Charge Coupled Device (CCD) • CMOS – http: //electronics. howstuffworks. com/digital-camera. htm

Sensor Array CMOS sensor

Sampling and Quantization

The Eye • The human eye is a camera! – Iris - colored annulus with radial muscles – Pupil - the hole (aperture) whose size is controlled by the iris – What’s the “film”? • photoreceptor cells (rods and cones) in the retina

The Retina

Retina up-close Light

Two types of light-sensitive receptors Cones cone-shaped less sensitive operate in high light color vision Rods rod-shaped highly sensitive operate at night gray-scale vision © Stephen E. Palmer, 2002

Rod / Cone sensitivity The famous sock-matching problem…

Distribution of Rods and Cones Night Sky: why are there more stars off-center? © Stephen E. Palmer, 2002

Electromagnetic Spectrum Human Luminance Sensitivity Function http: //www. yorku. ca/eye/photopik. htm

Visible Light Why do we see light of these wavelengths? …because that’s where the Sun radiates EM energy © Stephen E. Palmer, 2002

The Physics of Light Any patch of light can be completely described physically by its spectrum: the number of photons (per time unit) at each wavelength 400 - 700 nm. © Stephen E. Palmer, 2002

The Physics of Light Some examples of the spectra of light sources © Stephen E. Palmer, 2002

Radiometry

Radiometry Radiant exitance Radiant intensity

Radiometry Radiance Irradiance

Photometry

Radiometry for color Horn, 1986 Spectral radiance: power in a specified direction, per unit area, per unit solid angle, per unit wavelength Spectral irradiance: incident power per unit area, per unit wavelength

Simplified rendering models: reflectance Often are more interested in relative spectral composition than in overall intensity, so the spectral BRDF computation simplifies a wavelength-by-wavelength multiplication of relative energies. . * Foundations of Vision, by Brian Wandell, Sinauer Assoc. , 1995 =

Simplified rendering models: transmittance . * Foundations of Vision, by Brian Wandell, Sinauer Assoc. , 1995 =

The Physics of Light % Photons Reflected Some examples of the reflectance spectra of surfaces Red 400 Yellow 700 400 Blue 700 400 Wavelength (nm) Purple 700 400 700 © Stephen E. Palmer, 2002

The Psychophysical Correspondence There is no simple functional description for the perceived color of all lights under all viewing conditions, but …. . . A helpful constraint: Consider only physical spectra with normal distributions mean area variance © Stephen E. Palmer, 2002

The Psychophysical Correspondence Hue # Photons Mean Wavelength © Stephen E. Palmer, 2002

The Psychophysical Correspondence Saturation # Photons Variance Wavelength © Stephen E. Palmer, 2002

The Psychophysical Correspondence Brightness # Photons Area Wavelength © Stephen E. Palmer, 2002

Physiology of Color Vision Three kinds of cones: © Stephen E. Palmer, 2002

More Spectra metamers

Metameric Whites

Metameric lights Foundations of Vision, by Brian Wandell, Sinauer Assoc. , 1995

Color Matching

Color matching experiment 1

Color matching experiment 1 p 2 p 3

Color matching experiment 1 p 2 p 3

Color matching experiment 1 The primary color amounts needed for a match p 1 p 2 p 3

Color matching experiment 2

Color matching experiment 2 p 1 p 2 p 3

Color matching experiment 2 p 1 p 2 p 3

Color matching experiment 2 We say a “negative” amount of p 2 was needed to make the match, because we added it to the test color’s side. p 1 p 2 p 3 The primary color amounts needed for a match: p 1 p 2 p 3

Foundations of Vision, by Brian Wandell, Sinauer Assoc. , 1995

Grassman’s Laws • For color matches: – – symmetry: U=V <=>V=U transitivity: U=V and V=W => U=W proportionality: U=V <=> t. U=t. V additivity: if any two (or more) of the statements U=V, W=X, (U+W)=(V+X) are true, then so is the third • These statements are as true as any biological law. They mean that additive color matching is linear. Forsyth & Ponce

Color Matching Functions p 1 = 645. 2 nm p 2 = 525. 3 nm p 3 = 444. 4 nm

Since we can define colors using almost any set of primary colors, let’s agree on a set of primaries and color matching functions for the world to use…

CIE XYZ color space • Commission Internationale d’Eclairage, 1931 • “…as with any standards decision, there are some irratating aspects of the XYZ color-matching functions as well…no set of physically realizable primary lights that by direct measurement will yield the color matching functions. ” • “Although they have served quite well as a technical standard, and are understood by the mandarins of vision science, they have served quite poorly as tools for explaining the discipline to new students and colleagues outside the field. ” Foundations of Vision, by Brian Wandell, Sinauer Assoc. , 1995

CIE XYZ: Color matching functions are positive everywhere, but primaries are “imaginary” (require adding light to the test color’s side in a color matching experiment). Usually compute x, y, where x=X/(X+Y+Z) y=Y/(X+Y+Z) Foundations of Vision, by Brian Wandell, Sinauer Assoc. , 1995

A qualitative rendering of the CIE (x, y) space. The blobby region represents visible colors. There are sets of (x, y) coordinates that don’t represent real colors, because the primaries are not real lights (so that the color matching functions could be positive everywhere). Forsyth & Ponce

• CIE chromaticity diagram encompasses all the perceivable colors in 2 D space (x, y) by ignoring the luminance

A plot of the CIE (x, y) space. We show the spectral locus (the colors of monochromatic lights) and the black-body locus (the colors of heated black-bodies). I have also plotted the range of typical incandescent lighting. Forsyth & Ponce

Pure wavelength in chromaticity diagram • Blue: big value of Z, therefore x and y small

Pure wavelength in chromaticity diagram • Then y increases

Pure wavelength in chromaticity diagram • Green: y is big

Pure wavelength in chromaticity diagram • Yellow: x & y are equal

Pure wavelength in chromaticity diagram • Red: big x, but y is not null

Color Gamut • The color gamut for n primaries in CIE chromaticity diagram is the convexhull of the color positions

Color Gamut

Complementary Colors • Illuminant C (Average sunlight)

Dominant Wavelength • The spectral color which can be mixed with white light in order to reproduce the desired color • C 2 have spectral distributions with subtractive dominant wave lengths

CIE color space • Can think of X, Y , Z as coordinates • Linear transform from typical RGB or LMS • Always positive (because physical spectrum is positive and matching curves are positives) • Note that many points in XYZ do not correspond to visible colors!

Color Gamut of RGB

XYZ vs. RGB • Linear transform • XYZ is rarely used for storage • There are tons of flavors of RGB – s. RGB, Adobe RGB – Different matrices! • XYZ is more standardized • XYZ can reproduce all colors with positive values • XYZ is not realizable physically !! – What happens if you go “off” the diagram – In fact, the orthogonal (synthesis) basis of XYZ requires negative values.

RGB color space • RGB cube – – Easy for devices But not perceptual Where do the grays live? Where is hue and saturation?

HSV • Hue, Saturation, Value (Intensity) – RGB cube on its vertex • Decouples the three components (a bit) • Use rgb 2 hsv() and hsv 2 rgb() in Matlab

600 700 nm 400 500 600 700 nm blue magenta 500 green 400 yellow red cyan Color names for cartoon spectra 400 500 600 700 nm

red Additive color mixing 500 600 700 nm 400 500 600 700 nm yellow green 400 When colors combine by adding the color spectra. Example color displays that follow this mixing rule: CRT phosphors, multiple projectors aimed at a screen, Polachrome slide film. Red and green make… Yellow! 400 500 600 700 nm

cyan Subtractive color mixing 500 600 700 nm yellow 400 500 600 700 nm green 400 When colors combine by multiplying the color spectra. Examples that follow this mixing rule: most photographic films, paint, cascaded optical filters, crayons. Cyan and yellow (in crayons, called “blue” and yellow) make… Green! 400 500 600 700 nm

NTSC color components: Y, I, Q

NTSC - RGB

§ CMY color model § subtractive model (colors of pigments are subtracted) § used in color output devices § CMYK color model - K for black ink for reducing the amount of ink

Uniform color spaces • Mc. Adam ellipses (next slide) demonstrate that differences in x, y are a poor guide to differences in color • Construct color spaces so that differences in coordinates are a good guide to differences in color. Forsyth & Ponce

Variations in color matches on a CIE x, y space. At the center of the ellipse is the color of a test light; the size of the ellipse represents the scatter of lights that the human observers tested would match to the test color; the boundary shows where the just noticeable difference is. The ellipses on the left have been magnified 10 x for clarity; on the right they are plotted to scale. The ellipses are known as Mac. Adam ellipses after their inventor. The ellipses at the top are larger than those at the bottom of the figure, and that they rotate as they move up. This means that the magnitude of the difference in x, y coordinates is a poor guide to the difference in color. Forsyth & Ponce

Perceptually Uniform Space: Mac. Adam • In perceptually uniform color space, Euclidean distances reflect perceived differences between colors • Mac. Adam ellipses (areas of unperceivable differences) become circles • Non-linear mapping, many solutions have been proposed Source: [Wyszecki and Stiles ’ 82]

CIELAB (a. k. a. CIE L*a*b*) • The reference perceptually uniform color space • L: lightness • a and b: color opponents • X 0, Y 0, and Z 0 are used to colorbalance: they’re the color of the reference white Source: [Wyszecki and Stiles ’ 82]

Color Sensing in Camera (RGB) • 3 -chip vs. 1 -chip: quality vs. cost • Why more green? Why 3 colors? http: //www. cooldic http: //www. cooldi tionary. com/words/Bayer-filter. wikipedia

Practical Color Sensing: Bayer Grid • Estimate RGB at ‘G’ cels from neighboring values http: //www. cooldictionary. com/ words/Bayer-filter. wikipedia

White Balance • Chromatic adaptation – If the light source is gradually changed in color, humans will adapt and still perceive the color of the surface the same

Color Temperature • Blackbody radiators

The infamous gamma curve • A gamma curve x -> x 1/ is used for many reasons: – CRT response – Color quantization – Perceptual effect • Sometimes with >1, sometimes <1 • These issues are often oversimplified/confused, including in prominent textbooks – i. e. they are explained wrong

Color quantization gamma • The allowable number of intensity levels can be distributed over the range from 0. 0 to 1. 0 so that this distribution corresponds to our perception of equal intensity intervals between levels • The human visual system is more sensitive to ratios – We perceive the difference between intensities 0. 20 and 0. 22 to be the same as the difference between 0. 80 and 0. 88

Color quantization gamma • Perceived difference between levels • For example, I 0=1/8 and n=3

Color quantization gamma • If we use linear encoding, we have tons of information between 128 and 255, but very little between 1 and 2! • Ideal encoding? Log • Problems with log? Gets crazy around zero Solution: gamma

Gamma encoding • From Greg Ward • only 6 bits for emphasis

Cathode Ray Tube gamma • The relationship between voltage and light intensity is non linear • Can be approximated by an exponent about 1. 7 to 2. 5 • Must be inverted to get linear response From Ponton’s FAQ http: //www. poynton. com/

Cathode Ray Tube gamma NTSC gamma A typical monitor response curve A video lookup correction curve

Perceptual effect One development solution Source: Ansel Adams Two development solution: the dark areas are the same, but bright areas are different

Halftoning • Simulate the look of a continuous tone image with limited colors

Halftoning • How to make gray using tiny dots of black ink

Color Halftoning

Color Halftoning

Halftone Patterns • For displaying intensity levels more than display device capability (clustered-dot ordered dither) • Intensity → spatial frequency of pixels or varying dot size • Spatial resolution vs. # of intensity level tradeoff

Halftone Patterns • A matrix of pixel position numbers

Halftone Patterns • 12 level intensity representations on a fourintensity-level system using a 2 by 2 Halftone pattern

Halftone Patterns • A 2 by 2 pixel-grid pattern for displaying RGB colors – Five intensity levels (0 through 4) for each color

Dithering • Dithering refers to techniques for approximating halftones without reducing resolution • But, dithering is sometimes used also as a synonym for any halftone-approximation scheme

Dithering • Floyd-Steinberg dithering by error diffusion

Dithering • Floyd-Steinberg dithering by error diffusion – Input: An original matrix M of pixel intensities – Output: A matrix I of pixel intensities with lesser levels For( j=0; j<m; j++ ) for( k=0; k<n; k++ ) { /* Determine the available system intensity value * that is closest to the value of M[j][k] and * assign this value to I[j][k] */ error = M[j][k] – I[j][k] I[j ][k+1] = M[j ][k+1] + alpha * error; I[j+1][k-1] = M[j+1][k-1] + beta * error; I[j+1][k ] = M[j+1][k ] + gamma * error; I[j+1][k+1] = M[j+1][k+1] + delta * error; }

Dithering • Reduced to web-safe color palettes

Dithering • Floyd-Steinberg dithering

Dithering • No dithering vs. Dithering using 16 colors palettes