3 D Computer Vision CSc 83029 Photometric Stereo

3 -D Computer Vision CSc 83029 Photometric Stereo & Shape from Shading 3 -D Computer Vision CSc 83029 / Ioannis Stamos

Photometric Stereo & Shape from Shading § Technique for recovering 3 -D shape information from image intensity (brightness) § We will discuss: § Reflectance maps. § Photometric stereo. § Shape from shading. 3 -D Computer Vision CSc 83029 / Ioannis Stamos

Radiometry and Reflectance Image irradiance Brightness falloff 1 / F-number of lens Scene radiance We assume (calibration is needed) that: 3 -D Computer Vision CSc 83029 / Ioannis Stamos

Lambertian Reflectance Model I(x, y) p n θi v P or: s A Lambertian sphere k : Source brightness ρ’: Surface albedo (reflectance) ρ : Effective albedo (absorbs source brightness) : REFLECTANCE MAP 3 -D Computer Vision CSc 83029 / Ioannis Stamos

Lambertian Reflectance Model I(x, y) p n θi v P s A Lambertian sphere : REFLECTANCE MAP Relates Image Irradiance E to surface orientation for given source direction and surface reflectance 3 -D Computer Vision CSc 83029 / Ioannis Stamos

Representation of surface normal Z n ry Z=Z(x, y) Unit vector rx y Appendix A. 5 (Trucco) x 3 -D Computer Vision CSc 83029 / Ioannis Stamos

Gradient Space (p, q) Source z 1. 0 p q y x Surface normal can be represented by a point (p, q) on a plane! Source direction can be represented by a point (ps, qs)! **We want to calculate (p, q) from intensity I(x, y) 3 -D Computer Vision CSc 83029 / Ioannis Stamos

Gradient Space (p, q) Source z 1. 0 q y p x Surface normal Source direction Assumption: SOURCE DIR. IS CONSTANT FOR ENTIRE SCENE. 3 -D Computer Vision CSc 83029 / Ioannis Stamos

Reflectance Map (Lambertian) OR: Source z q Constant θi p y x 3 -D Computer Vision CSc 83029 / Ioannis Stamos ISO-BRIGHTNESS CONTOUR

Reflectance Map (Lambertian) ISO-BRIGHTNESS CONTOURS NOTE: R(p, q) is maximum when (p, q)=(ps, qs) 3 -D Computer Vision CSc 83029 / Ioannis Stamos

Reflectance Map (Lambertian) Examples. Where is the source with respect to the sphere? 3 -D Computer Vision CSc 83029 / Ioannis Stamos

Reflectance Map (Glossy Surfaces) 3 -D Computer Vision CSc 83029 / Ioannis Stamos

Shape from Shading ISO-BRIGHTNESS CONTOURS PROBLEM: Given 1) source direction 2) surface reflectance (ρ) 3) one intensity image I(x, y) Can we find 3 -Dunique surface orientation (p, q)? Computer Vision CSc 83029 / Ioannis Stamos

Two reflectance maps? 3 -D Computer Vision CSc 83029 / Ioannis Stamos

Two reflectance maps? Intersections: 2 solutions for p and q. What if we don’t know the albedo? 3 -D Computer Vision CSc 83029 / Ioannis Stamos

Photometric Stereo Use multiple light sources to resolve ambiguity In surface orientation. Note: Scene does not move – Correspondence between points in different images is easy! Notation: Direction of source i: or Image intensity produced by source i: 3 -D Computer Vision CSc 83029 / Ioannis Stamos

Lambertian Surfaces (special case) Use THREE sources in directions Image Intensities measured at point (x, y): orientation albedo 3 -D Computer Vision CSc 83029 / Ioannis Stamos

Photometric Stereo: RESULT INPUT 3 -D Computer Vision CSc 83029 / Ioannis Stamos albedo orientation

From Surface Orientations to Shape Integrate needle map 3 -D Computer Vision CSc 83029 / Ioannis Stamos

Calibration Objects & Look-up Tables Calibration: Useful when Reflectance Map Equations are uknown. Use Calibration sphere of known size and same reflectance as scene objects. Each point on the sphere has a unique known orientation. 3 -D Computer Vision CSc 83029 / Ioannis Stamos

Calibration Objects & Look-up Tables Illuminate the sphere with one source at a time and obtain an image. Each surface point with orientation (p, q) produces three images (I 1, I 2, I 3) Generate a LOOK-UP TABLE (I 1, I 2, I 3) -> (p, q) I 3 (p, q) ARRAY I 1 I 2 For an object of uknown shape but same reflectance, obtain 3 images using same sources. For each image Computer Stamos point use LUT to 3 -D map (I 1, Vision I 2, CSc 83029 I 3) ->/ Ioannis (p, q)

Photometric Stereo: Remarks (1) Reflectance & illumination must be known a-priori. (2) Local Method. (3) Major Assumption: No interreflections. Concave surfaces exhibit interreflections. 3 -D Computer Vision CSc 83029 / Ioannis Stamos

Shape from Shading § Can we recover shape from a Single Image? 3 -D Computer Vision CSc 83029 / Ioannis Stamos

Human Perception of Shape from Shading We assume light source is above us. Ramachandran 88 3 -D Computer Vision CSc 83029 / Ioannis Stamos

Human Perception of Shape from Shading We assume light source is above us. Ramachandran 88 3 -D Computer Vision CSc 83029 / Ioannis Stamos

Human Perception of Shape from Shading Surface boundaries have strong influence on perceived shape 3 -D Computer Vision CSc 83029 / Ioannis Stamos

Finding the Illumination Direction Assumption: The surface normals of the scene are distributed uniformly in 3 -D space Illumination direction Mean intensity Mean square intensity Mean image gradient 3 -D Computer Vision CSc 83029 / Ioannis Stamos

Shape from Shading Smoothness constraint 3 -D Computer Vision CSc 83029 / Ioannis Stamos

Shape from Shading Calculus of Variations -> Discrete Case: Is the average of the 4 neighboring values Iterative solution 3 -D Computer Vision CSc 83029 / Ioannis Stamos

Shape from Shading We know the normal at the contour. This provides boundary conditions. OCCLUDING BOUNDARY 3 -D Computer Vision CSc 83029 / Ioannis Stamos