Object recognition using shading Object recognition using shading

Object recognition using shading

Object recognition using shading Strong claim #1: to recognize an object using shading, we reconstruct the object’s 3 D shape and then we recognize this 3 D shape.

Object recognition using shading Strong claim #2: shape from shading is not used in object recognition

What is the role of shape from shading in object recognition ? 1. To what extent do humans perceive 3 D shape from shading ? 2. To what extent do humans use 3 D shape perception to recognize objects ? 3. We need to answer Q 1 before we can answer Q 2.

Human perception of local shape from shading under variable lighting Michael S. Langer* Heinrich H. Bülthoff Max-Planck-Institute for Biological Cybernetics Tübingen, Germany *Mc. Gill University, Montreal, Canada

Illumination models 1. Sunny day (Horn ‘ 70, …) 2. Cloudy day (Langer and Zucker ICCV ’ 93, Stewart and Langer CVPR ’ 97)

Overview • Experiment 1: SFS on a sunny day • Experiment 2: SFS on a cloudy day

SFS on a Sunny Day L N(x) I(x) = r N(x) L

Depth-reversal ambiguity on a sunny day valley hill

Hollow Mask Illusion (Luckiesh, 1916)

“Hollow mask illusion is due to two factors” (Johnston et al ’ 92 , Hill and Bruce ‘ 94) face familiarity (recognition) vs. hollow mask global convexity

Our experiments use unfamiliar surfaces convex concave “face” “mask”

Procedure

Task: hill or valley ?

Three factors were tested 1. light source direction (above > below) 2. global shape (convex > concave) 3. viewpoint (above > below) 4.

1. light source direction light from above light from below

2. global shape convex concave

3. Viewpoint (Reichel & Todd 1990) from above from below (floor) (ceiling)

Factor 3: viewpoint view from above view from below

Factor 3: viewpoint view from below view from above

Design • three factors (2 x 2) - light direction (above, below) • - global shape (convex, concave) - viewpoint (floor, ceiling) 512 trials (64 x 8 conditions)

Results (linear regression) percent correct (hill or valley ? ) = 51 + 10 * light direction + 13 * global shape + 11 * viewpoint (Each factor had value of – 1 or 1)

Conclusions: Experiment 1 • many factors play a role in resolving the depth-reversal ambiguity (light direction, global shape, viewpoint, …. )

Conclusions: Experiment 1 • many factors play a role in resolving the depth-reversal ambiguity (light direction, global shape, viewpoint, …. ) • observers often ignore available image information (perspective, occluding contours, shadows)

Experiment 2: Shape from shading on a cloudy day

Shading on a cloudy day (x)

Shading on a Cloudy Day I(x) N(x) L d. L (x) = angle of visible light source

Shading in a valley on a cloudy day local intensity maxima

Local intensity maxima in valleys

Experiment 2: How well do humans perceive shape from shading in the presence of these local intensity maxima ?

Experiment 2: How well do humans perceive shape from shading in the presence of these local intensity maxima ? Hypothesis: (shape from shading skeptic) Humans use “dark means deep” model.

Procedure

Task: Which is higher ?

Two conditions intensity correlated anti-correlated _ + height + correlated _ anti-correlated

Which is higher ? percent correct 1 N=17 0. 8 overall score 65% 0. 6 0. 4 0. 2 (above chance) 0 + _

Conclusion: Experiment 2 “Dark means deep” is too simple a model to explain human perception of shape from shading on a cloudy day.

Big Picture: What role does 3 D shape perception play in 3 D object recognition?

Big Picture: What role does 3 D shape perception play in 3 D object recognition? 1. To what extent do we perceive 3 D shape ? We’re on the way to answering this question.

Big Picture: What role does 3 D shape perception play in 3 D object recognition? 1. To what extent do we perceive 3 D shape ? We’re on the way to answering this question. 2. To what extent do we use these 3 D shape percepts to recognize objects ? 3. The answer to Q 2 depends on Q 1.

Computer vision psychophysics I(x) = (x) I(x) = (Langer and Zucker ‘ 93) N(x) Ld. L (Stewart and Langer ‘ 96)

Psychophysics: Human vs. Computer percent correct 1 0. 8 , N 0. 6 0. 4 0. 2 0 + - human + - + computer

Conclusions: Experiment 2 • “Dark means deep” is too simple to explain human perception of shape from shading on a cloudy day • Computing local shape in valleys is an inherently difficult computational problem on a cloudy day

Which point is brighter ? percent correct 1 N=10 0. 8 0. 6 0. 4 0. 2 0 _ +Correlation height brightness

Examples: 87 % (best) { } 15% (worst)