Optical Snow and the Aperture Problem Michael Langer

Optical Snow and the Aperture Problem Michael Langer Richard Mann School of Computer Science Mc. Gill U. School of Computer Science U. Waterloo

Optical snow e. g. falling snow

Optical snow

Optical snow Egomotion in a 3 D cluttered scene

Optical snow

Relevance to “Heading” Human observers can judge their direction of heading through a “ 3 D cloud of dots”. (W. Warren and others) However, it is unclear how. Computational models assume accurate velocity estimates can be computed. (see Langer and Mann, ECVP ’ 01, ICCV ’ 01)

Overview of Talk 1. Fourier model of image motion 2. Distributed codes in the brain (V 1) 3. “Aperture problem”

Fourier model of image translation (Watson & Ahumada ’ 85) f tt fy f x v f + f = 0 x x y y t

Optical Snow Image velocities are (α v , α v ) x y

Fourier model of optical snow f t f “bowtie” t fx α v f x x +α v f + f = 0 y y t

Example of bowtie in power spectrum bush image sequence

Overview of Talk 1. Fourier model of image motion 2. Distributed codes in the brain (V 1) 3. “Aperture problem”

What about the brain ? ft fy f x Oriented, directionally tuned cells in V 1.

What about the brain ? ft fy f x Oriented, directionally tuned cells in V 1.

Image translation (v x , v y ) f ft tt fy fy f f x (see Heeger ’ 87, Yuille and Grzywacz ’ 90, Simoncelli and Heeger ‘ 97) x

Optical snow f (α v x , α v y ) ft t fy f x

Overview of Talk 1. Fourier model of image motion 2. Distributed codes in the brain (V 1) 3. “Aperture problem”

The Aperture Problem 1. Perceived motion of a line is in “normal” direction. 2. Problem can be resolved if endpoints or corners are visible.

Aperture problem ? yes

Aperture problem ? no

Aperture problem ? ? Falling ellipsoids with random phase spectrum.

Open problems • How do dominant spatial orientations affect perceived direction(s) of motion ? • How do T-junctions, perspective, etc. affect perception of optical snow ? • Computational models ?

Aperture problem f ft t fy f x