Motion Optical Flow II Estimation of Motion Field

Motion / Optical Flow II Estimation of Motion Field Avneesh Sud Computer Vision - Fall 2001

Outline • Motion Field & Optical Flow • Constraints • Methods of Estimating Motion Field – Differential Techniques • Least Squares • Horn-Schunck Algorithm • Comments • Results by Miguel Computer Vision - Fall 2001 2

Motion Field • 2 -D projection of velocities of the image points, induced by the relative motion between camera and scene – Not directly measurable from an image Computer Vision - Fall 2001 3

Optical Flow • A vector field subject to Image Brightness Constancy Equation (IBCE) • Apparent motion of the image brightness pattern Computer Vision - Fall 2001 4

Optical Flow Vs. Motion Field • Optical flow does not always correspond to motion field • Optical flow is an approximation of the motion field. The error is small at points with high spatial gradient under some simplifying assumptions (Trucco p 195) Computer Vision - Fall 2001 5

Ambiguity in Local Optical Flow • Correspondence between points on isobrightness contours? • A constant patch of uniform brightness – multiple optical flow solutions – Use additional constraints ! Computer Vision - Fall 2001 6

IBCE Revisited Assume the image intensity of each visible scene point is unchanging over time Also known as the Horn and Schunck optical flow constraint equation Computer Vision - Fall 2001 7

Aperture Problem • Constraint corresponds to a line in velocity space v Constraint line (Ex, Ey) u • Given local info, can determine component of optical flow vector only in direction of brightness gradient Computer Vision - Fall 2001 8

Estimating Motion Field • Differential techniques : based on spatial & temporal variations of the image at all pixels • Matching (feature-based) techniques : rely on special image points (features) and track them through frames Computer Vision - Fall 2001 9

Differential Techniques : Least Squares • Optical Flow Algorithm (Trucco, p 196) – For each pixel p • Must satisfy ( E)v + Et = 0 • Assumption : This equation holds in the neighborhood of p with constant v • Write this equation for a small (typically 5 x 5) patch centered at p • Then we find least square fit of v - this is the calculated optical flow for pixel p Computer Vision - Fall 2001 10

Least Squares : Assumptions • Assumed that ICBE holds in the neighborhood of p with constant v • In case of rigid motion, the motion field of a moving plane is a quadratic polynomial in the coordinates (x, y, f) of the image points. (Trucco p 187) – Therefore, if the object is smooth & rigid, we can assume the motion field varies smoothly Computer Vision - Fall 2001 11

Differential Techniques : Horn. Schunck Algorithm • Optical flow constraint equation gives the component in direction of brightness gradient : • Additional Constraint : smoothness of optical flow! Neighboring surface points of a rigid object have approximately same local displacement vectors Computer Vision - Fall 2001 12

Horn-Schunck Algorithm • Two criteria: – Optical flow is smooth, Fs(u, v) – Small error in optical flow constraint equation, Fh(u, v) • Minimize a combined error functional Fc(u, v) = Fs(u, v) + λ Fh(u, v) λ is a weighting parameter • Variation calculus gives a pair of second order differential equations that can be solved iteratively Computer Vision - Fall 2001 13

Horn-Schunck Algorithm : Discrete Case • Derivatives (and error functionals) are approximated by difference operators • Leads to an iterative solution: Computer Vision - Fall 2001 14

Intuition of the Iterative Scheme v (E , E ) x y Constraint line (u, v) u The new value of (u, v) at a point is equal to the average of surrounding values minus an adjustment in the direction of the brightness gradient Computer Vision - Fall 2001 15

Horn-Schunck Algorithm begin for j : = 1 to N do for I: = 1 to M do begin calculate the values Ex(i, j, t), Ey(i, j, t) and Et(i, j, t) using a selected approx formula initialize the values u(I, j) and v(i, j) to zero end {for} choose a suitable weighting value choose a suitable number n 0 1 of iterations n : = 1 while n n 0 do begin for j : = 1 to N do for i : = 1 to M do begin compute u, v, update u(i, j), v(i, j) end {for} n : = n + 1 end {while} end Computer Vision - Fall 2001 17

Comments • There are reliable methods for estimating optical flow. • Optical flow is a vector field, from which the motion field can be estimated under certain conditions. • Horn – Schunk Algorithm as presented does not handle discontinuities (silhouettes) well. • Some real-world results! Computer Vision - Fall 2001 18

References • Introductory Techniques for 3 -D Computer Vision, Emanuele Trucco and Allessandro Verri, Prentice Hall, 1998. Chapter 8 • Robot Vision, B. K. P. Horn, MIT Press 1986. Chapter 12 • Computer Vision: Three-Dimensional Data from Images, Reinhard Klette, Karsten Schluns, Andreas Koschan, Springer 1998. Topic 5. 2 Computer Vision - Fall 2001 19

References • Mike. Talk: A Talking Facial Display Based on Morphing Visemes, Tony Ezzat and Tomaso Poggio, Proceedings of the Computer Animation Conference Philadelphia, PA, June 1998 • Optical Flow - CVonline: Motion and Time Sequence Analysis (http: //www. dai. ed. ac. uk/CVonline/motion. htm) Computer Vision - Fall 2001 20