Introduction To Tracking Mario Haddad What is Tracking

  • Slides: 55
Download presentation
Introduction To Tracking Mario Haddad

Introduction To Tracking Mario Haddad

What is Tracking? • Estimating pose (state) • Possible from a variety of measured

What is Tracking? • Estimating pose (state) • Possible from a variety of measured sensors – – – Electrical Mechanical Inertial Optical Acoustic Magnetic 2

DYNAMIC SCENE ANALYSIS

DYNAMIC SCENE ANALYSIS

Typical Applications Motion detection. Often from a static camera. Object localization. Three-dimensional shape from

Typical Applications Motion detection. Often from a static camera. Object localization. Three-dimensional shape from motion. Object tracking.

Example Application

Example Application

Object Tracking Definition v Object tracking is the problem of determining (estimating) the positions

Object Tracking Definition v Object tracking is the problem of determining (estimating) the positions and other relevant information of moving objects in image sequences.

Difficulties In Reliable Object Tracking Rapid appearance changes caused by Ø image noise, Ø

Difficulties In Reliable Object Tracking Rapid appearance changes caused by Ø image noise, Ø illumination changes, Ø non-rigid motion, Ø . . . Non-stable background Ø Interaction between multiple objects Ø . . .

Difficulties In Reliable Object Tracking t o n t u ! b t, ible

Difficulties In Reliable Object Tracking t o n t u ! b t, ible l u s c s i ff po i D im Robust Density Comparison for Visual Tracking (BMVC 2009)

Difficulties In Reliable Object Tracking

Difficulties In Reliable Object Tracking

Motion Estimation

Motion Estimation

Block Matching Method For a given region in one frame, find the corresponding region

Block Matching Method For a given region in one frame, find the corresponding region in the next frame by finding the maximum correlation score (or other block matching criteria) in a search region

Block Matching Method

Block Matching Method

Block Matching Method

Block Matching Method

Optical Flow Motion Field (a) (b)

Optical Flow Motion Field (a) (b)

Visible Motion and True Motion OPTIC FLOW - apparent motion of the same (similar)

Visible Motion and True Motion OPTIC FLOW - apparent motion of the same (similar) intensity patterns Generally, optical flow corresponds to the motion field, but not always:

Local Features for Tracking If strong derivatives are observed in two orthogonal directions then

Local Features for Tracking If strong derivatives are observed in two orthogonal directions then we can hope that this point is more likely to be unique. Many trackable features are called corners. Harris Corner Detection !

Aperture Problem

Aperture Problem

The Aperture Problem Different motions – classified as similar source: Ran Eshel

The Aperture Problem Different motions – classified as similar source: Ran Eshel

The Aperture Problem Similar motions – classified as different source: Ran Eshel

The Aperture Problem Similar motions – classified as different source: Ran Eshel

Tracking Methods

Tracking Methods

Mean-Shift The mean-shift algorithm is an efficient approach to tracking objects whose appearance is

Mean-Shift The mean-shift algorithm is an efficient approach to tracking objects whose appearance is defined by histograms. (not limited to only color)

Motivation – to track non-rigid objects, (like a walking person), it is hard to

Motivation – to track non-rigid objects, (like a walking person), it is hard to specify an explicit 2 D parametric motion model. Appearances of non-rigid objects can sometimes be modeled with color distributions

Mean Shift Theory

Mean Shift Theory

Intuitive Description Region of interest Center of mass Mean Shift vector Objective : Find

Intuitive Description Region of interest Center of mass Mean Shift vector Objective : Find the densest region Distribution of identical billiard balls Stolen from: www. wisdom. weizmann. ac. il/~deniss/vision_spring 04/files/mean_shift. ppt

Intuitive Description Region of interest Center of mass Mean Shift vector Objective : Find

Intuitive Description Region of interest Center of mass Mean Shift vector Objective : Find the densest region Distribution of identical billiard balls Stolen from: www. wisdom. weizmann. ac. il/~deniss/vision_spring 04/files/mean_shift. ppt

Intuitive Description Region of interest Center of mass Mean Shift vector Objective : Find

Intuitive Description Region of interest Center of mass Mean Shift vector Objective : Find the densest region Distribution of identical billiard balls Stolen from: www. wisdom. weizmann. ac. il/~deniss/vision_spring 04/files/mean_shift. ppt

Intuitive Description Region of interest Center of mass Mean Shift vector Objective : Find

Intuitive Description Region of interest Center of mass Mean Shift vector Objective : Find the densest region Distribution of identical billiard balls Stolen from: www. wisdom. weizmann. ac. il/~deniss/vision_spring 04/files/mean_shift. ppt

Intuitive Description Region of interest Center of mass Mean Shift vector Objective : Find

Intuitive Description Region of interest Center of mass Mean Shift vector Objective : Find the densest region Distribution of identical billiard balls Stolen from: www. wisdom. weizmann. ac. il/~deniss/vision_spring 04/files/mean_shift. ppt

Intuitive Description Region of interest Center of mass Mean Shift vector Objective : Find

Intuitive Description Region of interest Center of mass Mean Shift vector Objective : Find the densest region Distribution of identical billiard balls

Intuitive Description Region of interest Center of mass Objective : Find the densest region

Intuitive Description Region of interest Center of mass Objective : Find the densest region Distribution of identical billiard balls Stolen from: www. wisdom. weizmann. ac. il/~deniss/vision_spring 04/files/mean_shift. ppt

Mean Shift Vector Given: Data points and approximate location of the mean of this

Mean Shift Vector Given: Data points and approximate location of the mean of this data: Task: Estimate the exact location of the mean of the data by determining the shift vector from the initial mean.

Mean Shift Vector

Mean Shift Vector

A Quick PDF Definition A probability density function (pdf), is a function that describes

A Quick PDF Definition A probability density function (pdf), is a function that describes the relative likelihood for this random variable to take on a given value.

Mean-Shift Object Tracking Target Representation Choose a reference target model Choose a feature space

Mean-Shift Object Tracking Target Representation Choose a reference target model Choose a feature space Quantized Color Space Stolen from: www. cs. wustl. edu/~pless/559/lectures/lecture 22_tracking. ppt Represent the model by its PDF in the feature space

Mean-Shift Object Tracking PDF Representation Target Model (centered at 0) Similarity Function: Stolen from:

Mean-Shift Object Tracking PDF Representation Target Model (centered at 0) Similarity Function: Stolen from: www. cs. wustl. edu/~pless/559/lectures/lecture 22_tracking. ppt Target Candidate (centered at y) Q is the target histogram, P is the object histogram (depends on location y)

Mean-Shift Object Tracking Target Localization Algorithm Start from the position of the model in

Mean-Shift Object Tracking Target Localization Algorithm Start from the position of the model in the current frame Search in the model’s neighborhood in next frame Stolen from: www. cs. wustl. edu/~pless/559/lectures/lecture 22_tracking. ppt Find best candidate by maximizing a similarity func.

Mean Shift Mean-Shift in tracking task: Ø track the motion of a cluster of

Mean Shift Mean-Shift in tracking task: Ø track the motion of a cluster of interesting features. 1. choose the feature distribution to represent an object (e. g. , color + texture), 2. start the mean-shift window over the feature distribution generated by the object 3. finally compute the chosen feature distribution over the next video frame.

Mean Shift Starting from the current window location, the mean-shift algorithm will find the

Mean Shift Starting from the current window location, the mean-shift algorithm will find the new peak or mode of the feature distribution, which (presumably) is centered over the object that produced the color and texture in the first place. In this way, the mean-shift window tracks the movement of the object frame by frame.

Examples

Examples

Examples

Examples

Other Mean Shift Applications

Other Mean Shift Applications

Edge Preserving Smoothing

Edge Preserving Smoothing

Segmentation

Segmentation

Contour Detection

Contour Detection

Kalman Filter Rudolf Emil Kalman • • • Born in 1930 in Hungary BS

Kalman Filter Rudolf Emil Kalman • • • Born in 1930 in Hungary BS and MS from MIT Ph. D 1957 from Columbia Filter developed in 1960 -61 Now retired

Kalman Filter • Noisy data in hopefully less noisy data out • The Kalman

Kalman Filter • Noisy data in hopefully less noisy data out • The Kalman filter operates recursively on streams of noisy input data to produce a statistically optimal estimate of the underlying system state.

Motivation

Motivation

Kalman Filter Applications Tracking objects (e. g. , missiles, faces, heads, hands) Navigation Many

Kalman Filter Applications Tracking objects (e. g. , missiles, faces, heads, hands) Navigation Many computer vision applications – Stabilizing depth measurements – Feature tracking – Cluster tracking – Fusing data from radar, laser scanner and stereo-cameras for depth and velocity measurements – Many more

Intuition Robot ü Odometer ü GPS Sand Previous state We may encounter: Wheel spin

Intuition Robot ü Odometer ü GPS Sand Previous state We may encounter: Wheel spin GPS inaccuracy Odometer GPS

Kalman Filter Not perfectly sure. Why ?

Kalman Filter Not perfectly sure. Why ?

Kalman Filter Kalman filter finds the most optimum averaging factor for each consequent state.

Kalman Filter Kalman filter finds the most optimum averaging factor for each consequent state. “somehow” remembers a little bit about the past states.

Kalman Filter State Prediction: Measurement Prediction:

Kalman Filter State Prediction: Measurement Prediction:

Kalman Filter • Two groups of the equations for the Kalman filter: o Time

Kalman Filter • Two groups of the equations for the Kalman filter: o Time update equations (Prediction) o Measurement update equations. (Correction) • The time update equations are responsible for projecting forward (in time) the current state and error covariance estimates to obtain the a priori estimates for the next time step. • The measurement update equations are responsible for the feedback—i. e. for incorporating a new measurement into the a priori estimate to obtain an improved a posteriori estimate.

Brace Yourselves. .

Brace Yourselves. .

Kalman Filter Update Predict 1. 2. Predict the state ahead: Predict the error covariance

Kalman Filter Update Predict 1. 2. Predict the state ahead: Predict the error covariance ahead: 1. Update the state estimate: 2. Update the error covariance: where Kalman gain Kt is: 55