Stanford CS 223 B Computer Vision Winter 2007
Stanford CS 223 B Computer Vision, Winter 2007 Lecture 12 Tracking Motion Professors Sebastian Thrun and Jana Košecká CAs: Vaibhav Vaish and David Stavens
Overview n n n The Tracking Problem Bayes Filters Particle Filters Kalman Filters Using Kalman Filters Sebastian Thrun and Jana Košecká CS 223 B Computer Vision, Winter 2007
The Tracking Problem Image 1 n n n Image 2 Image 3 Image 4 Can we estimate the position of the object? Can we estimate its velocity? Can we predict future positions? Sebastian Thrun and Jana Košecká CS 223 B Computer Vision, Winter 2007
The Tracking Problem n n n Given Sequence of Images Find center of moving object Camera might be moving or stationary n We assume: We can find object in individual images. The Problem: Track across multiple images. n Is a fundamental problem in computer vision n Sebastian Thrun and Jana Košecká CS 223 B Computer Vision, Winter 2007
Methods Bayes Filter Kalman Filter Uncented Kalman Filter Particle Filter Extended Kalman Filter Sebastian Thrun and Jana Košecká CS 223 B Computer Vision, Winter 2007
Further Reading… Sebastian Thrun and Jana Košecká CS 223 B Computer Vision, Winter 2007
Example: Moving Object Sebastian Thrun and Jana Košecká CS 223 B Computer Vision, Winter 2007
Kalman Filter Tracking Sebastian Thrun and Jana Košecká CS 223 B Computer Vision, Winter 2007
Particle Filter Tracking Sebastian Thrun and Jana Košecká CS 223 B Computer Vision, Winter 2007
Mixture of KF / PF (Unscented PF) Sebastian Thrun and Jana Košecká CS 223 B Computer Vision, Winter 2007
Overview n n n The Tracking Problem Bayes Filters Particle Filters Kalman Filters Using Kalman Filters Sebastian Thrun and Jana Košecká CS 223 B Computer Vision, Winter 2007
Example of Bayesian Inference p(staircase) Slow = 0. 28 Down! Cost Sensor model cost(fast p(imagewalk | staircase) = 0. 7= $1, 000 cost(fast p(imagewalk | no staircase) = 0. 2= $0 cost(slow+sense) = $1 Environment prior p(staircase) = 0. 1 Decision Bayesian Theory inference p(staircase walk)] | image) = $1, 000 0. 28 = $280 E[cost(fast p(image | staircasse) p(staircase) E[cost(slow+sense)] = $1 = p(im | stair) p(stair) + p(im | no stair) p(no stair) = 0. 7 0. 1 / (0. 7 0. 1 + 0. 2 0. 9) = 0. 28 ? Sebastian Thrun and Jana Košecká CS 223 B Computer Vision, Winter 2007
Bayes Filter Definition n Environment state xt Measurement zt n Can we calculate p(xt | z 1, z 2, …, zt) ? n Sebastian Thrun and Jana Košecká CS 223 B Computer Vision, Winter 2007
Bayes Filters Illustrated Sebastian Thrun and Jana Košecká CS 223 B Computer Vision, Winter 2007
Bayes Filters: Essential Steps n n n Belief: Bel(xt) Measurement update: Bel(xt) p(zt|xt) Time update: Bel(xt+1) Bel(xt) p(xt+1|ut, xt) Sebastian Thrun and Jana Košecká CS 223 B Computer Vision, Winter 2007
Bayes Filters = state = time = observation = action h = constant x t z u Bayes Markov Sebastian Thrun and Jana Košecká CS 223 B Computer Vision, Winter 2007
Bayes Filters Sebastian Thrun and Jana Košecká x t z u = state = time = observation = action CS 223 B Computer Vision, Winter 2007
Bayes Filters Illustrated Sebastian Thrun and Jana Košecká CS 223 B Computer Vision, Winter 2007
Bayes Filters n n Initial Estimate of State Iterate – Receive measurement, update your belief (uncertainty shrinks) – Predict, update your belief (uncertainty grows) Sebastian Thrun and Jana Košecká CS 223 B Computer Vision, Winter 2007
Methods Bayes Filter Kalman Filter Uncented Kalman Filter Particle Filter Extended Kalman Filter Sebastian Thrun and Jana Košecká CS 223 B Computer Vision, Winter 2007
Overview n n n The Tracking Problem Bayes Filters Particle Filters Kalman Filters Using Kalman Filters Sebastian Thrun and Jana Košecká CS 223 B Computer Vision, Winter 2007
Particle Filters: Basic Idea set of n particles Xt (equality for Sebastian Thrun and Jana Košecká ) CS 223 B Computer Vision, Winter 2007
Particle Filter Explained Sebastian Thrun and Jana Košecká CS 223 B Computer Vision, Winter 2007
![Basic Particle Filter Algorithm Initialization: X 0 n particles x 0 [i] ~ p(x](http://slidetodoc.com/presentation_image/28debf95a2b37b04aa7999245e182442/image-24.jpg "Basic Particle Filter Algorithm Initialization: X 0 n particles x 0 [i] ~ p(x")
Basic Particle Filter Algorithm Initialization: X 0 n particles x 0 [i] ~ p(x 0) particle. Filters(Xt -1 ){ for i=1 to n xt [i] ~ p(xt | xt -1[i]) (prediction) wt[i] = p(zt | xt[i]) (importance weights) endfor i=1 to n include xt [i] in Xt with probability wt[i] (resampling) } Sebastian Thrun and Jana Košecká CS 223 B Computer Vision, Winter 2007
Importance Sampling Weight samples: w = f / g Sebastian Thrun and Jana Košecká CS 223 B Computer Vision, Winter 2007
Particle Filter By Frank Dellaert Sebastian Thrun and Jana Košecká CS 223 B Computer Vision, Winter 2007
Case Study: Track moving objects from Helicopter 1. 2. 3. 4. 5. Harris Corners Optical Flow (with clustering) Motion likelihood function Particle Filter Centroid Extraction David Stavens, Andrew Lookingbill, David Lieb, CS 223 b Winter 2004 Sebastian Thrun and Jana Košecká CS 223 B Computer Vision, Winter 2007
1. Harris Corner Extraction David Stavens, Andrew Lookingbill, David Lieb, CS 223 b Winter 2004 Sebastian Thrun and Jana Košecká CS 223 B Computer Vision, Winter 2007
2. Optical Flow + Motion Detection David Stavens, Andrew Lookingbill, David Lieb, CS 223 b Winter 2004 Sebastian Thrun and Jana Košecká CS 223 B Computer Vision, Winter 2007
3. Motion Likelihood Function David Stavens, Andrew Lookingbill, David Lieb, CS 223 b Winter 2004 Sebastian Thrun and Jana Košecká CS 223 B Computer Vision, Winter 2007
4. Particle Filters David Stavens, Andrew Lookingbill, David Lieb, CS 223 b Winter 2004 Sebastian Thrun and Jana Košecká CS 223 B Computer Vision, Winter 2007
5. Extract Centroid David Stavens, Andrew Lookingbill, David Lieb, CS 223 b Winter 2004 Sebastian Thrun and Jana Košecká CS 223 B Computer Vision, Winter 2007
More Particle Filter Tracking David Stavens, Andrew Lookingbill, David Lieb, CS 223 b Winter 2004 Sebastian Thrun and Jana Košecká CS 223 B Computer Vision, Winter 2007
Some Robotics Examples n n n Tracking Hands, People Mobile Robot localization People localization Car localization Mapping Sebastian Thrun and Jana Košecká CS 223 B Computer Vision, Winter 2007
Examples Particle Filter Siu Chi Chan Mc. Gill University Sebastian Thrun and Jana Košecká CS 223 B Computer Vision, Winter 2007
Another Example Mike Isard and Andrew Blake Sebastian Thrun and Jana Košecká CS 223 B Computer Vision, Winter 2007
Tracking Fast moving Objects Sebastian Thrun and Jana Košecká K. Toyama, A. Blake CS 223 B Computer Vision, Winter 2007
Particle Filters: Illustration With: Wolfram Burgard, Dieter Fox, Frank Dellaert Sebastian Thrun and Jana Košecká CS 223 B Computer Vision, Winter 2007
Particle Filters (1) Sebastian Thrun and Jana Košecká CS 223 B Computer Vision, Winter 2007
Particle Filters (2) Sebastian Thrun and Jana Košecká CS 223 B Computer Vision, Winter 2007
Particles = Robustness Sebastian Thrun and Jana Košecká CS 223 B Computer Vision, Winter 2007
Tracking People from Moving Platform robot location (particles) people location (particles) laser measurements (wall) Sebastian Thrun and Jana Košecká With Michael Montemerlo CS 223 B Computer Vision, Winter 2007
Tracking People from Moving Platform robot location (particles) people location (particles) laser measurements (wall) Sebastian Thrun and Jana Košecká With Michael Montemerlo CS 223 B Computer Vision, Winter 2007
Particle Filters for Tracking Cars With Anya Petrovskaya Sebastian Thrun and Jana Košecká CS 223 B Computer Vision, Winter 2007
Mapping Environments (SFM) With Dirk Haehnel Sebastian Thrun and Jana Košecká CS 223 B Computer Vision, Winter 2007
Overview n n n The Tracking Problem Bayes Filters Particle Filters Kalman Filters Using Kalman Filters Sebastian Thrun and Jana Košecká CS 223 B Computer Vision, Winter 2007
Tracking with KFs: Gaussians! initial estimte prediction y y x Sebastian Thrun and Jana Košecká measurement update y x CS 223 B Computer Vision, Winter 2007 x
Kalman Filters Sebastian Thrun and Jana Košecká CS 223 B Computer Vision, Winter 2007
Kalman Filters posterior Measurement evidence Sebastian Thrun and Jana Košecká prior CS 223 B Computer Vision, Winter 2007
A Quiz Measurement evidence prior posterior? Sebastian Thrun and Jana Košecká CS 223 B Computer Vision, Winter 2007
Gaussians Univariate m -s s Multivariate m Sebastian Thrun and Jana Košecká CS 223 B Computer Vision, Winter 2007
Properties of Univariate Gaussians Sebastian Thrun and Jana Košecká CS 223 B Computer Vision, Winter 2007
Measurement Update Derived Sebastian Thrun and Jana Košecká CS 223 B Computer Vision, Winter 2007
Properties Multivariate Gaussians Essentially the same as in the 1 -D case, but with more general notation n We stay in the “Gaussian world” as long as we start with Gaussians and perform only linear transformations. Sebastian Thrun and Jana Košecká CS 223 B Computer Vision, Winter 2007
Linear Kalman Filter Estimates the state x of a discrete-time controlled process that is governed by the linear stochastic difference equation with a measurement Sebastian Thrun and Jana Košecká CS 223 B Computer Vision, Winter 2007
Components of a Kalman Filter Matrix (n n) that describes how the state evolves from t to t+1 without controls or noise. Matrix (n i) that describes how the control ut changes the state from t to t+1. Matrix (k n) that describes how to map the state xt to an observation zt. Random variables representing the process and measurement noise that are assumed to be independent and normally distributed with covariance Rt and Qt respectively. Sebastian Thrun and Jana Košecká CS 223 B Computer Vision, Winter 2007
Kalman Filter Algorithm 1. Algorithm Kalman_filter( mt-1, St-1, ut, zt): 2. 3. 4. Prediction: 5. 6. 7. 8. Correction: 9. Return mt, St Sebastian Thrun and Jana Košecká CS 223 B Computer Vision, Winter 2007
Kalman Filter Updates in 1 D measurement belief measu rement new belief old belief Sebastian Thrun and Jana Košecká CS 223 B Computer Vision, Winter 2007
Kalman Filter Updates in 1 D measu rement new belief old belief Sebastian Thrun and Jana Košecká CS 223 B Computer Vision, Winter 2007
Kalman Filter Updates in 1 D rement new belief measu newest belief old belief Sebastian Thrun and Jana Košecká CS 223 B Computer Vision, Winter 2007
Kalman Filter Updates belief latest belief measurement belief Sebastian Thrun and Jana Košecká belief CS 223 B Computer Vision, Winter 2007
The Prediction-Correction-Cycle Prediction Sebastian Thrun and Jana Košecká CS 223 B Computer Vision, Winter 2007
The Prediction-Correction-Cycle Correction Sebastian Thrun and Jana Košecká CS 223 B Computer Vision, Winter 2007
The Prediction-Correction-Cycle Prediction Correction Sebastian Thrun and Jana Košecká CS 223 B Computer Vision, Winter 2007
Kalman Filter Summary n Highly efficient: Polynomial in measurement dimensionality k and state dimensionality n: O(k 2. 376 + n 2) n Optimal for linear Gaussian systems! n Most robotics systems are nonlinear! Sebastian Thrun and Jana Košecká CS 223 B Computer Vision, Winter 2007
Overview n n n The Tracking Problem Bayes Filters Particle Filters Kalman Filters Using Kalman Filters Sebastian Thrun and Jana Košecká CS 223 B Computer Vision, Winter 2007
Let’s Apply KFs to Tracking Problem Image 1 Image 2 Sebastian Thrun and Jana Košecká Image 3 Image 4 CS 223 B Computer Vision, Winter 2007
Kalman Filter with 2 -Dim Linear Model n Linear Change (Motion) ? R , D , C t is Wha n Linear Measurement Model ? Q , , B A s i t Wha Sebastian Thrun and Jana Košecká CS 223 B Computer Vision, Winter 2007
Kalman Filter Algorithm 1. Algorithm Kalman_filter( mt-1, St-1, ut, zt): 2. 3. 4. Prediction: 5. 6. 7. 8. Correction: 9. Return mt, St Sebastian Thrun and Jana Košecká CS 223 B Computer Vision, Winter 2007
Kalman Filter in Detail n n Measurements Change Prediction Measurement Update initial position prediction y measurement y x Sebastian Thrun and Jana Košecká update y x CS 223 B Computer Vision, Winter 2007 x
Can We Do Better? Sebastian Thrun and Jana Košecká CS 223 B Computer Vision, Winter 2007
Kalman, Better! initial position prediction measurement update next prediciton Sebastian Thrun and Jana Košecká CS 223 B Computer Vision, Winter 2007
We Can Estimate Velocity! prediction past measurements Sebastian Thrun and Jana Košecká CS 223 B Computer Vision, Winter 2007
Kalman Filter For 2 D Tracking n Linear Measurement model (now with 4 state variables) n Linear Change Sebastian Thrun and Jana Košecká CS 223 B Computer Vision, Winter 2007
Putting It Together Again n Measurements n Change n Prediction n Measurement Update Sebastian Thrun and Jana Košecká CS 223 B Computer Vision, Winter 2007
Summary Kalman Filter n Estimates state of a system – Position – Velocity – Many other continuous state variables possible n KF maintains – Mean vector for the state – Covariance matrix of state uncertainty n Implements – Time update = prediction – Measurement update n Standard Kalman filter is linear-Gaussian – Linear system dynamics, linear sensor model – Additive Gaussian noise (independent) – Nonlinear extensions: extended KF, unscented KF: linearize n More info: – CS 226 – Probabilistic Robotics (Thrun/Burgard/Fox, MIT Press) Sebastian Thrun and Jana Košecká CS 223 B Computer Vision, Winter 2007
Particle Summary Kalman Filter n Estimates state of a system – Position and discrete – Velocity – Many other continuous state variables possible n KF maintains – Mean vector for the state – Covariance matrix of state uncertainty n set of particles (example states) Implements – Time update = prediction = predictive sampling – Measurement update = resampling, importance weights n Standard Kalman filter is linear-Gaussian – Linear system dynamics, linear sensor model fully nonlinear – Additive Gaussian noise (independent) – Nonlinear extensions: extended KF, unscented KF: linearize n More info: easy t o – CS 226 – Probabilistic Robotics (Thrun/Burgard/Fox, MIT Press) Sebastian Thrun and Jana Košecká implem ent CS 223 B Computer Vision, Winter 2007
Summary n n n The Tracking Problem Bayes Filters Particle Filters Kalman Filters Using Kalman Filters Sebastian Thrun and Jana Košecká CS 223 B Computer Vision, Winter 2007
- Slides: 78
