Multiscale Visual Tracking by Sequential Belief Propagation Gang

Multi-scale Visual Tracking by Sequential Belief Propagation Gang Hua, Ying Wu Dept. Electrical & Computer Engr. Northwestern University Evanston, IL 60208 {yingwu, ganghua}@ece. northwestern. edu 1/12/2022 CVPR'2004 1

Abrupt Motion sudden changes of target dynamics frame dropping large camera motion etc. 1/12/2022 CVPR'2004 2

Challenges Most existing visual tracking methods assume either small motion or accurate motion models Abrupt motion violates them Hierarchical search is not enough n n n Unidirectional information flow Error accumulation from coarse to fine No mechanism to recover failure in coarse scales 1/12/2022 CVPR'2004 3

Our Idea Different scales provide different salient visual features Bi-directional information flow among different scales should help Different scales “collaborate” 1/12/2022 CVPR'2004 4

Our Formulation A Markov network X={Xi , i=1. . L}—target state in different scales Z={Zi , i=1. . L}—Image observation of the target in different scales Undirected link— Potential function Ψij(fi(Xi), fj(Xj)), Directed link—Observation function Pi(Zi|Xi) The task is to infer Pi (Xi|Z), i=1. . L Fig. 1. Markov Network (MN) 1/12/2022 CVPR'2004 5

Belief propagation (BP) The joint posterior Belief propagation [Pearl’ 88, Freeman’ 99] 1/12/2022 CVPR'2004 6

Dynamic Markov Network Fig. 2. Dynamic Markov Network (DMN) modeling target dynamics 1/12/2022 CVPR'2004 Xt={Xt, i , i=1. . L}— Target states at time t Zt={Zt, i , i=1. . L}— Image observations at time t P(Xt, i|Xt-1, i)—Dynamic model in the ith scale Zt={Zk, k=1. . t}— Image observation up to time t 7

Bayesian inference in DMN Markovian assumption The Bayesian inference is Independent dynamics model 1/12/2022 CVPR'2004 8

Sequential BP Message Passing in DMN Belief update in DMN 1/12/2022 CVPR'2004 9

Sequential BP Monte Carlo To handle non-Gaussian densities Monte Carlo implementation A set of collaborative particle filters 1/12/2022 CVPR'2004 10

Algorithm 1/12/2022 CVPR'2004 11

Experiments: bouncing ball Sudden dynamics changes fail the single particle filters The tracking result of the Sequential BP 1/12/2022 CVPR'2004 12

Experiments: dropping frames Dropping 9/10 of the video frames BP iteration at a specific time instant 1/12/2022 CVPR'2004 13

Experiments: shaking camera 1/12/2022 CVPR'2004 14

Experiments: scale changes 1/12/2022 CVPR'2004 15

Conclusion& future work Contributions n n n A new multi-scale tracking approach A rigorous statistical formulation A sequential BP algorithm with Monte Carlo Future work n n 1/12/2022 Theoretic study& comparison of the BP with the mean field variational approach Learning model parameters CVPR'2004 16