Probabilistic Tracking and Recognition of Nonrigid Hand Motion

Probabilistic Tracking and Recognition of Non-rigid Hand Motion Huang Fei, Ian Reid Department of Engineering Science Oxford University

The Problem • Simultaneous Tracking and Recognition • Articulation and Self-Occlusion • Cluttered Background Scene and Occlusion Two Successive Frames From A Video Sequence

Previous Research • Kinematic Model v. s. Appearance Model • Toyama & Blake “Metric Mixture Tracker” Merits: -Exemplar v. s. Model -Spatial-Temporal Filtering Disadvantages: -Contour (Edges) v. s. Region (Silhouettes) -Joint Observation Density of Two Independent Processes

Method • System Diagram of Joint Bayes Filter

• The Interaction Between Two Components in Joint Bayes Filter

Discrete Appearance Tracker • Non-Rigid Appearances v. s. Speech Signal • Assumption: -Representative Hand Appearances -Non-Rigid Motion Observe Markov Dependence • The Aim of Learning: -Exemplar as Shape Tracker Representation -Articulated Human Motion Dynamics

Visualizing Non-Rigid Hand Motion • Local Linear Embedding Algorithm (S. Roweis & L. Saul 2000)

Robust Region Tracker • Use Probabilistic Colour Histogram Tracker et. al. ECCV 2002) As Global Region Estimator (P. Prez

Experiments • Tracking Global Region and Articulated Motion Frame 1 Frame 2 Frame 3 Frame 4 Frame 5 Frame 6 Frame 7 Frame 8 Frame 9 Frame 10 Frame 11 Frame 12 Frame 13 Frame 14 Frame 15

• Coping with Occlusion Clutter Frame 1 Frame 6 Frame 11 Frame 2 Frame 7 Frame 12 Frame 3 Frame 8 Frame 13 Frame 4 Frame 5 Frame 9 Frame 10 Frame 14 Frame 15

Conclusion • • Two Independent Dynamic Processes Two Bayesian Tracker =>Joint Bayes Filter Robust Global Region Estimator Robust State-Based Inference