A Layered Deformable Model for Gait Analysis Haiping

A Layered Deformable Model for Gait Analysis Haiping Lu, K. N. Plataniotis and A. N. Venetsanopoulos The Edward S. Rogers Sr. Department of Electrical and Computer Engineering University of Toronto

Outline l l l Motivation Overview The layered deformable model (LDM) LDM body pose recovery Experimental results Conclusions Haiping Lu FG 2006, Southampton, UK 2

Motivation l l Automated Human identification at a distance • Visual surveillance and monitoring applications • Banks, parking lots, airports, etc. USF Human. ID Gait Challenge problem Articulated human body model for gait recognition Manually labeled silhouettes • Layered, deformable Haiping Lu FG 2006, Southampton, UK 3

Overview Manual labeling LDM recovery Automatic extraction LDM recovery Haiping Lu FG 2006, Southampton, UK 4

The Layered Deformable Model (LDM) l Trade-off: l Match manual labeling: l Assumptions: • Complexity Vs. descriptiveness • Close to human’s subjective perception • Fronto-parallel, from right to left. Haiping Lu FG 2006, Southampton, UK 5

LDM – 22 Parameters l Ten segments l Static: l • • Lengths (6) Widths (3) Dynamic • • Positions (4) Angles (9) Haiping Lu FG 2006, Southampton, UK 6

LDM –Layers and deformation l Four layers l Deformation: Haiping Lu FG 2006, Southampton, UK 7

LDM – Summary l Summary: Realistic with moderate complexity • Compact: 13 dynamic parameters • Layered: model self-occlusion • Deformable: realistic limbs • Resemblance to manual labeling Haiping Lu FG 2006, Southampton, UK 8

Manual silhouettes pose estimation (ground truth & statistics) l Limb joint angles: • Reliable edge orientation • Spatial–Orientation mean-shift (mode- seeking): dominant modes limb orientation l Others: • Joint positions, limb widths and lengths • Simple geometry • Torso: bounding box: • Head: “head top” and “front face” Haiping Lu FG 2006, Southampton, UK 9

Post-processing l Human body constraints: l Temporal smoothing • Parameter variation limits • Limb angles inter-dependency • Moving average filtering Haiping Lu FG 2006, Southampton, UK 10

Automatic pose estimation l l l Silhouette extraction (ICME 06, Lu, et al. ) Static parameters • Coarse estimations: statistics from Gallery set Silhouette information extraction based on ideal human proportion: • Height, head and waist center, joint spatialorientation domain modes of limbs Haiping Lu FG 2006, Southampton, UK 11

Ideal proportion of the human eighthead-high figure in drawing Haiping Lu FG 2006, Southampton, UK 12

Automatic pose estimation l Dynamic parameters: • Geometry on static parameters and silhouette information, constraints. l Limb switching detection l Post-processing: smoothing • Thighs & lower legs: variations of angles. • Arms: opposite of thighs • Frames between successive switch Haiping Lu FG 2006, Southampton, UK 13

Experimental results l l l 285 sequences from five data sets, one gait cycle each sequence. Imperfection due to silhouette extraction noise and estimation algorithm Feedback LDM recovery to silhouette extraction process may help. Haiping Lu FG 2006, Southampton, UK 14

LDM recovery results l Raw l LDM manual l LDM auto Haiping Lu FG 2006, Southampton, UK 15

LDM recovery example (revisit) Manual labeling LDM recovery Silhouette extraction LDM recovery Haiping Lu FG 2006, Southampton, UK 16

Angle estimation – left & right thighs From manual silhouettes Haiping Lu From automatically extracted silhouettes FG 2006, Southampton, UK 17

Error rate (in percentage) for lower limb angles Haiping Lu FG 2006, Southampton, UK 18

Conclusions l l A layered deformable model for gait analysis • • 13 Dynamic and 9 static parameters Body pose recovery from manual (ground truth) and automatically extracted silhouettes. Average error rate for lower limb angles: 7% Overall: close match to manual labeling, accurate & efficient model for gait analysis Future work: model-based gait recognition Haiping Lu FG 2006, Southampton, UK 19

Acknowledgement l Thanks Prof. Sarkar from the University of South Florida (USF) for providing the manual silhouettes and Gait Challenge data sets. Haiping Lu FG 2006, Southampton, UK 20