Human Pose detection Abhinav Golas S Arun Nair

Human Pose detection Abhinav Golas S. Arun Nair

Overview Problem n Previous solutions n Solution, details n

Problem Segmentation of humans from video capture n Pose detection (by fitting onto body model) n Resistant to noise (background etc. ) n

Previous procedures n View problem as sequential process 1. 2. ¨ Segmentation Pose detection Problems: ¨ ¨ ¨ Not using prior knowledge of “what a human looks like” in segmentation Uses only information from detected “foreground” for pose detection All available information not used

Solution n Combine segmentation and pose detection as a single step ¨ Uses all available information in frame (for pose detection) ¨ Uses prior knowledge of human body for better segmentation n Pose. Cut: Bray, Kohli, Torr ¨ Model segmentation as Bayesian labeling problem with 2 labels: foreground, background

Details Model problem as energy minimization problem – model as an MRF n Use a basic stickman model as a human body model n Adaptive model for background – GMM n Neighbourhood terms – Generalised Potts model n

MRF – Markov Random Fields Markov property for time: P(event: t) depends on events at times k<t n Markov property for space: P(event: x) depends on events at N(x) – neighbourhood of x n Use Gibbs energy model for solving n We use neighbourhood of 8 pixels n

Stickman model n n Basic model 26 degrees of freedom

GMM – Gaussian Mixture Model each pixel of image as a weighted sum of Gaussian functions n Adapt functions using each new frame n Pixel matches expected value – background, else foreground n

Execution details n For each frame ¨ Calculate weights for GMM, Potts model ¨ For given value of 26 vector (based on degrees of freedom of stickman model) calculate energy cost for stickman model (by distance transform) ¨ Minimize energy for Bayesian labeling by graph cut ¨ Minimize 26 vector by repeated graph cuts by Powell's algorithm

Sample results n n n A – original frame B – segmentation by colour likelihood and contrast terms C – when GMM terms are taken D – with pose prior components E – deduced pose

Comparisons