Human Movement Summarization and Depiction from Videos Yijuan

Human Movement Summarization and Depiction from Videos Yijuan Lu Texas State University Hao Jiang Boston College

How do we describe human movement in videos? 2

Methods to illustration motion Muybridge’s photos to capture human movement

Methods to illustration motion 4

Manual drawing or depiction from mocap data S. Bouvier-Zappa, V. Ostromoukhov and P. Poulin, “Motion cues for illustration of skeletal motion capture data”, Symposium on Non-Photorealistic Animation and Rendering 2007.

Our method: depiction from 2 D videos video Our method automatically extracts human motion and generates depictions.

Challenges : w § Movement segmentation § Finding human body parts and linking them though time § Finding detailed trajectories of “particles” on the human subject § Generating depictions of the movement using arrows, colors, trajectories.

Movement segmentation Trajectories of particles on human subject can be used to quantify the movement at one instant. 8

Movement segmentation Comparing the trajectory features between successive frames: u v 9

Body part detection and association • Body part detections are in each single frame; we need to link them through time (tracking) • There are often errors that need to be corrected (error correction) 10

Body part tracking We formulate the tracking problem as a data association problem 11

Body part tracking We formulate the tracking problem as a data association problem 12

Correct tracking errors We use the following network to correct part tracking errors: Predictions 13

Correct tracking errors We use the following network to correct part tracking errors: Predictions Error correction becomes a shortest path problem. 14

Finding long term trajectories motion smoothness structure unary 15

Finding long term trajectories Gradient descent 16

Generate depiction Trick: polynomial curve fitting to generate pleasing Results. 17

Results 18

Results 19

Results 20

Results 21

More Results 22

Summary § It is indeed possible to automatically generate movement depictions from videos. § Not only the depiction can be treated as an art illustration, it also can serve the purpose of meaningful movement description. § Future work: finding the “optimal” movement depiction. 23

Thank You! 24