Vision based Animation The Inverse of an Inverse
Vision based Animation The Inverse of an Inverse Problem Henning Biermann Chris Bregler Aaron Hertzmann Lorie Loeb Kathy Pullen Danny Yang 2/14/00
The Standard Vision Pipeline Recognition Modeling Capture Image / Video 2/14/00
Vision for Graphics / Animation Image / Video Synthesis Modeling Capture Image / Video 2/14/00
Vision for Graphics / Animation Expressive Animations Modeling Capture Image / Video 2/14/00
The Standard Analogy 2/14/00
The Graphics Problem 2/14/00
The Vision Problem 2/14/00
Vision Based Animation 2/14/00
Vision for Graphics Image / Video Synthesis Modeling Capture Image / Video 2/14/00
Vision for Graphics Image / Video Synthesis Richer Modeling Capture Image / Video 2/14/00
Vision for Animation Image / Video Synthesis Kinematics Dynamics Richer Modeling Capture Image / Video 2/14/00
Models: 2 Issues • Explicit / Implicit tradeoff: Hand Coded 2/14/00 Learned
Models: 2 Issues • Explicit / Implicit tradeoff: Hand Coded Learned X Analytically derived 2/14/00 M Fit Black-Box Y
Models: 2 Issues • Explicit / Implicit tradeoff: Analytical Learned • Model / Data tradeoff: Pure Model 2/14/00 Pure Data
Models: 2 Issues • Explicit / Implicit tradeoff: Analytical Learned • Model / Data tradeoff: Pure Model 2/14/00 Pure Data
Case #1: Models of Kinematics - Approximate Kinematic Chains. - Fine-tune Kinematic Chains. - Acquisition of Nonrigid Models. 2/14/00
Approximate Kinematic Chains - Enough for Animation ? 2/14/00
Case #1: Models of Kinematics - Approximate Kinematic Chains. - Fine-tune Kinematic Chains. - Acquisition of Nonrigid Models. 2/14/00
Fine-Tune Kinematics Anna Rabinowicz 2/14/00
Fine-Tune Kinematics Kathy Pullen 2/14/00
Case #1: Models of Kinematics - Approximate Kinematic Chains. - Fine-tune Kinematic Chains. - Acquisition of Nonrigid Models. 2/14/00
Acquisition of Nonrigid 3 D Kinematics - With 2/14/00 Gene Alexander, Henning Biermann, Aaron Hertzmann, Lorenzo Torresani, Danny Yang
Solution for Non-rigid Tracking / Reconstruction - Prior Model: 2 D: PCA-based (Cootes et al 95) 2/14/00 3 D: Blanz+Vetter-99
3 D Model Acquisition - Multi-view input: 2/14/00 Pighin et al 98
Solution for Rigid 3 D Acquisition Structure from Motion: - Tomasi-Kanade-92 3 D Pose 3 D rigid Object Factorization 2/14/00
Acquisition without prior model ? • No Model available ? • Model too generic/specific ? • Stock-Footage only in 2 D ? 2/14/00
Solution based on Factorization - We want 3 things: - 3 D non-rigid shape model - for each frame: - 3 D Pose - non-rigid configuration (deformation) -> Tomasi-Kanade-92: W=PS 2/14/00 Rank 3
Solution based on Factorization - We want 3 things: - 3 D non-rigid shape model - for each frame: - 3 D Pose - non-rigid configuration (deformation) -> PCA-based representations: W = P non-rigid S 2/14/00 Rank K
Factoring twice - 2/14/00
Nonrigid 3 D Kinematics from point tracks - 2/14/00
Nonrigid 3 D Kinematics from dense flow - 2/14/00
Nonrigid 3 D Kinematics from dense flow - 2/14/00
Nonrigid 3 D Kinematics from dense flow - 2/14/00
Nonrigid 3 D Kinematics from dense flow Synthesis Modeling Motion Capture 2/14/00
Case #2: Models of Dynamics Pure Model 2/14/00 Pure Data
Case #2: Models of Dynamics Pure Model Learning Controller 2/14/00 Pure Data Mocap. Library
Case #2: Models of Dynamics Pure Model Parametric 2/14/00 Pure Data Non-Parametric
Case #2: Models of Dynamics Pure Model Pure Data Kernel-Based Densities 2/14/00
Case #2: Models of Dynamics Kathy Pullen 2/14/00
Case #2: Models of Dynamics “Motion Texture” Sampling / Matching 2/14/00
Case #2: Models of Dynamics Sampling / Matching: P(Li | Lj) 2/14/00
Synthesizing Subtleties + Variations Synthesis Modeling Motion Capture Kathy Pullen 2/14/00
Synthesizing Subtleties + Variations Synthesis Modeling Motion Capture Kathy Pullen 2/14/00
Synthesizing Subtleties + Variations Synthesis Modeling Motion Capture Kathy Pullen 2/14/00
Synthesizing Subtleties + Variations Synthesis Modeling Motion Capture Kathy Pullen 2/14/00
Synthesizing Subtleties + Variations Synthesis Modeling Motion Capture Kathy Pullen 2/14/00
Vision Based Animations Richer Modeling 2/14/00 Kinematics Dynamics
Human Movement Research Computer Science: Vision / Graphics Art: Animation / Dance 2/14/00 Bio-Mechanics: Motion Models/Analysis Medicine/ Neurology: Movement Disorders
Model Data 2/14/00
Expressive Animation Model Data 2/14/00
Human Movement Research Computer Science: Vision / Graphics Art: Animation / Dance 2/14/00 Bio-Mechanics: Motion Models/Analysis Medicine/ Neurology: Movement Disorders
Project Mariposa: Animation Principles Lorie Loeb 2/14/00
Project Mariposa: Animation Principles Lorie Loeb Realism is out 2/14/00
Project Mariposa: Animation Principles Lorie Loeb Realism is out Expressiveness is in 2/14/00
Expressive Animation Model Data 2/14/00
Controversy: Motion-Capture based Animation Model Data 2/14/00
Controversy: Motion-Capture based Animation Model Data 2/14/00
Controversy: Motion-Capture based Animation Model Data 2/14/00
Controversy: Motion-Capture based Animation Model Data 2/14/00
Synthesis by Analysis Animations 2/14/00
Enough for Animation ? Expressive Animations Video/Mocap 2/14/00
Expressive Animation Expressive Output Cartoon Capture 2/14/00
- Slides: 62
