Evidential modeling for pose estimation Fabio Cuzzolin Ruggero

Evidential modeling for pose estimation Fabio Cuzzolin, Ruggero Frezza Computer Science Department UCLA

Myself n n Master’s thesis on gesture recognition at the University of Padova Ph. D. thesis on theory of evidence Post-doc in Milan with the Image and Sound Processing group Post-doc at UCLA in the Vision Lab

My past work… n geometric approach to theory of belief functions n n space of belief functions geometry of Dempster’s rule

. . again. . n n algebra of compatible frames linear independence on lattices action recognition and object tracking metrics on the space of dynamical models

… and today’s talk 1 the pose estimation problem 2 model-free pose estimation 3 evidential model 4 experimental results

Pose estimation n estimating the “pose” pose (internal configuration) of a moving body from the available images t=0 n salient image measurements: features t=T

Model-based estimation n if you have an a-priori model of the object. . you can exploit it to help (or drive) the estimation example: kinematic model

Model-free estimation n if you do not have any information about the body. . the only way to do inference is to learn a map between features and poses directly from the data this can be done in a training stage

Collecting training data n motion capture system n 3 D locations of markers = pose

Training data n n n when the object performs some “significant” movements in front of the camera … … a finite collection of configuration values are provided by the motion capture system q 1 q. T y 1 y. T … while a sequence of features is computed from the image(s)

Learning feature-pose maps n n n Hidden Markov models provide a way to build feature-pose maps from the training data a Gaussian density for each state is set up on the feature space -> approximate feature space map between each region and the set of training poses qk with feature value yk inside it

Evidential model n n n approximate feature spaces. . and approximate parameter space. . form a family of compatible frames: the evidential model

Estimation new features are represented as belief functions. . these belief functions are projected onto the approximate parameter space. . and combined through Dempster’s rule a point-wise estimate of the pose is obtained by probabilistic approximation

Human body tracking four markers on the right arm n n six markers on both legs n two experiments, two views

Feature extraction 38 38 161 94 n 185 three steps: original image, color segmentation, bounding box

Performances n n n comparison of three models: left view only, right view only, both views “left” model ground truth n estimate associated with the “right” model n pose estimation yielded by the overall model

Estimation errors Euclidean distance between real and predicted marker position n 8 cm n marker 2 3 cm n marker 4

Visual estimate n compares the actual image with the weighted sum of the training images

Conclusions n n pose estimation of unknown objects is a difficult task a bottom-up model has to be built from the data in a training session the DS framework allows to formalize the idea of feature-pose maps in a natural way through the notion of compatible frames Dempster’s combination provides a method to integrate features to increase robustness