Active Appearance Models Dhruv Batra ECE CMU Active
Active Appearance Models Dhruv Batra ECE CMU
Active Appearance Models 1. 2. 3. T. F. Cootes, G. J. Edwards and C. J. Taylor. "Active Appearance Models", in Proc. European Conference on Computer Vision 1998 (H. Burkhardt & B. Neumann Ed. s). Vol. 2, pp. 484 -498, Springer, 1998 T. F. Cootes, G. J. Edwards and C. J. Taylor. "Active Appearance Models", IEEE PAMI, Vol. 23, No. 6, pp. 681 -685, 2001 G. J. Edwards, A. Lanitis, C. J. Taylor, T. F. Cootes. “Statistical Models of Face Images Improving Specificity”, BMVC (1996)
Essence of the Idea o “Interpretation through synthesis” o Form a model of the object/image (Learnt from the training dataset) I. Matthews and S. Baker, "Active Appearance Models Revisited, " International Journal of Computer Vision, Vol. 60, No. 2, November, 2004, pp. 135 - 164.
Essence of the Idea (cont. ) o Explain a new example in terms of the model parameters
So what’s a model Model “Shape” “texture”
Active Shape Models training set
Texture Models warp to mean shape
Random Aside o Shape Vector provides alignment = 43 Alexei (Alyosha) Efros, 15 -463 (15 -862): Computational Photography, http: //graphics. cmu. edu/courses/15 -463/2005_fall/www/Lectures/faces. ppt
Random Aside o Alignment is the key 1. Warp to mean shape 2. Average pixels Alexei (Alyosha) Efros, 15 -463 (15 -862): Computational Photography, http: //graphics. cmu. edu/courses/15 -463/2005_fall/www/Lectures/faces. ppt
Random Aside o Enhancing Gender more same original androgynous more opposite D. Rowland, D. Perrett. “Manipulating Facial Appearance through Shape and Color”, IEEE Computer Graphics and Applications, Vol. 15, No. 5: September 1995, pp. 70 -76
Random Aside (can’t escape structure!) Antonio Torralba & Aude Oliva (2002) Averages: Hundreds of images containing a person are averaged to reveal regularities in the intensity patterns across all the images. Alexei (Alyosha) Efros, 15 -463 (15 -862): Computational Photography, http: //graphics. cmu. edu/courses/15 -463/2005_fall/www/Lectures/faces. ppt
Random Aside (can’t escape structure!) Tomasz Malisiewicz, http: //www. cs. cmu. edu/~tmalisie/pascal/trainval_mean_large. png
Random Aside (can’t escape structure!) “ 100 Special Moments” by Jason Salavon, http: //salavon. com/Playboy. Decades. shtml
Random Aside (can’t escape structure!) “Every Playboy Centerfold, The Decades (normalized)” by Jason Salavon 1960 s 1970 s Jason Salavon, http: //salavon. com/Playboy. Decades. shtml 1980 s
Back (sadly) to Texture Models raster scan Normalizations
PCA Galore Reduce Dimensions of shape vector Reduce Dimension of “texture” vector They are still correlated; repeat. .
Object/Image to Parameters modeling ~80
Playing with the Parameters First two modes of shape variation First two modes of gray-level variation First four modes of appearance variation
Active Appearance Model Search o Given: Full training model set, new image to be interpreted, “reasonable” starting approximation o Goal: Find model with least approximation error o High Dimensional Search: Curse of the dimensions strikes again
Active Appearance Model Search o Trick: Each optimization is a similar problem, can be learnt o Assumption: Linearity o Perturb model parameters with known amount o Generate perturbed image and sample error o Learn multivariate regression for many such perterbuations
Active Appearance Model Search o o o Algorithm: current estimate of model parameters: normalized image sample at current estimate
Active Appearance Model Search o Slightly different modeling: o Error term: o Taylor expansion (with linear assumption) o Min (RMS sense) error: o Systematically perturb and estimate by numerical differentiation
Active Appearance Model Search (Results)
- Slides: 24