Facial Animation Wilson Chang Paul Salmon April 9

Facial Animation Wilson Chang Paul Salmon April 9, 1999 Computer Animation University of Wisconsin-Madison

Papers Used § Bregler C. , Covell M. , Slaney M. , Video Rewrite: Driving Visual Speech with Audio. In SIGGRAPH 97 Conference Proceedings. ACM SIGGRAPH, August 1997 § Guenter B. , Grimm C. , Wood D. , Malvar H. , Pighin F, Making Faces. In SIGGRAPH 98 Conference Proceedings. ACM SIGGRAPH, July 1998 § Pighin F, Hecker J. , Lischinski D. , Szeliski R. , Salesin D. , Synthesizing Realistic Facial Expressions from Photographs. In SIGGRAPH 1998. § Waters K. , A Muscle Model for Animating Three-Dimensional Facial Expression. In SIGGRAPH 1987.

Motivation § § Creation of Virtual Characters Teleconferencing & Video Compression Simulated Movement Facial Surgery Planning

Why facial animation is hard. § Humans are very good at reading expressions. § Any slight deviation from a “correct” expression will be immediately noticed. § Deep-rooted instinct.

Three general catagories § 2 -D Facial Model § 3 -D Facial Model § Muscular Model

2 -D Facial Animation § Video Rewrite - modify and sync an actors’ lip motion to a new soundtrack. § Keyframe approach. § Uses vision techniques to track mouth movement.

Video Rewrite registration § Hand annotation of 26 images with 54 eigenpoints each. § Morph pairs to 351 images. § Learn eigenpoint model. § Warp images to standard reference plane. § Eigenpoint analysis.

Audio Analysis § Video Rewrite uses TIMIT speech database. § Triphones - emphasize middle. § “teapot” = /SIL-T-IY/, /T-IY-P/, /IY-P-AA/, /P-AA-T/, /AA-T-SIL/

Video Synthesis § Triphone Footage selection error = Dp + (1 - )Ds § Dp phoneme-context distance. § Ds distance between lip shapes. l Overall Lip Width & Height l Inner Lip Height l Height of Visible Teeth

Finish Synthesis § Compress and Stretch video. § Align and blend mouth to face.

Results § Good Sync and natural articulation. § Missing Triphones result in unnatural speech

Making Faces § Motion capture. § 3 D mesh via Cyberware Laser scanner. § Deformed by l Position of 128 Dots • Manual identification - 1 st frame • Tracked by vision techniques § Texture Extraction l l Dot removal. Cylindrical map.

Synthesizing Realistic Facial Expressions from Photographs § 3 D facial models derived from photographs. § Smooth transitioning between model expressions. § Adaptation from one model to another.

Model Fitting § Generic 3 D mesh model. § Pose Recovery - using multiple subject views: l l l Identify feature points. Deduce camera pose. Iteratively refine the generic face model.

Model Fitting § Scattered Data Interpolation: l l Interpolate mesh between feature points. Uses radial basis functions. § Correspondence based shape refinement: l l l Use less accurate correspondences. Polylines for eyebrows, eyelids, lips, etc. Not used in pose processing due to error.

Texture Extraction § View independent vs View dependent. § Weight maps- bias selection of original photograph: l l Self-occlusion. Smoothness. Positional certainty. View similarity.

View Dependent Texture Extraction § § Select best photographs. Draw model for each photograph. Blend rendered image. Pros l adds detail. § Cons l l sensitive to original photo. More memory, slower.

View Independent Texture Extraction § Blend photographs to form single texture. l Map onto virtual cylinder.

View Independent Texture Extraction § Blurry

Special Case Textures § § § Fine Detail - hair. Occlusion - eyes, teeth. Intricate Projection - ears. Shadowing - eyes, teeth Solutions l l Use photo with highest visibility. Simulate shadowing

Expression Morphing § Simplified by common mesh. § Linearly interpolated vertices. § Blend result of rendering with each texture. § Synthesize new expressions via: l l l Global blend. Regional blend. Painterly interface.

Results § Smooth transitioned expressions:

Results § Applied transitions to different human subject:

Our conclusions § Good results between models. § Relatively inexpensive equipment. § Notable manual processing.

Muscular Modeling § Easy generalized across models. § 22 muscle groups § Facial Action Coding System (Ekman, Wallace) - Action Unit parameterization

Anatomy

Skin as Mesh § Nodal mobility l l Tensile Strength of skin Proximity to muscle attachment Depth of tissue & proximity to bone Elasticity & interaction with other muscles § Network of springs l p = F/k

Mesh expression examples

Muscle types modeled § Linear/parallel muscles § Sphincter muscles

Linear/parallel muscles

Sphincter muscles

Animating § § Not in paper Build a library Abstract language Keyframe