The Chinese University of Hong Kong Jianke Zhu

The Chinese University of Hong Kong Jianke Zhu, Steven C. H. Hoi and Michael R. Lyu Computer Science & Engineering Department The Chinese University of Hong Kong, Shatin, N. T. , Hong Kong {jkzhu, chhoi, lyu}@cse. cuhk. edu. hk 3 1 • A novel two-stage scheme for online non-rigid shape recovery toward Augmented Reality applications using Active Appearance Models (AAMs). • Obtain unbroken point correspondences across multiple frames to construct 3 D shape models, and provide 2 D to. 3 D vertex correspondences in the online tracking. • An efficient algorithm is proposed to estimate both 3 D pose and non-rigid shape parameters via local bundle adjustment. 2 I. Offline Construction of 3 D Shape Model II. Estimate the 3 D Pose and Non-Rigid Shape Parameters The AAMs are built up with 140 still face images belonging to 20 individuals, 7 images for each. The fitting experiment is performed on an AAM with 14 shape parameters, 68 texture parameters, and 36335 color pixels. The demo results using AAM fitting to a single image. 4 • The M-estimator in consider of outliers . • denotes the projection of 3 D shape given the parameters A[R|T] Tracking faces using proposed method in the augmented video sequences, the axis in the displayed frames indicates the current 3 D pose of tracked subject Adding glasses and beard to the subject in the augmented video sequence, the beard is deformed along with the expression changes Re-projection errors with # of shape basis Replacing face textures The encouraging results demonstrated that our proposed scheme is effective and promising for real-time Augmented Reality applications. Dept. of C. S. & E. , The Chinese University of Hong Kong The 9 th European Conference on Computer Vision