Human Computer Intelligent Interaction Emotion Recognition and Synthesis

Human Computer Intelligent Interaction Emotion Recognition and Synthesis Professor Thomas S. Huang Professor Mark Hasegawa-Johnson Dr. Zhihong Zeng (Beckman Fellow) Graduate Students: Raymond Fu Yuxiao Hu Hao Tang Jilin Tu

Motivation • Computers are ubiquitous • On the job, people want computers to be tools, but… • At home, people want computers to be friends/mentors: – – – Interactive tutoring programs Computer story telling (interactive DVD/ automatic audio books) Smart kiosk Driver monitoring Personal physical fitness trainer • Emotion recognition research at the Beckman Institute – Text – Speech – Facial expression • Emotion synthesis research at the Beckman Institute – Expressive speech – Emotive facial expression

Facial Expression: Recognition and Synthesis Demos Available Today • Robust real-time face tracking algorithms • Emotion recognition algorithm (based on tracking results) • Synthesize facial expressions (animation)

Video Demonstrations • Face tracking and emotion recognition – Detecting Smile and Surprise • Synthesis of facial expression – I am now happy – I am very sad today

Emotion of Mona Lisa

A Vision for the Future of HCII In the future, every object will be optionally computerized; every action will be optionally computer-assisted. • Computers as collaborators – Whole-body avatars – Gesture analysis, synthesis – Human activity recognition • Universal access to computer technology – HCI for users with disability • Audiovisual speech recognition for talkers w/dysarthria – Multilingual / multi-dialect HCII • Computer-assisted second language acquisition • Language technology for non-native speakers