PROJECT 10 FACIAL EMOTION RECOGNITION BASED ON MOUTH
PROJECT 10 FACIAL EMOTION RECOGNITION BASED ON MOUTH ANALYSIS 1 SSIP 08, Vienna http: //www. we-hope-project 10 -will-win. info
THE PROJECT Objective : To recognize emotional state / expression using mouth information Input: Mouth images (no make-up) Output: Emotional State/ Expression Happy, Neutral, Sad 2 http: //www. we-hope-project 10 -will-win. info
THE TEAM Péter Web programmer Sofia programmer Kornél programmer Naiem researcher Kamal programmer 3 http: //www. we-hope-project 10 -will-win. info
THE TASKS Create facial expressions photographic database Segment the mouth in the input image Use suitable features for expression characterization Design a reliable classifier to distinguish between different mouth expressions 4 http: //www. we-hope-project 10 -will-win. info
SSIP LIPS DATABASE Happy, Neutral and Sad Photos of SSIP students and lecturers Happy Neutral Sad Thank you all!!! 5 http: //www. we-hope-project 10 -will-win. info
MOUTH SEGMENTATION Input Image HSV Space - Hue Thresholding Morphological Operations 6 http: //www. we-hope-project 10 -will-win. info
SEGMENTATION RESULTS… And Segmentation Problems… 7 http: //www. we-hope-project 10 -will-win. info
LIPS FEATURES EXTRACTION Detect the leftmost and rightmost lip points Normalize images (rotation, translation and scaling) Calculate features Eccentricity Convex Area Minor Axis Ratio of Upper to Lower Lip 8 http: //www. we-hope-project 10 -will-win. info
EXPRESSION CLASSIFICATION SVM Classifier Two Stage Classification Mouth Features ☺ 9 http: //www. we-hope-project 10 -will-win. info
RESULTS 1 Differences between different classes were found to be statistically significant (p<0. 01) Classification Accuracy Stage 1 (Sad / Not Sad) 88% Stage 2 (Happy/ Neutral) 62% 10 http: //www. we-hope-project 10 -will-win. info
RESULTS 2 11 http: //www. we-hope-project 10 -will-win. info
Conclusion Mouth information is often insufficient for recognizing facial expression / emotional state Other face features such as eyes and eyebrows can contribute in emotional state recognition Future Work Acquire larger database for training and testing Test different facial expressions (such as anger and disgust) Other classifiers: NN, FIS 12 http: //www. we-hope-project 10 -will-win. info
GUI 13 http: //www. we-hope-project 10 -will-win. info
REFERENCES M. Gordan, C. Kotropoulos, I. Pitas, “Pseudoautomatic Lip Contour Detection Based on Edge Direction Patterns” J. Kim, S. Na, R. Cole, “Lip Detection Using Confidence-Based Adaptive Thresholding” F. Tang, “Facial Expression Recognition using AAM and Local Facial Features” M. Pantic, M. Tomc, L. Rothkrantz , “A Hybrid Approcah to Mouth Features Detection” 14 http: //www. we-hope-project 10 -will-win. info
Thank you for your attention!!! 15 http: //www. we-hope-project 10 -will-win. info
- Slides: 15