PROJECT 10 FACIAL EMOTION RECOGNITION BASED ON MOUTH

PROJECT 10 FACIAL EMOTION RECOGNITION BASED ON MOUTH ANALYSIS 1 SSIP 08, Vienna http: //www. we-hope-project 10 -will-win. info

THE PROJECT Objective : To recognize emotional state / expression using mouth information Input: Mouth images (no make-up) Output: Emotional State/ Expression Happy, Neutral, Sad 2 http: //www. we-hope-project 10 -will-win. info

THE TEAM Péter Web programmer Sofia programmer Kornél programmer Naiem researcher Kamal programmer 3 http: //www. we-hope-project 10 -will-win. info

THE TASKS Create facial expressions photographic database Segment the mouth in the input image Use suitable features for expression characterization Design a reliable classifier to distinguish between different mouth expressions 4 http: //www. we-hope-project 10 -will-win. info

SSIP LIPS DATABASE Happy, Neutral and Sad Photos of SSIP students and lecturers Happy Neutral Sad Thank you all!!! 5 http: //www. we-hope-project 10 -will-win. info

MOUTH SEGMENTATION Input Image HSV Space - Hue Thresholding Morphological Operations 6 http: //www. we-hope-project 10 -will-win. info

SEGMENTATION RESULTS… And Segmentation Problems… 7 http: //www. we-hope-project 10 -will-win. info

LIPS FEATURES EXTRACTION Detect the leftmost and rightmost lip points Normalize images (rotation, translation and scaling) Calculate features Eccentricity Convex Area Minor Axis Ratio of Upper to Lower Lip 8 http: //www. we-hope-project 10 -will-win. info

EXPRESSION CLASSIFICATION SVM Classifier Two Stage Classification Mouth Features ☺ 9 http: //www. we-hope-project 10 -will-win. info

RESULTS 1 Differences between different classes were found to be statistically significant (p<0. 01) Classification Accuracy Stage 1 (Sad / Not Sad) 88% Stage 2 (Happy/ Neutral) 62% 10 http: //www. we-hope-project 10 -will-win. info

RESULTS 2 11 http: //www. we-hope-project 10 -will-win. info

Conclusion Mouth information is often insufficient for recognizing facial expression / emotional state Other face features such as eyes and eyebrows can contribute in emotional state recognition Future Work Acquire larger database for training and testing Test different facial expressions (such as anger and disgust) Other classifiers: NN, FIS 12 http: //www. we-hope-project 10 -will-win. info

GUI 13 http: //www. we-hope-project 10 -will-win. info

REFERENCES M. Gordan, C. Kotropoulos, I. Pitas, “Pseudoautomatic Lip Contour Detection Based on Edge Direction Patterns” J. Kim, S. Na, R. Cole, “Lip Detection Using Confidence-Based Adaptive Thresholding” F. Tang, “Facial Expression Recognition using AAM and Local Facial Features” M. Pantic, M. Tomc, L. Rothkrantz , “A Hybrid Approcah to Mouth Features Detection” 14 http: //www. we-hope-project 10 -will-win. info

Thank you for your attention!!! 15 http: //www. we-hope-project 10 -will-win. info