Ada Boost Multiple Feature Selection with SVMbased Component

以特徵篩選的調適性推昇法結合 支持向量機實作性別和年齡辨識 Ada. Boost Multiple Feature Selection with SVM-based Component Classifiers for Gender and Age Classification 國立清華大學 100062553 資 所 郭哲綸 指導教授 張智星

Outline Introduction Related work System flowchart Preprocessing � Feature descriptor � Dimension reduction � Classification model � Experiment Database introduction � Performance evaluation � Experimental results � Experimental analysis � Conclusions and future work 2/33

Introduction Motivation Application for gender and age classification, ex: surveillance monitoring, human computer interaction � The eighth UTMVP (第八屆由田機器視覺獎), 1 st prize: NT$500, 000 � Objective Design a robust system for face-based analysis and recognition , including gender and age classification � Win the eighth UTMVP � Problems Illumination condition � Feature extraction and selection � High dimensionality � 3/33

Related Work Gender Classification � Lian, Hui-Cheng, and Bao-Liang Lu. "Multi-view gender classification using local binary patterns and support vector machines. " Advances in Neural Networks-ISNN 2006. Springer Berlin Heidelberg, 2006. 202 -209. � Xia, Bin, He Sun, and Bao-Liang Lu. "Multi-view gender classification based on local Gabor binary mapping pattern and support vector machines. " Neural Networks, 2008. IJCNN 2008. (IEEE World Congress on Computational Intelligence). IEEE International Joint Conference on. IEEE, 2008. � Jabid, Taskeed, M. Hasanul Kabir, and Oksam Chae. "Gender classification using local directional pattern (LDP). " Pattern Recognition (ICPR), 2010 20 th International Conference on. IEEE, 2010. Age Classification � Gunay, Asuman, and Vasif V. Nabiyev. "Automatic age classification with LBP. " Computer and Information Sciences, 2008. ISCIS'08. 23 rd International Symposium on. IEEE, 2008. � Gao, Feng, and Haizhou Ai. "Face age classification on consumer images with gabor feature and fuzzy lda method. " Advances in biometrics. Springer Berlin Heidelberg, 2009. 132 -141. � Luu, Khoa, et al. "Combined local and holistic facial features for age-determination. " Control Automation Robotics & Vision (ICARCV), 2010 11 th International Conference on. IEEE, 2010. � Chen, Cuixian, et al. "Learning gabor features for facial age estimation. " Biometric Recognition. Springer Berlin Heidelberg, 2011. 204 -213. Keywords � Local Binary Pattern (LBP), Local Directional Pattern (LDP), Local Ternary Pattern (LTP), Gabor feature, Local Gabor Binary Pattern (LGBP), Support Vector Machine (SVM) 4/33

System Flowchart 1 Preprocessing 2 Feature Descriptor 3 Dimension Reduction Image Capture 4 Answer Label Ex: male, 26 years old Classification Model 5/33

System Flowchart – ➀ Preprocessing 1. 2. 3. 4. 5. Face Detection [1] Calibration [2] Resize De-colorization Histogram equalization Crop the main face Decrease error made by rotation Reduce displacement error (w*h=57*76) Use gray scale to extract feature Reduce the lighting impact [1] Viola, Paul, and Michael Jones. "Rapid object detection using a boosted cascade of simple features. " Computer Vision and Pattern Recognition, 2001. CVPR 2001. Proceedings of the 2001 IEEE Computer Society Conference on. Vol. 1. IEEE, 2001. [2] Shih, Frank Y. , and Chao-Fa Chuang. "Automatic extraction of head and face boundaries and facial features. " Information Sciences 158 (2004): 117 -130. 6/33

System Flowchart – ➁ Feature Descriptor (1/6) LBP (Local Binary Pattern) � Patches: 64, Neighbors: 8 P Face Local Pattern 0 1 1 Binary: 01111000 0 x 1 Decimal: 120 0 0 1 LBP image Original image LBP image [3] Ojala, Timo, Matti Pietikainen, and Topi Maenpaa. "Multiresolution gray-scale and rotation invariant texture classification with local binary patterns. " Pattern Analysis and Machine Intelligence, IEEE Transactions on 24. 7 (2002): 971 -987. 7/33

System Flowchart – ➁ Feature Descriptor (2/6) LDP (Local Directional Pattern) � Patches: 64, rank K: 3, Neighbors: 8 Binary: 00010011 P Face Local Pattern Decimal: 19 LDP Kirsch Masks LDP image Original image LDP image [4] Jabid, Taskeed, M. Hasanul Kabir, and Oksam Chae. “Gender classification using local directional pattern (LDP). ” Pattern Recognition (ICPR), 2010 20 th International Conference on. IEEE, 2010. 8/33

System Flowchart – ➁ Feature Descriptor (3/6) LTP (Local Ternary Pattern) � Patches: 64, threshold: 5, Neighbors: 8 P Face Local Pattern t=5 0 1 1 -1 x 1 -1 -1 1 Binary: 01111000 Decimal: 120 Binary: 00000111 Decimal: 7 LTP image Original image LTP positive image. LTP negative image [5] Tan, Xiaoyang, and Bill Triggs. "Enhanced local texture feature sets for face recognition under difficult lighting conditions. " Image Processing, IEEE Transactions on 19. 6 (2010): 1635 -1650. 9/33

System Flowchart – ➁ Feature Descriptor (4/6) Gabor Filters � Frequencies: � Orientations: 40 Gabor filters 40 magnitudes of the Gabor feature [6] Deng, Hong-Bo, et al. "A new facial expression recognition method based on local gabor filter bank and pca plus lda. " International Journal of Information Technology 11. 11 (2005): 86 -96. 10/33

System Flowchart – ➁ Feature Descriptor (5/6) LGBP (Local Gabor Binary Pattern) 0 1 1 0 x 1 0 0 1 LBP 40 magnitudes of the Gabor feature 40 LGBP maps [7] Zhang, Wenchao, et al. "Local Gabor binary pattern histogram sequence (LGBPHS): A novel non-statistical model for face representation and recognition. " Computer Vision, 2005. ICCV 2005. Tenth IEEE International Conference on. Vol. 1. IEEE, 2005. 11/33

System Flowchart – ➁ Feature Descriptor (6/6) Summary of the different features Feature Description Properties Dimension LBP Standard local binary patterns with a neighborhood of 8 pixels and a radius of 1 pixel Offers illumination invariance and is computationally efficient. However, it tends to be sensitive to noise 16384 Robust in lighting condition and aging effects 16384 More discriminant and less sensitive to noise in uniform regions 32768 Optimal localization properties in both spatial analysis and frequency domain Gabor filters offer strong illumination invariance as well as powerful descriptive features. However, the feature vector has high dimensionality 173280 LDP LTP Gabor LGBP Local directional pattern computes the edge response values in 8 different directions and encodes the texture Local ternary pattern extends LBP to 3 -valued codes 40 Gabor filters generated by 5 frequencies and 8 orientations Local binary patterns are extracted from Gabor images, where 40 different Gabor images are composed from applying Gabor kernels at different scales and orientations 655360 12/33

System Flowchart – ➂ Dimension Reduction How to find the best dimension of features in PCA and LDA � Test the effects of PCA dimension reduction on the classification via SVM and 2 -fold cross validation � Keep the best dimensions after PCA, and then test the effects of LDA dimension by the same method � The following are the figures for the basic example of gender classification using the PAL[8] as database and the LBP as feature (best PCA at No. 75, and best LDA at No. 50) [8] Minear, Meredith, and Denise C. Park. "A lifespan database of adult facial stimuli. " Behavior Research Methods, Instruments, & Computers 36. 4 (2004): 630 -633. 13/33

System Flowchart – ➃ Classification Model (1/3) Existing method: Adaboost with SVM-based component classifiers[9] (Adaboost. SVM) Proposed method: Adaboost. SVM with multiple feature selection n Use descending σ values as component classifiers n Use descending σ values plus different features as component classifiers RBFSVM + σ1 + one feature RBFSVM + σ1 + LBP RBFSVM + σ2 + one feature RBFSVM + σ2 + LGBP RBFSVM + σ3 + one feature RBFSVM + σt + one feature Adaboost. SVM V. S. RBFSVM + σ3 + Gabor RBFSVM + σt + LBP Adaboost. SVM with multiple feature selection [9] Li, Xuchun, Lei Wang, and Eric Sung. "Ada. Boost with SVM-based component classifiers. " Engineering Applications of Artificial Intelligence 21. 5 (2008): 785 -795. 14/33

System Flowchart – ➃ Classification Model (2/3) Why do we change σ rather than C in RBFSVM？ [10] Valentini, Giorgio, and Thomas G. Dietterich. "Bias-variance analysis of support vector machines for the development of SVMbased ensemble methods. " The Journal of Machine Learning Research 5 (2004): 725 -775. 15/33

System Flowchart – ➃ Classification Model (3/3) Input: a set of training samples L with labels [(x 1, y 1), (x 2, y 2), . . . , (x. N, y. N)] � Initialize: the weights of training samples: wi 1 = 1/N, for all i = 1, …, N σini is set as the scatter radius of the training sample σmin is set as the average minimal distance between any two training samples � Training Data Converge Feature Extraction Component Classifier Final Classifier RBFSVM component classifiers Compute Weighted Error Feature Selection Update Weights Select the feature with the minimum 16/33

Experiment – Database Introduction (1/3) PAL face database (total: 578 unique images) Gender Counts Age Range Counts female 351 0~20 53 male 227 21~30 174 31~40 42 41~50 35 51 - 274 [8] Minear, Meredith, and Denise C. Park. "A lifespan database of adult facial stimuli. " Behavior Research Methods, Instruments, & Computers 36. 4 (2004): 630 -633. 17/33

Experiment – Database Introduction (2/3) Some properties about MIR-Google Crawl from the Google Image Search � Keywords of the Google Image Search � � � 小孩大頭照、老人大頭照、明星大頭照 、畢業大頭照、履歷大頭照 Labeled by over 100 persons The gender and age of the face are voted by 3 persons All faces are all Asians All faces are frontal Total 1384 images 18/33

Experiment – Database Introduction (3/3) MIR-Google face database (total: 1384 images) Gender Counts Age Range Counts female 667 0~20 402 male 652 21~30 584 31~40 249 41~50 98 51 - 51 19/33

Experiment – Performance Evaluation Recognition rate � Mean average precision (MAP) � # of the corrrect data / # of the total data (0~100%) Mean ( # of correct data belonging one class / # of the total data belonging to one class) (0~100%) UTMVP scoring rule � Scoring table for age classification by UTMVP Answer Predict 0~20 21~30 31~40 41~50 50+ 0~20 3 1 0 0 0 21~30 1 2 1 0 0 31~40 0 1 2 1 0 41~50 0 0 1 2 1 50+ 0 0 0 1 3 � Score per image (total score / # of the total images) 20/33

Experimental Results – ➀ Gender Classification (1/4) Gender classification model by PAL face database We use the proposed adaboost. SVM model to train the classification models with different combination of the features � The combination of the all feature is the best (All feature: LBP+LDP+LTP+Gabor+LGBP) � - 1: LBP 2: LDP 3: LTP 4: Gabor 5: LGBP 21/33

Experimental Results – ➀ Gender Classification (2/4) PAL face database (5 -fold cross validation) Adaboost. SVM and SVM are comparable � Proposed method is the best in the Recognition rate and the MAP � Recognition Rate MAP 100% 90% 80% 70% 60% 50% 40% 30% 20% 10% 0% SVM All Feature Adaboost. SVM LBP LDP LTP Proposed Method Gabor LGBP 22/33

Experimental Results – ➀ Gender Classification (3/4) Gender classification model by MIR-Google face database We use the proposed adaboost. SVM model to train the classification models with different combination of the features � The combination of the all feature is the best (All feature: LBP+LDP+LTP+Gabor+LGBP) � - 1: LBP 2: LDP 3: LTP 4: Gabor 5: LGBP 23/33

Experimental Results – ➀ Gender Classification (4/4) MIR-Google face database (5 -fold cross validation) Adaboost. SVM and SVM are comparable � Proposed method is slightly better in the MAP � Recognition Rate MAP 100% 90% 80% 70% 60% 50% 40% 30% 20% 10% 0% SVM All Feature Adaboost. SVM LBP LDP LTP Proposed Method Gabor LGBP 24/33

Experimental Results – ➁ Age Classification (1/4) Age classification model by PAL face database We use the proposed adaboost. SVM model to train the classification models with different combination of the features � The combination of the all feature is the best (All feature: LBP+LDP+LTP+Gabor+LGBP) � - 1: LBP 2: LDP 3: LTP 4: Gabor 5: LGBP 25/33

Experimental Results – ➁ Age Classification (2/4) PAL face database (5 -fold cross validation) Adaboost. SVM and SVM are comparable � Proposed method is the best in the Recognition rate and the Score per Image � Recognition Rate Score per Image 2. 50 80% 70% 2. 00 60% 50% 1. 50 40% 1. 00 30% 20% 0. 50 10% 0% SVM All Feature Adaboost. SVM LBP LDP LTP Proposed Method Gabor LGBP 0. 00 SVM All Feature Adaboost. SVM LBP LDP LTP Proposed Method Gabor LGBP 26/33

Experimental Results – ➁ Age Classification (3/4) Age classification model by MIR-Google face database We use the proposed adaboost. SVM model to train the classification models with different combination of the features � The combination of the all feature is the best (All feature: LBP+LDP+LTP+Gabor+LGBP) � - 1: LBP 2: LDP 3: LTP 4: Gabor 5: LGBP 27/33

Experimental Results – ➁ Age Classification (4/4) MIR-Google face database (5 -fold cross validation) Adaboost. SVM and SVM are comparable � Proposed method is the best in the Recognition rate and the Score per Image � Recognition Rate Score per Image 2. 00 80% 1. 80 70% 1. 60 60% 1. 40 50% 1. 20 40% 1. 00 30% 0. 80 20% 0. 60 0. 40 10% 0% 0. 20 SVM All Feature Adaboost. SVM LBP LDP LTP Proposed Method Gabor LGBP 0. 00 SVM All Feature Adaboost. SVM LBP LDP LTP Proposed Method Gabor LGBP 28/33

Experimental Error Analysis – Gender Classification (1/2) • We can’t determine the gender of some people easily PAL database female • female female MIR-Google database female 29/33

Experimental Error Analysis – Age Classification (2/2) • It’s difficult to determine the age of some people at range 21~50 PAL database 41～ 50 • 31～ 40 21～ 30 MIR-Google database 21～ 30 31～ 40 41～ 50 21～ 30 30/33

Demonstration Gender & Age Estimation � Database: MIR-Google 31/33

Conclusion and future work Conclusion LBP is good at gender classification, LGBP is good at age classification adaboost. SVM with one feature and standard SVM are comparable Propose an adaboost. SVM model with multiple feature selection for gender and age classification, and get the better performance. Future work Add more features Feature reduction LBP -> Uniform LBP (256 -> 59) [11] LTP -> Extended LTP (6561 -> 139) [12] Do 3 -class age classification first (1~20, 21~50, 51+), and then do age regression for the range 21~50 [11] Ojala, Timo, Matti Pietikainen, and Topi Maenpaa. "Multiresolution gray-scale and rotation invariant texture classification with local binary patterns. " Pattern Analysis and Machine Intelligence, IEEE Transactions on 24. 7 (2002): 971 -987. [12] Liao, Wen-Hung. "Region description using extended local ternary patterns. " Pattern Recognition (ICPR), 2010 20 th International Conference on. IEEE, 2010. 32/33

Thanks for Listening Q&A 33/33

Appendix 1– ➀ Gender Classification (2/4) PAL face database (5 -fold cross validation) Adaboost. SVM and SVM are comparable � “All feature“ is the best � MAP Recognition Rate Classifier Adaboost. SVM SVM LBP 87. 02% 87. 72% 86. 44% 86. 70% LDP 80. 80% 80. 62% 80. 30% 79. 92% LTP 87. 37% 86. 85% 86. 64% 86. 06% Gabor 86. 33% 87. 20% 85. 56% 86. 19% LGBP 85. 12% 85. 29% 83. 55% 83. 07% All Feature 88. 93% Feature Descriptor 87. 77% 34/33

Appendix 2 – ➀ Gender Classification (4/4) MIR-Google face database (5 -fold cross validation) Adaboost. SVM and SVM are comparable � “All feature“ is slightly better than SVM in the MAP � MAP Recognition Rate Classifier Adaboost. SVM SVM LBP 86. 20% 86. 06% 86. 22% 86. 06% LDP 80. 29% 80. 36% 80. 29% 80. 38% LTP 85. 52% 86. 20% 85. 54% 86. 22% Gabor 83. 40% 84. 53% 83. 41% 84. 55% LGBP 85. 90% 86. 28% 85. 91% 86. 29% All Feature 86. 28% Feature Descriptor 86. 30% 35/33

Appendix 3 – ➁ Age Classification (2/4) PAL face database (5 -fold cross validation) Adaboost. SVM and SVM are comparable � “All feature“ is the best in the Recognition rate and the Score per Image � Recognition Rate Classifier MAP Score per Image Adaboost. SVM SVM Adaboost. SVM LBP 72. 84% 73. 18% 42. 62% 43. 67% 2. 13 LDP 69. 38% 69. 55% 39. 35% 39. 24% 2. 06 2. 07 LTP 72. 84% 73. 01% 44. 34% 45. 77% 2. 13 2. 11 Gabor 71. 97% 72. 49% 45. 01% 46. 17% 2. 13 LGBP 74. 57% 74. 05% 42. 47% 2. 17 2. 15 All Feature 75. 95% Feature Descriptor 44. 32% 2. 17 36/33

Appendix 4 – ➁ Age Classification (4/4) MIR-Google face database (5 -fold cross validation) Adaboost. SVM and SVM are comparable � “All feature“ is the best � Recognition Rate Classifier MAP Score per Image Adaboost. SVM SVM Adaboost. SVM LBP 64. 74% 64. 96% 48. 88% 48. 94% 1. 81 1. 82 LDP 62. 57% 62. 50% 48. 17% 47. 83% 1. 78 1. 77 LTP 65. 39% 65. 53% 49. 30% 49. 13% 1. 82 Gabor 65. 90% 66. 33% 50. 07% 49. 40% 1. 84 1. 85 LGBP 66. 40% 66. 33% 42. 47% 49. 40% 1. 85 All Feature 67. 85% Feature Descriptor 51. 90% 1. 86 37/33

Appendix 5 Parameter Setting Preprocessing � Feature descriptor � � � All images are resizes to w*h=57*76 LBP: patches = 64, neighbors = 8 LDP: patches = 64, neighbors = 8, rank = 3 LTP: patches = 64, neighbors = 8, threshold = 5 Gabor: LGBP: LBP + LGBP Dimension reduction PCA: 75 � LDA: 50 � Classification model SVM: LIBSVM, C = 1 � σini : the scatter radius of the training sample � σmin : the average minimal distance between any two training samples � σstep = 10 � 38/33

Appendix 6 PAL face database MIR-Goolge face database 39/33

Appendix 7 PAL face database MIR-Goolge face database 40/33

Appendix 8 Adaboost. M 2 41/33

Experimental Results – ➀ Gender Classification (x) FERET face database (5 -fold cross validation) Adaboost. SVM and SVM are comparable � Proposed method is the best in the Recognition rate and the MAP � Recognition Rate MAP 100% 90% 80% 70% 60% 50% 40% 30% 20% 10% 0% SVM All Feature Adaboost. SVM LBP LDP LTP Proposed Method Gabor LGBP 42/33

Experimental Results – ➁ Age Classification (x) FERET face database (5 -fold cross validation) Adaboost. SVM and SVM are comparable � Proposed method is the best in the Recognition rate and the Score per Image � Recognition Rate Score per Image 2. 00 60% 1. 80 50% 1. 60 1. 40 40% 1. 20 30% 1. 00 0. 80 20% 0. 60 0. 40 10% 0. 20 0% SVM All Feature Adaboost. SVM LBP LDP LTP Proposed Method Gabor LGBP 0. 00 SVM All Feature Adaboost. SVM LBP LDP LTP Proposed Method Gabor LGBP 43/33