Chaotic Neural Networks and MultiDimensional Data Analysis in

Chaotic Neural Networks and Multi-Dimensional Data Analysis in Biometric Applications Presented by: Kushan Ahmadian Department of Computer Science, University of Calgary kahmadia@ucalgary. ca 1

Outline �Introduction � Research Contributions � Motivations �Background Research � Neural Network � Dimensionality Reduction � Biometrics �Proposed Methodology � Subspace Clustering � Chaotic Associative Memory � Overall System Architecture �Preliminary Experimental Results �Conclusion and Future Work 2

Research Goal 1 Introduction 2 Background 3 Methodology 4 Experiments 5. Conclusions The purpose of my research is to develop a novel methodology based on the subspace clustering dimension reduction technique and chaotic neural network to improve the performance and circumvention of multimodal biometric system. 3

My Research Contributions 1 Introduction 2 Background 3 Methodology 4 Experiments �A novel correlation clustering approach accounting for the feature relevance and/or feature correlation problem in multi-modal biometric system 5. Conclusions �Design and utilization of a chaotic associative neural memory with original noise injection policy to learn the patterns of biometric features �Designing and evaluating the performance of the system comparing the results to the postclassification (decision level) fusion results 4

Motivation 1 Introduction 2 Background 3 Methodology 4 Experiments 5. Conclusions �Alleviate problems of current dimensionality reduction methods such as “curse of dimensionality” and “locality” by proposing a new subspace clustering based dimensionality reduction for biometric data. �Reducing the FAR (False Acceptance Rate) and FRR (False Rejection Rate) by minimizing the effect of noise, template aging and other errors using a novel feature selection method. �Utilizing a brain-like associative memory (chaotic neural network) for the first time in biometric to enhance the ability of patternbased data retrieval from memory. 5

Biometrics 1 Introduction 2 Background 3 Methodology 4 Experiments 5. Conclusions Biometrics comprises methods for uniquely recognizing humans based upon one or more intrinsic physical or behavioral traits. Source: http: //360 biometrics. com/ 6

Multi-modal biometric Matchers Fused Authors Level of Fusion methodology Hand Kumar et al (2003) Feature, Match score Feature concatenation/ sum rule Pulmprint (geometry, You, et al (2004) Decision Hierarchical matching Jain and Ross (2002) Sensor, feature Mosaicing of local texture) Fingerprint(2 impressions) templates Fingerprint Wilson et al (2004) Match score Sum rule Face (global and local Ferrez et al (2005) Feature level Feature concatenation Voice Cheung et al(2004) Match score Zero sum fusion Face, Iris and Gavrilova and Rank Level Markov model Signature Monwar (2009) features) Examples of fusion methods. 7

Matchers Fused Authors Level of Fusion methodology Hand Kumar et al (2003) Feature, Match score Feature concatenation/ sum rule Pulmprint (geometry, You, et al (2004) Decision Hierarchical matching Jain and Ross (2002) Sensor, feature Mosaicing of local texture) Fingerprint(2 impressions) templates Fingerprint Wilson et al (2004) Match score Sum rule Face (global and local Ferrez et al (2005) Feature level Feature concatenation Voice Cheung et al(2004) Match score Zero sum fusion Face, Iris and Gavrilova and Rank Level Markov model Signature Monwar (2009) features) 8

Feature Space and Dimensionality Reduction � Transform the data in the high-dimensional space to a space of fewer dimensions. 1 Introduction 2 Background 3 Methodology 4 Experiments 5. Conclusions Subspace obtained by PCA and ideal resulted subspace projected clustering (Han and Kamber, 2001) DBSCAN (Ester et. al. 1996) Specifications of clustering methods (Achtert and Böhm, 2007). 9

Reducing Dimensionality by Subspace Analysis �The principle for subspace analysis is based on a generalized description of spherical coordinates. �A point in data space is represented by a sinusoidal curve in parameter space P. �A point in parameter space corresponds to a (d − 1)-dimensional hyperplane in data space. 10

Neural network 1 Introduction 2 Background 3 Methodology 4 Experiments 5. Conclusions �Chaotic Neural Networks un pattern Rec. (Wang, 2006) �CSA (Chen and Aihara, 1997) �Applications of Optimization (Wang, 1998) 11

Traditional System Architecture 1 Introduction 2 Background 3 Methodology 4 Experiments Biometric Database User samples Eigenfaces vectors PCA-based dimensionality reduction 5. Conclusions Learner 1 Aggregation method Yes/No Traditional multimodal architecture 12

Proposed System Architecture 1 Introduction 2 Background 3 Methodology 4 Experiments Biometric Database Mean faces Novel representation of Feature Vector User samples 5. Conclusions Train neural networks Y Train? Testing neural network N Verified Yes/No ? Proposed biometric recognition system 13

Subspace Clustering Step 1 1 Introduction For each person (class) compute the mean image 2 Background 3 Methodology 4 Experiments 5. Conclusions Mean image for each class Input Data 14

Eigenface images • The eigenvectors are sorted in order of descending eigenvalues and the greatest eigenvectors are chosen to represent face space. • This reduces the dimensionality of the image space, yet maintains a high level of variance between face images throughout the image subspace. • Any face image can then be represented as a vector of coefficients, corresponding to the ‘contribution’ of each eigenface. Each eigenvector can be displayed as an image and due to the likeness to faces (FERET database) 15

Subspace Clustering Step 2 1 Introduction 2 Background 3 Methodology 4 Experiments 5. Conclusions Number of dimensions: m (number of mean images) Number of points in the high dimensional space: x*y 16

Reducing Dimensionality by Subspace Analysis 1 Introduction 2 Background 3 Methodology 4 Experiments 5. Conclusions • Three points p 1, p 2, p 3 on a plane (b) Corresponding parameterization functions. 17

18

Reducing Dimensionality by Subspace Analysis 1 Introduction 2 Background 3 Methodology 4 Experiments 5. Conclusions �Find the clusters within an error range of ε. �Use the mean vector as the candidate for the members of a cluster and create the new vector space. The number of points of the new space is: M << x*y Next, we try to learn the pattern using a learner (Chaotic Neural Network) 19

Associative Memory 1 Introduction 2 Background 3 Methodology 4 Experiments 5. Conclusions • The neuron signals comprise an output pattern. • The neuron signals are initially set equal to some input pattern. • The network converges to the nearest stored pattern 20

Chaotic Associative Memory 1 Introduction 2 Background 3 Methodology 4 Experiments 5. Conclusions Chaotic and period doubling noise injection policies • To overcome the drawback of nonautonomous methods is their blind noiseinjecting strategy • Proposing the adjacency matrix to evaluate the chances of a neuron to receive chaotic noise 21

Fingerprint Neural Based Method – Case Study 1 Introduction 2 Background 3 Methodology 4 Experiments 5. Conclusions The general goal is to train the network using the Delaunay triangulation of minutiae points. 22

DT based Matching -Experimental Results 1 Introduction 2 Background 3 Methodology 4 Experiments 5. Conclusions 23

Multimodal Training Phase 1 Introduction 2 Background 3 Methodology User 1 4 Experiments User 2 User N Data acquisition 5. Conclusions Biometric 1 Biometric 2 Biometric 3 Analyzing and obtaining the best set of feature vectors Training the chaotic associative memory with the obtained vectors 24

User Testing 1 Introduction Data acquisition Biometric 1 Biometric 2 Biometric 3 2 Background 3 Methodology 4 Experiments Dimensionality reduction – New feature space 5. Conclusions User’s obtained feature space vector Feeding the new vector space into the associative memory Network Convergence (Matching) Yes/ No 25

Experimental Results 1 Introduction 2 Background 3 Methodology 4 Experiments 5. Conclusions � My method: Subspace Clustering (SC) and Chaotic Noise Neural Network (CNNN) � Compared methods: � Simple-Sum (SS), Min-Score (MIS), Max-Score (MAS), Matcher Weighting (MW), User Weighting (UW) � Min-Max Score (MM), Zero Score (ZS), Tanh (TH), Quadratic Line Quadric (QLQ), Subspace Clustering (SC) 26

Experimental Results 1 Introduction 2 Background 3 Methodology 4 Experiments 5. Conclusions EER rate, SC with different fusion techniques EER rate, CNN with different normalization techniques 27

Experimental Results 1 Introduction 2 Background 3 Methodology 4 Experiments 5. Conclusions EER rate, Combination of different fusion and normalization techniques 28

Conclusions 1 Introduction 2 Background 3 Methodology 4 Experiments 5. Conclusions �The contributions are : �Introducing method for selecting a proper set of input features to reduce the dimensionality of biometric data and consequently enhancing the performance of the system. �Introducing chaotic associative memories in biometric system, which have significant advantages over conventional memories in terms of capacity of the memory �Implementing and enhancing performance of the biometric multimodal verification system. 29

Future Work 1 Introduction 2 Background 3 Methodology 4 Experiments 5. Conclusions �Continuing research on the axis-parallel subspace clustering. �Comparing to a newly proposed system where the data analysis is run over the vectors of each biometric separately. The benefit of such a system would being more tolerable over the absence of each biometric. �Enhancing the capacity of the associative memories which is the current drawback of associative based memories. �Finding a better candidate vector for subspace clustered data to improve the quality of data reduction method. �Continuing research on subspace clustering methods to further decrease FAR and FRR rates 30

Project Timeline 1 Introduction Phase Task Name Start End Duration 1 st Required course taken. Literature review on biometrics, feature selection techniques and associative memories. Problem statement formulation. Sept. 2008 Aug. 2009 12 months Sept. 2009 Aug. 2010 12 months Sept. 2010 Aug. 2011 12 months Sept. 2011 Jan. 2012 5 months Feb. 2012 Jul. 2012 6 months 2 Background 3 Methodology 2 nd 4 Experiments 5. Conclusions 3 rd 4 th 5 th Required course taken. Prototype system development with various associative neural memory models including chaotic neural network and send results to different peer reviewed journals and conferences for reviewer's feedback. Complete system development with different feature selection policies. Comments from various reviewers are highly considered during the complete system development. Validation of developed system. Performing performance analysis of the proposed system against different biometric databases. The results will be communicated through appropriate venues and consequently modify the system according to the feedback The thesis will be prepared as part of the Ph. D degree requirements. 2008 2009 2010 2011 2012 31

Key References � Bohm C, Kailing K, Kriegel H. P, Kroger P, (2004) Density Connected Clustering with Local Subspace Preferences, Proceedings of the Fourth IEEE International Conference on Data Mining, p. 27 -34, � Jain A. K. , Ross A. , and Prabhakar A. (2004) An Introduction to Biometric Recognition. IEEE Transactions on Circuits and Systems for Video Technology, Special Issue on Imageand Video-Based Biometrics, 14(1): 4– 20. � Kriegel H. P, Kröger P, Zimek Z. (2009) Clustering High Dimensional Data: A Survey on Subspace Clustering, Pattern-based Clustering, and Correlation Clustering � ACM Transactions on Knowledge Discovery from Data pp. 1 -58, � Wang L, and Shi H (2006) A gradual noisy chaotic neural network for solving the broadcast scheduling problem in packet radio networks. IEEE Transactions on neural networks, vol 17, no. 4: 989 - 1001 � Zhao L. and Yang Y. H. , (1999) “Theoretical Analysis of Illumination in PCA-Based Vision Systems, ” Pattern Recognition, Vol. 32, No. 4, pp. 547 -564. � Belhumeur P. N, Hespanha J. P, and Kriegman D. J. (1997) Eigenfaces vs. Fisherfaces: recognition using class specific linear projection, IEEE Trans. Pattern Analysis and Machine Intelligence, Vol. 19, No. 7, pp. 711 -720. 32

Publications � K. Ahmadian and M. Gavrilova, “Transiently Chaotic Associative Network for Fingerprint Image Analysis”, Special Issue on Intelligent Computing for Multimedia Assurance in the International Journal of Neural Network World, A. Abraham editor, 2009 -2010, 21 pages, in print (accepted in May 2009) � K. Ahmadian and M. Gavrilova “On-Demand Chaotic Neural Network for Broadcast Scheduling Problem”, Journal of Supercomputing, 18 pages, Springer ( accepted with minor revisions in May 2010). � K. Ahmadian, and M. Gavrilova, “Multi-objective Evolutionary Approach for Biometric Fusion, ” IEEE International Conference on Biometrics and Kansei Engineering, pp. 12 -17, June 25 -28, Poland, 2009. � K. Ahmadian, M. Gavrilova and D. Taniar, “Multi-criteria Optimization in GIS: Continuous K-Nearest Neighbor search in mobile navigation, ” ICCSA, pp. 574589, March 2010, Japan. 33