King Fahd University of Petroleum and Minerals Information

King Fahd University of Petroleum and Minerals Information and Computer Science Department Kernel-based Fuzzy Local Binary Pattern for Gait Recognition Authors: Amer G. Binsaadoon El-Sayed M. El-Alfy 1

2 Reason for Absence • Renewing passport which takes long time due to abnormal situations in Yemen • Supervisor Professor: El-Sayed M. El-Alfy Computer Science Department King Fahd University of Petroleum and Minerals alfy@kfupm. edu. sa

3 Outline • Introduction • Research Problem • Contributions • Proposed Method • Experiments and Results • Conclusions And Future Work

4 BACKGROUND • The gait biometric identifies people by their way of walking. • • Each gait sequence consists of set of frames. Gait is periodic in nature; multiple gait cycles within gait sequence.

5 BACKGROUND • Gait cycle Figure 1: Gait cycle

6 Gait Recognition System Figure 2: Generic gait recognition system

7 Gait Advantages • Gait does not request the target subject to interact in a predefined and cooperative manner such as being close to the acquisition device or standing at a specific angle. • In gait, he process of image acquisition is non-intrusive. Thus, it can be done in public areas without attracting the attention of subjects under surveillance.

8 Gait Advantages (cont. ) • The gait-based system can work at longer distances (e. g. 10 m or more), unlike most of the other biometrics. • The gait modality is difficult to disguise and can be of low resolution

9 Challenges and Limitations • It can be greatly affected by a number of conditions like type of shoes, clothes, walking surfaces, illness. • The discriminating power of walking style can also be degraded by certain physical factors such as injuries.

10 Applications • • Forensic and criminal investigation Visual surveillance Authentication and identification Access control

11 Research Problem • Most gait recognition approaches conduct experiments on frame by frame basis (Temporal model-free). • • It is expensive in term of computation and storage. The main problem of the Basic Local Binary Pattern (LBP) operator is its utilization of hard thresholding in computing its code and thus is more sensitive to noise and has less discrimination power.

12 Contributions • It investigates the application of fuzzy local binary patterns (FLBP) for extracting more discriminative features from the spatio-temporal gait representation to help in improving the process of human identification. • It proposes an effective extension of FLBP with the name “multi-kernels fuzzy local binary patterns (KFLBP)” which achieved notable and robust performance against different gait covariates.

13 Proposed Methods Figure 3: Flowchart of the proposed framework

14 Gait Period Detection • • We implemented Wang et al. [1]. • Subtract mean and divide by standard deviation to remove background component. • • • Smoothing using average moving filter. Width and height aspect ratio of silhouette bounding box. Autocorrelation to find peaks. Find peaks positions using first derivative.

Gait Energy Image (GEI) Construction • 15

16 GEI Construction(cont. ) Figure 4: GEI examples

17 GEI Construction(cont. ) Figure 5: GEI for a Male and a Female in three different covariates

18 Partitioning • To enhance the performance of the proposed features, we explore partitioning the GEI into predefined different -sized non-overlapping regions. • The partitioning has been conducted as a fraction of the subject’s height and width. • It is denoted by horizontal and vertical lines.

19 Partitioning (cont. ) • The underlying idea is to separate moving parts such as head, arms, chest, back, legs, etc. • We statically set the boundaries between regions.

20 Partitioning (cont. ) Figure 6: Two examples of non-overlapping partitioning of GEI with 5, 8, and 10 regions

21 Basic LBP operator • Properties of the neighborhood pixels to describe each pixel. Figure 7: An example of LBP computation

22 Basic LBP operator(cont. ) •

23 Basic LBP operator(cont. ) • After passing the operator over the whole or block of the image, a histogram of the binary patterns is constructed to represent the feature vector.

24 Fuzzy LBP operator (FLBP) • Fuzzy LBP incorporates fuzzy logic with LBP in order to alleviate the effect of noise on LBP and increase its distinguishing capabilities. • The difference between LBP and FLBP is that in FLBP each pixel can be characterized by more than one binary code which in turn contributes in more than one bin of FLBP histogram.

25 FLBP operator (cont. ) •

26 FLBP operator (cont. ) Figure 8: An example of FLBP computation, T = 5

27 FLBP operator (cont. ) •

28 FLBP operator (cont. ) •

29 FLBP operator (cont. ) • • Basic LBP histograms may have bins of zero value. However, FLBP histograms have no zero-valued bins and thus are more informative and less sensitive to noise than the basic LBP.

Multi-kernels FLBP (KFLBP) • • Neighbor pixels p are first spread over multiple radii (kernels) K and then incorporated in an clockwise alternative manner to form the multi -kernel FLBP final binary code. Each kernel has separate operator with same or different neighbor points p. KFLBP scheme where K=2, pr 1=pr 2=4 30 KFLBP scheme where K=2, pr 1=pr 2=8

31 KFLBP (cont. ) •

32 KFLBP (cont. ) • KFLBP histogram size is still the same as that of the traditional FLBP.

33 Experiments and Results • • • Dataset Gait Classification Performance Measure Experimental Setup Discussion

34 Dataset • We evaluated our proposed approach on CASIA B gait database. • • • 124 subjects of 93 males and 31 females. 11 different views: from 0 to 180 with 18 degrees interval. Each subject was asked to walk 10 times through a straight line of concrete ground (6 normal walking, 2 wearing a coat, 2 carrying a bag).

35 Gait Classification • In this stage, Support Vector Machine (SVM) classifier with a linear kernel is used for gait recognition using the extracted feature vectors. • In our study, we built the model using Matlab Lib. SVM library which implements one-against-one for multiclassification.

36 Performance Measure •

37 Experimental Setup CASIA B • • One training (gallery) and three test (probe) sets. • And three probe sets: The gallery set of four sequences with normal walking of all subjects is used to train the SVM model. • • • Probe Set A: when subject normally walking. Probe Set B: when subject carrying a bag. Probe Set C: when subject wearing a coat.

38 Experimental Setup (cont. ) • And to evaluate the partitioning, the experiments were conducted over two scenarios: • • Scenario 1: without partitioning. Scenario 2: with partitioning.

Evaluation: KFLBP on CASIA B 39

40 Results and Discussion (cont. ) Table 1: Performance comparison of KFLBP using CASIA B under Normal-Walking covariate without partitioning.

41 Results and Discussion (cont. ) Table 2: Performance comparison of KFLBP using CASIA B under Carrying-Bag covariate without partitioning.

42 Results and Discussion (cont. ) Table 3: Performance comparison of KFLBP using CASIA B under Wearing-Coat covariate without partitioning.

43 Results and Discussion (cont. ) • The results demonstrate the following: • KFLBP improve the gait recognition performance over CASIA B dataset under most viewing angles and among the three covariates. • The performance was superior in the case of Probe Set A when subjects are normally walking.

Partitioning Evaluation on CASIA B 44

Results and Discussion (cont. ) Table 4: Comparison of recognition rates of KFLBP under Normal-Walking scenario with different non-overlapping partitioning 45

Results and Discussion (cont. ) Table 5: Comparison of recognition rates of KFLBP under Carrying-Bag scenario with different non-overlapping partitioning 46

Results and Discussion (cont. ) Table 6: Comparison of recognition rates of KFLBP under Wearing-Coat scenario with different non-overlapping partitioning 47

48 Results and Discussion (cont. ) • The results demonstrate the following: • • Mostly holistic image has lower performance than partitioning. No guarantee that with more partitions lead to enhanced results.

49 Conclusions • We adopt the Gait Energy Image (GEI) as the spatiotemporal gait representation to overcome the computation and storage burden. • We extended the fuzzy local binary patterns (FLBP) into multi-kernels FLBP (KFLBP) to improve its discriminatory power for gait recognition purposes. • The GEIs images were partitioned into a predefined nonoverlapping regions.

50 Conclusions (cont. ) • Experimental results using CASIA B benchmark gait dataset demonstrate the ouperformance of the proposed method. • Moreover, the results demonstrated that using partitioning most probably can enhance the performance to promising levels.

51 Future Work • More sophisticated preprocessing could be applied to improve the performance. • • More gait recognition approaches could be included. • Several classifiers could be utilized and compared Several gait databases could be involved in experiments. Experiments could include more gait covariates. Further investigation could be done for cross viewing and cross speed variations.

52 Thank you