Recognition of Human Gaits Alessandro Bissacco Computer Science

Recognition of Human Gaits Alessandro Bissacco Computer Science UCLA Yi Ma Electrical and Computer Engineering University of Illinois at Urbana. Champaign Alessandro Chiuso Dipartimento di Elettronica e Informatica Universita di Padova Stefano Soatto Computer Science UCLA Presented by Peter Schwer

Summary § Problem Description § Related Work § Overview of Gait Recognition Process § Tracking § Learning the Model § Recognition § Results

The Human Gait § Observable from great distances § Discernable independent of skew (relative direction of subject’s gait) § Communicates: § Identity § Mood

Good Recognition Traits § Invariant to: § Photometric Factors: §Illumination, clothing § Geometric Factors: §Camera Position §Length of subject’s limbs §Relative direction of gait

Good Recognition Traits § Capable of discerning: § Different types of gaits: §Running §Walking §Skipping, etc. § Mood §Not addressed, but dynamical models can be extended to this area.

Related Work University of Southampton Carnegie Mellon University (CMU) Massachusetts Insitute of Technology (MIT) Georgia Tech (GATech) University of Maryland University of South Florida Analyzing gait with spatiotemporal surfaces (1994) Sourabh A. Niyogi, Edward H. Adelson

The Recognition Problem §The Sub-problems: §Tracking §Learning the model §Recognizing

Tracking § “The emphasis of our work is not tracking. ” § Images → Skeletons § Results: y(t), t = 1 … τ (Sequence of joint positions)

Learning the Model

The Right Model § Ideally, § Unique model for each dataset § Method for calculating similarity (distance) between models § Catch: § “The set of dynamical models in canonical form is not a linear space (even if the model itself is linear!)” § Distances between dynamical models are not trivially computed. § Data, as usual, is inherently noisy.

Creating the Dynamical Model § Input: y(t) – Sequence of Joint Positions § What are some of the statistical properties of this sequence of joint positions? § Stochastic Stationary Second-order process § How is noise distributed in the data? § We use a Gauss-Markov Model as a representative of each class.

The ARMA Model x(t+1) = Ax(t) + v(t), v(t) ~ N(0, Q); x(0) = x 0 y(t) = C(x(t) + w(t); w(t) ~ N(0, R)

The ARMA Model § Other Applications: § Speech Recognition § Tracking § Geophysics: Forecasting and Simulation § Pattern Recognition § Dynamic Texture Recognition

The ARMA model x(t) is a process describing the “state” at time t (vector) y(t) is the measured joint positions (vector) Q, R are input and output noise covariances (matrices) x(t+1) = Ax(t) + v(t), v(t) ~ N(0, Q); x(0) = x 0 y(t) = C(x(t) + w(t); w(t) ~ N(0, R) (A: [n × n], B: [m × n]) PROBLEM: Non-uniqueness of A, C, Q, R, S

Innovation/Canonical Representation § Identify stochastic subspace models without forming the covariance matrix. § Principal Angles and Directions § Source: P. Van Overschee and B. De Moor Subspace Algorithms for the stochastic identification problem. Automatica, 29: 649660, 1993.

A Canonical Representation x(t) is a process describing the “state” (vector) y(t) is the measured joint positions (vector) Q, R are input and output noise covariances (matrices) x(t+1) = Ax(t) + v(t), v(t) ~ N(0, Q); x(0) = x 0 y(t) = C(x(t) + w(t); w(t) ~ N(0, R) From Overschee and De Moor we have {A, C, Q, R} A and C describe the gait.

Recognizing

Our Dynamic Model § A, C, Q, R provided § What are they? § How might they be useful?

How to Describe Gait § Tracking produces a “waveform” that describes a series of joint positions/angles: What is the best way to compare these to waveforms? Gait 1 Gait 2

Distance Between Models § Subspace/principal angles § Start with: O(M)- infinite observability matrix

Form O(m) § O(m) is a subspace § We want to compare the observability subspaces of two models M 1 and M 2. § Form O(m) for each model: § Call the results H 1 and H 2, respectively.

Form Orthonormal Matrices § Gram-Schmidt Orthogonalization on H 1 and H 2 § Described: http: //lagrange. la. asu. edu/Virtual. Class/Algebra/Gram Schm. html § Applet: http: //www. mste. uiuc. edu/exner/ncsa/orthogonal/#sim ulation § Call the result orthonormal matrices QH 1 and QH 2

Calculate Principal Angles § Compute n-ordered Singular Values of QTH 1 QH 2 § n-ordered singular values can be denoted: cos 2(θ 1), … , cos 2(θn) § Principal Angles between the subspaces H 1 and H 2 are denoted by the n-tuple: § (θ 1 , θ 2 , …, θn ) sorted in descending order.

Calculate the Distance § First measure is not necessarily non-negative (less fit) § Second Measure is the largest principal angle (more fit) § The Martin and Finsler Distance, respectively.

Or § Just use their matlab code: function theta = subspace_angles(A 1, K 1, C 1, A 2, K 2, C 2) n = size(A 1, 1); m = size(C 1, 1); A = [ A 1 zeros(n, 3*n); zeros(n) A 2 -K 2*C 2 zeros(n, 2*n); zeros(n, 2*n) A 2 zeros(n); zeros(n, 3*n) A 1 -K 1*C 1]; C = [ C 1 -C 2 -C 1 ]; Q = dlyap(A’, C’*C); E = eig([zeros(2*n) pinv(Q(1: 2*n, 1: 2*n))*Q(1: 2*n, 2*n+1: 4*n); pinv(Q(2*n+1: 4*n, 2*n+1: 4*n))*Q(2*n+1: 4*n, 1: 2*n) zeros(2*n)]); E = max(-ones(size(E)), E); E = min(ones(size(E)), E); theta = acos(E(1: 2*n));

Results

Example Gaits

Name that Gait Pensive and determined.

Obligatory Gait Video 2 Not in a rush.

Obligatory Gait Video 3 Sad walking.

Obligatory Gait Video 4 There is no stopping me.

Results “The pairwise distance between each sequence in the dataset is displayed in this plot. Each row/column of a matrix represents a sequence, and sequences corresponding to similar gaits are grouped in block rows/columns. Dark indicates a small distance, light a large distance. The minimum distance is of course along the diagonal, and for each column the next closest sequence is indicated by a circle, while the second nearest is indicated by a cross. ”

Results § How accurate are the results? § How does this compare with human ability?

Questions?

From Discussion Board § Diem: How are the ARMA model and Kalman filters related? § Mei-fang: Is there any method to find the best size training set, and the best k for classifying the new sample? § Matt Clothier: What is the advantage of the Martin distance over the Finsler distance? § Neil and Mike: What do they mean by “Transient Actions” and why does the previous assumption not hold for them? :