Elsevier

Neurocomputing

Volume 152, 25 March 2015, Pages 139-150
Neurocomputing

Gait recognition across different walking speeds via deterministic learning

https://doi.org/10.1016/j.neucom.2014.10.079Get rights and content

Abstract

Deformation of gait silhouettes caused by objects under different walking speeds has a significant effect on the performance of gait recognition. In this paper, we present an algorithm via deterministic learning theory to eliminate the effect of walking speed for efficient gait recognition in the lateral view. Three kinds of silhouette features are selected. They capture the spatio-temporal characteristics of each individual׳s movement and represent the dynamics of gait motion. They also can sensitively reflect the variance between gait patterns under different walking speeds. The gait recognition approach consists of two phases: a training phase and a recognition phase. In the training phase, human gait dynamics underlying different individuals׳ gaits across different walking speeds are locally accurately approximated by radial basis function (RBF) neural networks. Obtained knowledge of approximated gait dynamics is stored in constant RBF networks. In order to handle the problem of speed change no matter the variation is small or significantly large, the training patters under different walking speeds constitute a uniform training dataset containing all kinds of gait dynamics of each individual under different walking speeds. In the recognition phase, a bank of dynamical estimators is constructed for all the training gait patterns. Prior knowledge of human gait dynamics represented by the constant RBF networks is embedded in the estimators. By comparing the set of estimators with a test gait pattern whose speed pattern included in the prior training dataset, a set of recognition errors are generated. The average L1 norms of the errors are taken as the similarity measure between the dynamics of the training gait patterns and the dynamics of the test gait pattern. Finally, comprehensive experiments are carried out on the most well-known public gait databases: the CMU, the OU-ISIR, and the CASIA gait database C to demonstrate the recognition performance of the proposed algorithm.

Introduction

Gait, as a promising unobtrusive biometric, has attracted many researchers in recent years. Many studies have demonstrated that gait has the potential to become a powerful biometric for surveillance, activity monitoring, clinical analysis, and access control. However, there are a number of covariate factors that affect gait recognition performance. Hence, in gait recognition, one important requirement is of robustness to variations including different lighting conditions, poses, carrying objects, view angles, shoe types, walking surfaces and walking speeds [1].

Gait is a dynamic shape model such that it varies in pose and size throughout a walking cycle [2]. In practice, the gait shape of an individual can be easily altered by many factors, particularly by the change of walking speed. Individuals can walk at various speeds in any real-world situations, and human motion changes nonlinearly according to its speed. This will lead to significant changes to walking patterns and generate difficulties for gait recognition. Understanding the above factors is crucial to develop robust and accurate gait recognition algorithms. In this paper, we will focus on proposing a new solution to deal with the effect of walking speed variation on gait recognition.

Considering the effect of walking speed on recognition by gait, previous studies can be roughly divided into two categories: (1) identifying speed-invariant gait features; and (2) transforming gait features under various walking speeds onto a common walking speed [3]. In one example of the first category, six simple projective features were proposed to describe human gait and applied principal component analysis to reduce the dimension of raw gait features [4]. It revealed that the projective normalization generally could improve the robustness of projective features against speed variation. However, the limitation of the method is that it cannot well deal with large change of walking speed. In [3], Procrustes shape analysis was adopted for gait signature description and relevant similarity measurement. It proposed a higher order shape configuration for gait shape description and constructed a differential composition model to differentiate the different effects caused by walking speed changes on various human body parts. This method was able to tolerate the varying walking speed. In [5], three-way data consisting of x-, y- and time-axes were used to effectively extract spatio-temporal local geometric features characterizing the motion. It was expected to be relatively robust against variations in walking speed since it only used the sums of local features over a gait sequence.

In one example of the second category, a method of gait silhouette transformation from one speed to another was proposed to cope with walking speed changes [6]. Static and dynamic features were first separated from gait silhouettes by fitting a human model. Then, a factorization based speed transformation model was created to transform the dynamic features from a reference speed to another arbitrary speed. In [7], a method of a combination of cubic higher order local autocorrelation and a statistical framework provided by hidden Markov models was proposed, which was robust against walking speed variations. In [8], cell phone based accelerometers and a cycle extraction approach were used to study the influence of different walking speeds and surfaces on gait performance. However, in most of these methods, researchers have assumed that walking speed does not change much within or across gait sequences. Their performance may degrade greatly when walking speed varies significantly.

Challenges to the two categories are: (1) all the walking speeds must be known in advance and they are not applicable to unknown walking speed; and (2) gait recognition is carried out under the same walking speed or under cross-speed, which means various training datasets are needed to be constructed corresponding to the testing patterns under different walking speeds. Obviously, this is not applicable. In real-world applications, if there exists a training dataset consisting of gait patterns under different walking speeds such that, when a test gait pattern whose speed pattern included in the prior training dataset appears, it can be recognized. Then, speed-invariant gait recognition can be achieved. However, to the best of the authors׳ knowledge, this cannot be seen from previous research works in which one training dataset only contains gait patterns under the same walking speed [3], [4], [5], [6], [7], [8]. In this paper, we focus on constructing such a uniform training dataset containing gait patterns under different walking speeds to tackle the problem concerning the walking speed variation no matter it is small or large.

In our previous studies, gait dynamics represented by suitable periodic features, such as lower limb joint angles and/or angular velocities, will be approximated by radial basis function (RBF) neural network (NN) via deterministic learning theory [9], [10]. The difference of gait dynamics between different individuals during walking can be used for gait recognition. Following this idea, we continue to search for periodic gait features that constitute gait dynamics and can reflect the variance between gait patterns under different walking speeds. Recent gait research revealed that silhouette cues, such as the width of the outer contour and the silhouette area, play a primary role in gait recognition [11], [12], [13], [14], [15], [16], [17], [18], [19], [20], [21]. This finding enlightens us to extract silhouette features for gait identification. Since walking speed variation can significantly change the gait shape description, suitable silhouette features representing gait dynamics under different walking speeds will be proposed in this paper for gait recognition. Compared with methods presented in [9], [10], the amount of discriminability provided by the dynamics of silhouette features is similar to the discriminability provided by the dynamics of physical parameters like joint angles and/or angular velocities. Moreover, silhouette features will be more easily and accurately extracted than joint angle/angular velocity features. This will guarantee the accuracy of the feature extraction and the performance of gait recognition.

In this paper, we propose a new silhouette-based method for speed-invariant gait recognition in the lateral view via deterministic learning theory. First, the aspect ratio of the boundingbox of the moving silhouette, the width of the binarized silhouette of the lower limbs, and the silhouette area model the periodic deformation of gait silhouettes. They are selected as gait features for recognition. They capture the spatio-temporal characteristics of each individual׳s movement and represent the dynamics of gait motion. They also can sensitively reflect the variance between gait patterns under different walking speeds. Second, human gait dynamics underlying different individuals׳ gaits across different walking speeds are locally accurately approximated by RBF networks. The obtained knowledge of approximated gait dynamics is stored in constant RBF networks. Third, in order to handle the problem of speed change no matter the variation is small or significantly large, the training patters under different walking speeds constitute a uniform training dataset containing all kinds of gait dynamics of each individual under different walking speeds. A bank of dynamical estimators is constructed for all the training gait patterns. Prior knowledge of gait dynamics represented by the constant RBF networks obtained from the training phase is embedded in the estimators. By comparing the set of estimators with a test pattern whose speed pattern included in the prior training dataset, a set of recognition errors are generated. The average L1 norms of the errors are taken as the similarity measure between the dynamics of the training gait patterns and the dynamics of the test gait pattern. The test gait pattern similar to one of the training gait patterns can be recognized according to the smallest error principle.

The rest of the paper is organized as follows. Section 2 introduces preliminary knowledge about deterministic learning theory and problem formulation. Silhouette features extraction and gait signature derivation are presented in Section 3. Training and learning mechanism is given in Section 4 to obtain gait dynamics of individuals across different walking speeds. In Section 5, gait recognition scheme is presented to achieve speed-invariant recognition. The experiments of gait recognition are included in Section 6 to verify the effectiveness of our approach. Section 7 contains the conclusions.

Section snippets

Deterministic learning theory

In deterministic learning theory, identification of system dynamics of general nonlinear systems is achieved according to the following elements: (i) employment of localized RBF networks; (ii) satisfaction of a partial persistence of excitation (PE) condition; (iii) exponential stability of the adaptive system along the periodic or recurrent orbit; (iv) locally accurate NN approximation of the unknown system dynamics [22].

The RBF networks can be described by fnn(Z)=i=1Nwisi(Z)=WTS(Z), where ZΩ

Gait feature extraction and gait signature derivation

In this section, we investigate three kinds of periodic silhouette features. They seem suitable for capturing discriminatory gait information from sequences of extracted silhouettes and for representing gait dynamics under different walking speeds. An important issue in gait recognition is the extraction of salient features that will effectively capture gait characteristics. The features must reflect the walking speed variation and should yield good discriminability across individuals. For each

Training and learning mechanism based on silhouette features

In this section, based on deterministic learning theory, we present a scheme for identification of gait system dynamics under different walking speeds.

In order to more accurately describe human walking, gait dynamics can be modeled as the following form:ẋ=F(x;p)+v(x;p)where x=[x1,,xn]TRn are the states of system (5) which represent the combined silhouette features of the human body under different walking speeds, p is a constant vector of system parameters. F(x;p)=[f1(x;p),,fn(x;p)]T is a

Gait recognition mechanism

In this section, we present a scheme for rapid recognition of human gait using the learned gait system dynamics under different walking speeds.

Consider a training dataset containing dynamical human gait patterns φζk under different walking speeds, k=1,…,M, with the kth gait training pattern φζk generated fromẋ=Fk(x;pk)+vk(x;pk),x(t0)=xζ0where Fk(x;pk) denotes the gait system dynamics, vk(x;pk) denotes the modeling uncertainty, pk is the system parameter vector.

As shown in Section 4, the

Experiments

In our experiments, three widely adopted gait databases are used to evaluate the performance of the proposed method, which include: (1) CASIA gait database C [30]; (2) CMU MoBo gait database [31]; and (3) OU-ISIR gait database [6]. These databases directly support the study of gait recognition with respect to the variation of walking speed. Moreover, from the research perspective, there are different advantages from the first three databases: (1) the CASIA gait database C contains a large

Conclusions

A new speed-invariant gait recognition approach based on silhouette features via deterministic learning theory is presented in this paper. Based on the method for feature extraction, silhouette features representing human gait dynamics and reflecting the walking speed variation on gait shapes can be extracted. The gait system dynamics of each individual across different walking speeds can be accurately approximated by RBF networks and the obtained knowledge will be stored in constant RBF

Acknowledgments

The authors would like to thank the anonymous reviewers for the constructive comments. This work was supported by the National Natural Science Foundation of China (Grant no. 61304084), by the National Science Fund for Distinguished Young Scholars (Grant no. 61225014), by China Postdoctoral Science Foundation (Grant no. 2013M531851), by Natural Science Foundation of Guangdong Province (Grant no. S2013040016859), by Natural Science Foundation of Fujian Province (Grant no. 2014J05078), by the

Wei Zeng received the M.E. degree from the Department of Automation, Xiamen University, Xiamen, China, in 2008, and the Ph.D. degree from the College of Automation Science and Engineering, South China University of Technology, Guangzhou, China, in 2012. Currently, he is a postdoctoral fellow at South China University of Technology. His current research interests include dynamical pattern recognition, adaptive NN control/identification, and deterministic learning theory.

References (31)

  • A. Kobayashi et al.

    Three-way auto-correlation approach to motion recognition

    Pattern Recognit. Lett.

    (2009)
  • A. Tsuji, Y. Makihara, Y. Yagi, Silhouette transformation based on walking speed for gait identification, in:...
  • M.R. Aqmar, K. Shinoda, S. Furui, Robust gait recognition against speed variation, in: Proceedings of the 20th...
  • M. Muaaz, C. Nickel, Influence of different walking speeds and surfaces on accelerometer-based biometric gait...
  • W. Zeng et al.

    Model-based human gait recognition via deterministic learning

    Cogn. Comput.

    (2014)
  • Cited by (21)

    • Multi-view gait recognition using a doubly-kernel approach on the Grassmann manifold

      2016, Neurocomputing
      Citation Excerpt :

      Gait recognition has appeared as an attractive alternative to biometric solution due to its non-invasive nature and the possibility of recognition at a distance [1–4].

    View all citing articles on Scopus

    Wei Zeng received the M.E. degree from the Department of Automation, Xiamen University, Xiamen, China, in 2008, and the Ph.D. degree from the College of Automation Science and Engineering, South China University of Technology, Guangzhou, China, in 2012. Currently, he is a postdoctoral fellow at South China University of Technology. His current research interests include dynamical pattern recognition, adaptive NN control/identification, and deterministic learning theory.

    Cong Wang received the B.E. and M.E. degrees from Beijing University of Aeronautics and Astronautics, Beijing, China, in 1989 and 1997, respectively, and the Ph.D. degree from the National University of Singapore, Singapore, in 2002. Currently, he is a professor at the College of Automation Science and Engineering, South China University of Technology, Guangzhou, China. He has authored and co-authored over 60 papers in international journals and conferences, and is a co-author of the book Deterministic Learning Theory for Identification, Recognition and Control (CRC Press, Boca Raton, FL, 2009). His current research interests include adaptive NN control/identification, deterministic learning theory, dynamical pattern recognition, pattern-based intelligent control, oscillation fault diagnosis, and cognitive and brain sciences.

    View full text