View-independent person identification from human gait

doi:10.1016/j.neucom.2005.06.002

Neurocomputing

Volume 69, Issues 1–3, December 2005, Pages 250-256

https://doi.org/10.1016/j.neucom.2005.06.002 Get rights and content

Abstract

Based on a three-dimensional (3D) linear model and the Bayesian rule, a method is explored to identify human walkers from two-dimensional (2D) motion sequences taken from different viewpoints. Principal component analysis constructs the 3D linear model from a set of Fourier represented examples. The sets of coefficients derived from projecting 2D motion sequences onto the 3D model by means of a maximum a posterior estimate is used as a signature of a walker. Simulating an identification experiment on a set of walking data we show that these signatures show invariance across viewpoints and can be used for viewpoint-independent person identification.

Introduction

Human gait contains biometric signatures that can be used for person identification [2]. Most practical application fields cannot rely on constant viewing conditions but require viewpoint independent approaches. A number of researchers have addressed the problem. For instance, Shakhnarovich et al. [4] developed a view-normalization recognition algorithm by synthesizing virtual sequences rendered from canonical viewpoints. Their recognition is based on image sequences from multiple cameras, instead of one image sequence. Grauman et al. [3] reconstructed a visual hull from a contour-based representation of human gait. Then, they used the visual hull to infer the three-dimensional (3D) structure of the human body in terms of its major joints. Even though 3D structure in terms of 19 joint locations could be reconstructed from pedestrian sequences, it had to be inferred from the viewpoint of four known training positions, not from an unknown viewpoint. Furthermore, it is not clear how well the algorithm would perform in a person identification task as this was not a subject of the study.

Here, we explore a view-independent gait identification method, which is motivated by a linear model of human gait [5], [6]. Human gait is a cyclic motion and can be approximated by a Fourier expansion. Applying principal components analysis (PCA) to the Fourier representation we obtain a low-dimensional basis for our linear model. A gait signature of a two-dimensional (2D) motion sequence is defined as its projective coefficients, calculated using a Bayesian approach to obtain a maximum a posterior (MAP) estimate. The performance of identification of 2D projections from different viewpoints is quantitatively evaluated by means of a cross-validation procedure.

Section snippets

Linear model

Using an optical motion capture system (Vicon, Oxford Metrics, 120 Hz), we acquire human walking data as a time series of postures $s_{n} (t) : t = 1, 2, \dots, T_{n}, n = 1, 2, \dots, N$ specified by the locations of 15 discrete markers representing the main joints of the human body [5]. N is the number of walkers and T_n is the number of postures for walker n. Because each joint has three coordinates in 3D space, a posture s is a 45-dimensional vector. Using a second-order Fourier expansion, we can represent a time series s(

Gait signature

Analogous to the 3D Fourier representation, a 2D motion sequence $\hat{s} (t) : t = 1, 2, \dots, T$ has the following 2D Fourier representation: $\hat{w} = ({\hat{p}}_{0}, {\hat{p}}_{1}, {\hat{q}}_{1}, {\hat{p}}_{2}, {\hat{q}}_{2}) .$

The average posture ${\hat{p}}_{0}$ , and the characteristic postures ${\hat{p}}_{1}$ , ${\hat{q}}_{1}$ , ${\hat{p}}_{2}$ , and ${\hat{q}}_{2}$ are 30-dimensional vectors containing 2D coordinates of the 15 markers, so the 2D Fourier representation $\hat{w}$ is a 150-dimensional vector.

Approximating a 3D Fourier representation of a walker by a linear combination of J eigenwalkers $e_{1}, \dots, e_{J},$ the corresponding

Experiment

Using walking data acquired with an optical marker-based motion capture system (see [5] for details) we conducted cross-validated simulations on a data set of 80 walkers using six different learning views and a large number of testing views. All 2D views were orthographic projections of the 3D walkers. Because human walking is a symmetric motion and the elevation angle is not very large in practical applications, we chose all views within a range of 0° (frontal) to 90° for azimuth angle α and

Conclusions

This paper proposes a view-independent gait identification method, based on a linear model and a Bayesian approach. The model is constructed by submitting a set of Fourier representations of human walking to a principal component analysis. The calculation of gait signatures uses prior information to reduce ambiguity in a Bayesian framework. We evaluate this method quantitatively using walking data from different viewpoints. Results show generally very high performance. Furthermore, they suggest

References (6)

D. Cunado et al.
Automatic extraction and description of human gait models for recognition purposes
Comput. Vis. Image Understand.
(2003)
V. Bruce et al.
The basis of the 3/4 view advantage in face recognition
Appl. Cognitive Psychol.
(1987)
K. Grauman, G. Shakhnarovich, T. Darrell, A Bayesian approach to image-based visual hull reconstruction, in:...

There are more references available in the full text version of this article.

Cited by (61)

Vision-based approaches towards person identification using gait
2021, Computer Science Review
Visual surveillance has exponentially increased the growth of security devices and systems in the digital era. Gait-based person identification is an emerging biometric modality for automatic visual surveillance and monitoring as the walking patterns highly correlate to the subject’s identity. The scientific research on person identification using gait has grown dramatically over the past two decades due to its several benefits. It does not require active collaboration from users and can be performed without their cooperation. It is difficult to be impersonated and identification can be validated from low-resolution videos and with simple instrumentation. This paper presents a comprehensive overview of the exiting techniques, their key stages, and recent developments in vision-based person identification using gait. We reviewed the historical research on gait locomotion and explain that how it is used to recognize the identity. The article summarizes the different types of features that have been proposed to encode the biomechanics of gait and also groups them into different categories and subcategories based upon the similarity in their implementation. We also present the impact of different covariate factors that affect the performance of gait recognition systems and also discuss the recent works to cope with these challenges. Furthermore, a comparison of the recognition accuracies reported by the existing algorithms to assess their performance under verification and identification mode is also presented. A detailed summary of publicly available vision-based gait databases is also provided. Finally, it offers insight into the challenges and open problems for future perspectives in the field of gait recognition that can help to set the directions for future research in this field.
Robust gait recognition using hybrid descriptors based on Skeleton Gait Energy Image
2021, Pattern Recognition Letters
Citation Excerpt :
It is rational to combine these two features together and produce a hybrid gait representation. Approaches addressing the problem of view changes for gait recognition can be divided into three categories: (1) reconstruct 3D gait features [1,36]; (2) extract view-invariant gait features [7,12,35]; (3) establish relationships between viewing angles [5,13,24]. However, a fully-controlled and well-cooperative multi-camera environment is required for the first category, and it takes efforts to explore gait features invariant to viewing changes for the second category.
Gait features have been widely applied in human identification. The commonly-used representations for gait recognition can be roughly classified into two categories: model-free features and model-based features. However, due to the view variances and clothes changes, model-free features are sensitive to the appearance changes. For model-based features, there is great difficulty in extracting the underlying models from gait sequences. Based on the confidence maps and the part affinity fields produced by a two-branch multi-stage CNN network, a new model-based representation, Skeleton Gait Energy Image (SGEI), has been proposed in this paper. Another contribution is that a hybrid representation has been produced, which uses SGEI to remedy the deficiency of model-free features, Gait Energy Image (GEI) for instance. The experimental performances indicate that our proposed methods are more robust to the cloth changes, and contribute to increasing the robustness of gait recognition in the unconstrained environments with view variances and clothes changes.
A non-linear view transformations model for cross-view gait recognition
2020, Neurocomputing
Citation Excerpt :
Bodor et al. [31] proposed a 3D visual hull model to construct the view-invariant gait features using the input images from multiple cameras. In [32], a 3-D linear model is proposed to construct a view-independent gait feature using Bayesian rules. Zhao et al. [33] constructed a 3D human model for gait recognition using the video sequences captured from multiple viewpoints.
Gait has emerged as an important biometric feature which is capable of identifying individuals at distance without requiring any interaction with the system. Various factors such as clothing, shoes, and walking surface can affect the performance of gait recognition. However, cross-view gait recognition is particularly challenging as the appearance of individual’s walk drastically changes with the change in the viewpoint. In this paper, we present a novel view-invariant gait representation for cross-view gait recognition using the spatiotemporal motion characteristics of human walk. The proposed technique trains a deep fully connected neural network to transform the gait descriptors from multiple viewpoints to a single canonical view. It learns a single model for all the videos captured from different viewpoints and finds a shared high-level virtual path to project them on a single canonical view. The proposed deep neural network is learned only once using the spatiotemporal gait representation and applied to testing gait sequences to construct their view-invariant gait descriptors which are used for cross-view gait recognition. The experimental evaluation is carried out on two large benchmark cross-view gait datasets, CASIA-B and OU-ISIR large population, and the results are compared with current state-of-the-art methods. The results show that the proposed algorithm outperforms the state-of-the-art methods in cross-view gait recognition.
Human gait recognition based on deterministic learning and knowledge fusion through multiple walking views
2020, Journal of the Franklin Institute
Citation Excerpt :
The lower limbs motion trajectories were then extracted from the 3D model for further matching and recognition. In [16], human walkers were identified from two-dimensional motion sequences in multiple viewpoints based on a three-dimensional linear model and the Bayesian rule. In [17], a 3D walking model was constructed by using multiple cameras and image-based rendering technique was employed based on the 3D model for automatically constructing the proper view.
Deformation of gait silhouettes caused by different view angles heavily affects the performance of gait recognition. In this paper, a new method based on deterministic learning and knowledge fusion is proposed to eliminate the effect of view angle for efficient view-invariant gait recognition. First, the binarized walking silhouettes are characterized with three kinds of time-varying width parameters. The nonlinear dynamics underlying different individuals’ width parameters is effectively approximated by radial basis function (RBF) neural networks through deterministic learning algorithm. The extracted gait dynamics captures the spatio-temporal characteristics of human walking, represents the dynamics of gait motion, and is shown to be insensitive to the variance across various view angles. The learned knowledge of gait dynamics is stored in constant RBF networks and used as the gait pattern. Second, in order to handle the problem of view change no matter the variation is small or large, the learned knowledge of gait dynamics from different views is fused by constructing a deep convolutional and recurrent neural network (CRNN) model for later human identification task. This knowledge fusion strategy can take advantage of the encoded local characteristics extracted from the CNN and the long-term dependencies captured by the RNN. Experimental results show that promising recognition accuracy can be achieved.
Predicting body movements for person identification under different walking conditions
2018, Forensic Science International
Citation Excerpt :
Human walking motion or gait analyses have long been investigated and have produced various promising applications such as human identification, human recognition, and human motion synthesis. Specifically, walking motions can be analyzed for forensics purposes to provide similarity measurements [9], similar to finger prints, in order to complement other identifying information. Identifying a person based on gait for use in surveillance applications was inspired from the fact that motions can be collected without contacting the subjects via non-invasive characteristics [4].
Human motion during walking provides biometric information which can be utilized to quantify the similarity between two persons or identify a person. The purpose of this study was to develop a method for identifying a person using their walking motion when another walking motion under different conditions is given. This type of situation occurs frequently in forensic gait science. Twenty-eight subjects were asked to walk in a gait laboratory, and the positions of their joints were tracked using a three-dimensional motion capture system. The subjects repeated their walking motion both without a weight and with a tote bag weighing a total of 5% of their body weight in their right hand. The positions of 17 anatomical landmarks during two cycles of a gait trial were generated to form a gait vector. We developed two different linear transformation methods to determine the functional relationship between the normal gait vectors and the tote-bag gait vectors from the collected gait data, one using linear transformations and the other using partial least squares regression. These methods were validated by predicting the tote-bag gait vector given a normal gait vector of a person, accomplished by calculating the Euclidean distance between the predicted vector to the measured tote-bag gait vector of the same person. The mean values of the prediction scores for the two methods were 96.4 and 95.0, respectively. This study demonstrated the potential for identifying a person based on their walking motion, even under different walking conditions.
Multi-view gait recognition using a doubly-kernel approach on the Grassmann manifold
2016, Neurocomputing
Citation Excerpt :
Virtual image for an arbitrary view was synthesized from the 3D gait model to produce a gait feature at that view. Apart from that, Zhang et al [17]. applied principal component analysis to construct a 3D linear model from a set of Fourier representations.
View variation is one of the greatest challenges faced by the gait recognition research community. Recently, there are studies that model sets of gait features from multiple views as linear subspaces, which are known to form a special manifold called the Grassmann manifold. Conjecturing that modeling via linear subspace representation is not completely sufficient for gait recognition across view change, we take a step forward to consider non-linear subspace representation. A collection of multi-view gait features encapsulated in the form of a linear subspace is projected to the non-linear subspace through the expansion coefficients induced by kernel principal component analysis. Since subspace representation is inherently non-Euclidean, naïve vectorization as input to the vector-based pattern analysis machines is expected to yield suboptimal accuracy performance. We deal with this difficulty by embedding the manifold in a Reproducing Kernel Hilbert Space (RKHS) through a positive definite kernel function defined on the Grassmann manifold. A closer examination reveals that the proposed approach can actually be interpreted as a doubly-kernel method. To be specific, the first kernel maps the linear subspace representation non-linearly to a feature space; while the second kernel permits the application of kernelization-enabled machines established for vector-based data on the manifold-valued multi-view gait features. Experiments on the CASIA gait database shows that the proposed doubly-kernel method is effective against view change in gait recognition.

View all citing articles on Scopus

Zonghua Zhang received his B.S., M.S. and Ph.D. degrees from Tianjin University in 1992, 1998 and 2000, respectively. Then, he was a post-doctorate at the Mechanical Engineering Department of Tianjin University from December 2000 to October 2002. During April 2002 and September 2002, he was a research assistant at the Department of Computing of Hong Kong Polytechnic University. Supported by the Volkswagen Foundation, he was a post doctoral researcher in the BioMotionLab at Ruhr University in Bochum, Germany and later at Queen's University, Ontario, Canada. In 2005 he came to Heriot-Watt University where he is a research associate now. His research experiences include 3D imaging and modeling, human motion analysis, digital signal and image processing, and computer graphics.

Nikolaus Troje received his Ph.D. from the University of Freiburg, Germany, in 1994. Between 1994 and 1997 he worked as a researcher at the Max Planck Institute for Biological Cybernetics in Tübingen where he studied human face recognition and established the Max Planck Face Database. After 2 years at Queen's University in Kingston, Ontario, he recieved the prestigious Volkswagen Young Researcher's Award and set up the BioMotionLab at Ruhr University in Bochum, Germany. In 2003 he moved back to Queen's University where he is now a professor of psychology and Computer Science and a Canada Research Chair in Vision and Behavioural Sciences.

View full text

LettersView-independent person identification from human gait

Abstract

Introduction

Section snippets

Linear model

Gait signature

Experiment

Conclusions

Comput. Vis. Image Understand.

The basis of the 3/4 view advantage in face recognition

Appl. Cognitive Psychol.

Letters
View-independent person identification from human gait