Abstract
As a promising biometric identification method, gait recognition has many advantages, such as suitable for human identification at a long distance, requiring no contact and hard to imitate. However, due to the complex external factors in the gait data sampling process and the clothing changes of the person to be identified, gait recognition still faces numerous challenges in practical applications. In this paper, we present a novel solution for gait feature extraction and gait classification. Firstly, two kinds of Two-branch Convolution Neural Network (TCNN), i.e., middle-fusion TCNN and last-fusion TCNN, to improve the correct recognition rate of gait recognition are presented. Secondly, we construct Multi-Frequency Gait Energy Images (MF-GEIs) to train the proposed TCNNs networks and then extract refined gait features using the trained TCNNs. Finally, the output of each TCNN is utilized to train an SVM gait classifier separately which will be used for gait classification and recognition. In addition, the proposed solution is measured on CASIA dataset B and OU-ISIR LP dataset. Both experimental results show that our solution outperforms various existing methods.
Similar content being viewed by others
References
Abdulsattar F, Carter J (2016) Performance analysis of gait recognition with large perspective distortion. In: IEEE International conference on identity security and behavior analysis, Sendai
Ariyanto G, Nixon M (2011) Model-based 3D gait biometrics. In: International conference on biometrics. Washington, DC
Boulgouris N, Huang X (2013) Gait recognition using hmms and dual discriminative observations for sub-dynamics analysis. IEEE Trans Image Process 22(09):3636–3647
Connie T, Goh M, Teoh A (2017) A Grassmannian approach to address view change problem in gait recognition. IEEE Trans Cybern 47(06):1395–1408
DeLong ER, DeLong DM, Clarke-Pearson DL (1988) Comparing the areas under two or more correlated receiver operating characteristic curves: a nonparametric approach. Biometrics 44(3):837–845
Gaur S, Shah VA, Thakker M (2012) Biometric recognition techniques: a review. Int J Adv Res Electric Electron Instrum Eng 1(4):282–191
Gu Y, Chu M, Wu S et al (2017) Rich features and precise localization with region proposal network for object detection. In: Biometric recognition. Springer International Publishing, Cham, pp 605–614
Guan Y, Li C, Roli F (2015) On reducing the effect of covariate factors in gait recognition: a classifier ensemble method. IEEE Trans Pattern Anal Mach Intell 37(07):1521–1529
Han J, Bhanu B (2006) Individual recognition using gait energy image. IEEE Trans Pattern Anal Mach Intell 28(02):316–323
Hinton GE, Srivastava N, Krizhevsky A, Sutskever I, Salakhutdinov R (2012) Improving neural networks by preventing co-adaptation of feature detectors. arXiv:https://arxiv.org/abs/1207.0580
Huang X, Boulgouris N (2012) Gait recognition with shifted energy image and structural feature extraction. IEEE Trans Image Process 21(04):2256–2268
Iwama H, Okumura M, Makihara Y, Yagi Y (2012) The OU-ISIR gait database comprising the large population dataset and performance evaluation of gait recognition. IEEE Trans Inform Forens Secur 7(5):1511–1521
Jia N, Sanchez V, Li C (2017) Learning optimized representations for view-invariant gait recognition. In: International joint conference on biometrics, Denver, pp 774–780
Kusakunniran W, Wu Q, Li H, Zhang J (2009) Multiple views gait recognition using view transformation model based on optimized gait energy image. In: IEEE ICCV, Kyoto, pp 1058–1064
Kusakunniran W, Wu Q, Zhang J, Li H, Wang L (2014) Recognizing gaits across views through correlated motion co-clustering. IEEE Trans Image Process 23(2):696–709
LeCun Y, Bengio Y, Hinton G (2015) Deep learning. Nature 521(5):436–445
Li X (2018) Preconditioned stochastic gradient descent. IEEE Trans Neural Netw Learn Syst 29(5):1454–1466
Luo J, Tang J, Tjahjadi T (2016) Robust arbitrary view gait recognition based on parametric 3D human body reconstruction and virtual posture synthesis. Pattern Recogn 60:361–377
Mahfouf Z, FMerouani H, Bouchrika I, Harrati N (2018) Investigating the use of motion-based features from optical flow for gait recognition. Neurocomputing 283:140–149
Makihara Y, Sagawa R, Mukaigawa Y, Echigo T, Yagi Y (2006) Gait recognition using a view transformation model in the frequency domain. In: IEEE ECCV, Graz, pp 151–163
Makihara Y, Mannami H, Tsuji A, Hossain MA, Sugiura K, Mori A, Yagi Y (2012) The ou-isir gait database comprising the treadmill dataset. IPSJ Trans Comput Vis Appl 4:53–62
Marcolin F, Vezzetti E (2017) Novel descriptors for geometrical 3d face analysis. Multimed Tools Appl 76(12):13805–13834
Matthews BW (1975) Comparison of the predicted and observed secondary structure of t4 phage lysozyme. Biochimica et Biophysica Acta (BBA) - Protein Struct 405 (2):442–451
Muramatsu D, Shiraishi A, Makihara Y, Uddin MZ, Yagi Y (2015) Gait-based person recognition using arbitrary view transformation model. IEEE Trans Image Process 24(1):140–154
Sarkar S, Phillips P, Liu Z (2005) The humanid gait challenge problem: data sets, performance, and analysis. IEEE Trans Pattern Anal Mach Intell 27(02):162–177
Shiraga K, Makihara Y, Muramatsu D (2016) GEINet: view-invariant gait recognition using a convolutional neural network. In: International conference on biometrics. Halmstad
Takemura N, Makihara Y, Muramatsu D, Echigo T, Yagi Y (2018) On input/output architectures for convolutional neural network-based cross-view gait recognition. IEEE Trans Circ Syst Video Technol 28:1
Theekhanont P, Kurutach W, Miguet S (2012) Gait recognition using gei and pattern trace transform. In: 2012 International symposium on information technologies in medicine and education. Hokodate, pp 936–940
Uddin MZ, Ngo TT, Makihara Y, Takemura N, Li X, Muramatsu D, Yagi Y (2018) The ou-isir large population gait database with real-life carried object and its performance evaluation. IPSJ Trans Comput Vis Appl 10(1):1–8
Vezzetti E, Marcolin F, Stola V (2013) 3d human face soft tissues landmarking method: an advanced approach. Comput Industr 64:1326–1354
Wan M, Li M, Yang G, Gai S, Jin Z (2014) Feature extraction using two-dimensional maximum embedding difference. Inform Sci 274:55–69
Wan M, Lai Z, Yang G, Yang Z, Zhang F, Zheng H (2017) Local graph embedding based on maximum margin criterion via fuzzy set. Fuzzy Set Syst 318:120–131
Wan M, Yang G, Gai S, Yang Z (2017) Two-dimensional discriminant locality preserving projections (2ddlpp) and its application to feature extraction via fuzzy set. Multimed Tools Appl 76(1):355– 371
Wang X, Yan K (2016) Human gait recognition using continuous density hidden Markov models. Pattern Recogn Artif Intell 29(8):709–717
Wang X, Yan WQ (2019) Cross-view gait recognition through ensemble learning. Neural Comput Applic 1(1):1–13
Wang X, Wang J, Yan K (2018) Gait recognition based on gabor wavelets and (2D)2PCA. Multimed Tools Appl 77(10):12545–12561
Wolf T, Babaee M, Rigoll G (2016) Multi-view gait recognition using 3D convolutional neural networks. In: IEEE international conference on image processing, Phoenix, pp 4165–4169
Wu S, Xu Y (2019) Dsn: a new deformable subnetwork for object detection. IEEE Trans Circ Syst Video Technol 1(1):1–1
Wu Z, Huang Y, Wang L, Wang X, Tan T (2017) A comprehensive study on cross-view gait based human identification with deep cnns. IEEE Trans Pattern Anal Mach Intell 39(02):209–226
Yao Y, Rosasco L, Caponnetto A (2007) On early stopping in gradient descent learning. Constr Approx 26(2):289–315
Yu S, Tan D, Tan T (2006) A framework for evaluating the effect of view angle, clothing and carrying condition on gait recognition. In: International conference on pattern recognition, Hong Kong, pp 441–444
Yu S, Chen H, Wang Q, Shen L, Huang Y (2017) Invariant feature extraction for gait recognition using only one uniform model. Neurocomputing 239:81–93
Zhang R, Xu Z, Huang G, Wang D (2012) Global convergence of online BP training with dynamic learning rate. IEEE Trans Neural Netw Learn Syst 23(2):330–341
Acknowledgements
This work is supported by the National Natural Science Foundation of China under grants No. 61303146 and No. 61602431.
Author information
Authors and Affiliations
Corresponding author
Additional information
Publisher’s note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
Wang, X., Zhang, J. Gait feature extraction and gait classification using two-branch CNN. Multimed Tools Appl 79, 2917–2930 (2020). https://doi.org/10.1007/s11042-019-08509-w
Received:
Revised:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11042-019-08509-w