Skip to main content
Log in

Gait feature extraction and gait classification using two-branch CNN

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

As a promising biometric identification method, gait recognition has many advantages, such as suitable for human identification at a long distance, requiring no contact and hard to imitate. However, due to the complex external factors in the gait data sampling process and the clothing changes of the person to be identified, gait recognition still faces numerous challenges in practical applications. In this paper, we present a novel solution for gait feature extraction and gait classification. Firstly, two kinds of Two-branch Convolution Neural Network (TCNN), i.e., middle-fusion TCNN and last-fusion TCNN, to improve the correct recognition rate of gait recognition are presented. Secondly, we construct Multi-Frequency Gait Energy Images (MF-GEIs) to train the proposed TCNNs networks and then extract refined gait features using the trained TCNNs. Finally, the output of each TCNN is utilized to train an SVM gait classifier separately which will be used for gait classification and recognition. In addition, the proposed solution is measured on CASIA dataset B and OU-ISIR LP dataset. Both experimental results show that our solution outperforms various existing methods.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6

Similar content being viewed by others

References

  1. Abdulsattar F, Carter J (2016) Performance analysis of gait recognition with large perspective distortion. In: IEEE International conference on identity security and behavior analysis, Sendai

  2. Ariyanto G, Nixon M (2011) Model-based 3D gait biometrics. In: International conference on biometrics. Washington, DC

  3. Boulgouris N, Huang X (2013) Gait recognition using hmms and dual discriminative observations for sub-dynamics analysis. IEEE Trans Image Process 22(09):3636–3647

    Article  Google Scholar 

  4. Connie T, Goh M, Teoh A (2017) A Grassmannian approach to address view change problem in gait recognition. IEEE Trans Cybern 47(06):1395–1408

    Article  Google Scholar 

  5. DeLong ER, DeLong DM, Clarke-Pearson DL (1988) Comparing the areas under two or more correlated receiver operating characteristic curves: a nonparametric approach. Biometrics 44(3):837–845

    Article  Google Scholar 

  6. Gaur S, Shah VA, Thakker M (2012) Biometric recognition techniques: a review. Int J Adv Res Electric Electron Instrum Eng 1(4):282–191

    Google Scholar 

  7. Gu Y, Chu M, Wu S et al (2017) Rich features and precise localization with region proposal network for object detection. In: Biometric recognition. Springer International Publishing, Cham, pp 605–614

  8. Guan Y, Li C, Roli F (2015) On reducing the effect of covariate factors in gait recognition: a classifier ensemble method. IEEE Trans Pattern Anal Mach Intell 37(07):1521–1529

    Article  Google Scholar 

  9. Han J, Bhanu B (2006) Individual recognition using gait energy image. IEEE Trans Pattern Anal Mach Intell 28(02):316–323

    Article  Google Scholar 

  10. Hinton GE, Srivastava N, Krizhevsky A, Sutskever I, Salakhutdinov R (2012) Improving neural networks by preventing co-adaptation of feature detectors. arXiv:https://arxiv.org/abs/1207.0580

  11. Huang X, Boulgouris N (2012) Gait recognition with shifted energy image and structural feature extraction. IEEE Trans Image Process 21(04):2256–2268

    Article  MathSciNet  Google Scholar 

  12. Iwama H, Okumura M, Makihara Y, Yagi Y (2012) The OU-ISIR gait database comprising the large population dataset and performance evaluation of gait recognition. IEEE Trans Inform Forens Secur 7(5):1511–1521

    Article  Google Scholar 

  13. Jia N, Sanchez V, Li C (2017) Learning optimized representations for view-invariant gait recognition. In: International joint conference on biometrics, Denver, pp 774–780

  14. Kusakunniran W, Wu Q, Li H, Zhang J (2009) Multiple views gait recognition using view transformation model based on optimized gait energy image. In: IEEE ICCV, Kyoto, pp 1058–1064

  15. Kusakunniran W, Wu Q, Zhang J, Li H, Wang L (2014) Recognizing gaits across views through correlated motion co-clustering. IEEE Trans Image Process 23(2):696–709

    Article  MathSciNet  Google Scholar 

  16. LeCun Y, Bengio Y, Hinton G (2015) Deep learning. Nature 521(5):436–445

    Article  Google Scholar 

  17. Li X (2018) Preconditioned stochastic gradient descent. IEEE Trans Neural Netw Learn Syst 29(5):1454–1466

    Article  MathSciNet  Google Scholar 

  18. Luo J, Tang J, Tjahjadi T (2016) Robust arbitrary view gait recognition based on parametric 3D human body reconstruction and virtual posture synthesis. Pattern Recogn 60:361–377

    Article  Google Scholar 

  19. Mahfouf Z, FMerouani H, Bouchrika I, Harrati N (2018) Investigating the use of motion-based features from optical flow for gait recognition. Neurocomputing 283:140–149

    Article  Google Scholar 

  20. Makihara Y, Sagawa R, Mukaigawa Y, Echigo T, Yagi Y (2006) Gait recognition using a view transformation model in the frequency domain. In: IEEE ECCV, Graz, pp 151–163

    Chapter  Google Scholar 

  21. Makihara Y, Mannami H, Tsuji A, Hossain MA, Sugiura K, Mori A, Yagi Y (2012) The ou-isir gait database comprising the treadmill dataset. IPSJ Trans Comput Vis Appl 4:53–62

    Article  Google Scholar 

  22. Marcolin F, Vezzetti E (2017) Novel descriptors for geometrical 3d face analysis. Multimed Tools Appl 76(12):13805–13834

    Article  Google Scholar 

  23. Matthews BW (1975) Comparison of the predicted and observed secondary structure of t4 phage lysozyme. Biochimica et Biophysica Acta (BBA) - Protein Struct 405 (2):442–451

    Article  Google Scholar 

  24. Muramatsu D, Shiraishi A, Makihara Y, Uddin MZ, Yagi Y (2015) Gait-based person recognition using arbitrary view transformation model. IEEE Trans Image Process 24(1):140–154

    Article  MathSciNet  Google Scholar 

  25. Sarkar S, Phillips P, Liu Z (2005) The humanid gait challenge problem: data sets, performance, and analysis. IEEE Trans Pattern Anal Mach Intell 27(02):162–177

    Article  Google Scholar 

  26. Shiraga K, Makihara Y, Muramatsu D (2016) GEINet: view-invariant gait recognition using a convolutional neural network. In: International conference on biometrics. Halmstad

  27. Takemura N, Makihara Y, Muramatsu D, Echigo T, Yagi Y (2018) On input/output architectures for convolutional neural network-based cross-view gait recognition. IEEE Trans Circ Syst Video Technol 28:1

    Article  Google Scholar 

  28. Theekhanont P, Kurutach W, Miguet S (2012) Gait recognition using gei and pattern trace transform. In: 2012 International symposium on information technologies in medicine and education. Hokodate, pp 936–940

  29. Uddin MZ, Ngo TT, Makihara Y, Takemura N, Li X, Muramatsu D, Yagi Y (2018) The ou-isir large population gait database with real-life carried object and its performance evaluation. IPSJ Trans Comput Vis Appl 10(1):1–8

    Article  Google Scholar 

  30. Vezzetti E, Marcolin F, Stola V (2013) 3d human face soft tissues landmarking method: an advanced approach. Comput Industr 64:1326–1354

    Article  Google Scholar 

  31. Wan M, Li M, Yang G, Gai S, Jin Z (2014) Feature extraction using two-dimensional maximum embedding difference. Inform Sci 274:55–69

    Article  Google Scholar 

  32. Wan M, Lai Z, Yang G, Yang Z, Zhang F, Zheng H (2017) Local graph embedding based on maximum margin criterion via fuzzy set. Fuzzy Set Syst 318:120–131

    Article  MathSciNet  Google Scholar 

  33. Wan M, Yang G, Gai S, Yang Z (2017) Two-dimensional discriminant locality preserving projections (2ddlpp) and its application to feature extraction via fuzzy set. Multimed Tools Appl 76(1):355– 371

    Article  Google Scholar 

  34. Wang X, Yan K (2016) Human gait recognition using continuous density hidden Markov models. Pattern Recogn Artif Intell 29(8):709–717

    Google Scholar 

  35. Wang X, Yan WQ (2019) Cross-view gait recognition through ensemble learning. Neural Comput Applic 1(1):1–13

    MathSciNet  Google Scholar 

  36. Wang X, Wang J, Yan K (2018) Gait recognition based on gabor wavelets and (2D)2PCA. Multimed Tools Appl 77(10):12545–12561

    Article  Google Scholar 

  37. Wolf T, Babaee M, Rigoll G (2016) Multi-view gait recognition using 3D convolutional neural networks. In: IEEE international conference on image processing, Phoenix, pp 4165–4169

  38. Wu S, Xu Y (2019) Dsn: a new deformable subnetwork for object detection. IEEE Trans Circ Syst Video Technol 1(1):1–1

    Google Scholar 

  39. Wu Z, Huang Y, Wang L, Wang X, Tan T (2017) A comprehensive study on cross-view gait based human identification with deep cnns. IEEE Trans Pattern Anal Mach Intell 39(02):209–226

    Article  Google Scholar 

  40. Yao Y, Rosasco L, Caponnetto A (2007) On early stopping in gradient descent learning. Constr Approx 26(2):289–315

    Article  MathSciNet  Google Scholar 

  41. Yu S, Tan D, Tan T (2006) A framework for evaluating the effect of view angle, clothing and carrying condition on gait recognition. In: International conference on pattern recognition, Hong Kong, pp 441–444

  42. Yu S, Chen H, Wang Q, Shen L, Huang Y (2017) Invariant feature extraction for gait recognition using only one uniform model. Neurocomputing 239:81–93

    Article  Google Scholar 

  43. Zhang R, Xu Z, Huang G, Wang D (2012) Global convergence of online BP training with dynamic learning rate. IEEE Trans Neural Netw Learn Syst 23(2):330–341

    Article  Google Scholar 

Download references

Acknowledgements

This work is supported by the National Natural Science Foundation of China under grants No. 61303146 and No. 61602431.

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Xiuhui Wang.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Wang, X., Zhang, J. Gait feature extraction and gait classification using two-branch CNN. Multimed Tools Appl 79, 2917–2930 (2020). https://doi.org/10.1007/s11042-019-08509-w

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-019-08509-w

Keywords

Navigation