Skip to main content
SpringerLink
Log in

Gait classification through CNN-based ensemble learning

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

Gait is a biological characteristic for video surveillance and many other applications, which can be used to identify individuals at a large distance. In this paper, a gait classification framework based on CNN Ensemble (GCF-CNN) is proposed, which includes three modules: 1) Feature extraction and preprocessing: use random sampling with replacement strategy to generate a serial of training sets from gait silhouette images; 2) Gait models training: construct and train primary CNN classifiers using different hyper-parameters, and train them a secondary classifier to combine them; 3) Gait classification: utilize the trained two-level classifier to achieve gait classification. In addition, the proposed classification framework is evaluated on the CASIA Gait Database and OU-ISIR Gait Database. And it is demonstrated by comprehensive experiments that the proposed classification framework can achieve outstanding performance in correct classification rate with respect to several state-of-the-art methods.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9

Similar content being viewed by others

References

  1. Ariyanto G, Nixon M (2011) Model-based 3D gait biometrics. In: International Conference on Biometrics. Washington, DC USA

  2. Aussem A, Elghazel H (2015) Unsupervised feature selection with ensemble learning. Mach Learn 98:157–180

    Article  MathSciNet  MATH  Google Scholar 

  3. Connor ARP (2018) Biometric recognition by gait: A survey of modalities and features. Comput Vis Image Underst 167(01):1–27

    Article  Google Scholar 

  4. Goodfellow I, Bengio Y, Courville A (2016) Deep Learning. MIT Press

  5. Han J, Bhanu B (2006) Individual recognition using gait energy image. IEEE Trans Pattern Anal Mach Intell 28(02):316–323

    Article  Google Scholar 

  6. Jia N, Sanchez V, Li C (2017) Learning optimized representations for view-invariant gait recognition. In: International Joint Conference on Biometrics, pp 774–780, Denver, USA

  7. Kusakunniran W, Wu Q, Li H, Zhang J (2009) Multiple views gait recognition using view transformation model based on optimized gait energy image. In: IEEE ICCV, pp 1058–1064, Kyoto, Japan

  8. Kusakunniran W, Wu Q, Zhang J, Li H, Wang L (2014) Recognizing gaits across views through correlated motion co-clustering. IEEE Trans Image Process 23(2):696–709

    Article  MathSciNet  MATH  Google Scholar 

  9. LeCun Y, Bengio Y, Hinton G (2015) Deep learning. Nature 521(5):436–445

    Article  Google Scholar 

  10. Li X (2018) Preconditioned stochastic gradient descent. IEEE Transactions on Neural Networks and Learning Systems 29(5):1454–1466

    Article  MathSciNet  Google Scholar 

  11. Li J, Ma S, Le T, Liu L, Liu J (2017) Causal decision trees. IEEE Trans Knowl Data Eng 29(2):257–271

    Article  Google Scholar 

  12. Luo J, Tang J, Tjahjadi T (2016) Robust arbitrary view gait recognition based on parametric 3D human body reconstruction and virtual posture synthesis. Pattern Recogn 60:361–377

    Article  Google Scholar 

  13. Makihara Y, Sagawa R, Mukaigawa Y, Echigo T, Yagi Y (2006) Gait recognition using a view transformation model in the frequency domain. In: IEEE ECCV, pp 151–163, Graz, Austria

  14. Muramatsu D, Shiraishi A, Makihara Y, Uddin MZ, Yagi Y (2015) Gait-based person recognition using arbitrary view transformation model. IEEE Trans Image Process 24(1):140–154

    Article  MathSciNet  MATH  Google Scholar 

  15. Sahu A, Runger G, Apley D (2011) Image denoising with a multi-phase kernel principal component approach and an ensemble version. In: 2011 IEEE Applied Imagery Pattern Recognition Workshop (AIPR), pages 1–7

  16. Sarkar S, Phillips P, Liu Z (2005) The humanid gait challenge problem: Data sets, performance, and analysis. IEEE Trans Pattern Anal Mach Intell 27 (02):162–177

    Article  Google Scholar 

  17. Schaar MVD Tekin C, Yoon J (2015) Adaptive ensemble learning with confidence bounds. IEEE Trans Signal Process 99:1–10

    Google Scholar 

  18. Shiraga K, Makihara Y, Muramatsu D (2016) GEINet: View-invariant gait recognition using a convolutional neural network. In: International Conference on Biometrics. Halmstad, Sweden

  19. Takemura N, Makihara Y, Muramatsu D, Echigo T, Yagi Y (2018) On input/output architectures for convolutional neural network-based cross-view gait recognition. IEEE Transactions on Circuits and Systems for Video Technology, 28(1)

  20. Tang J, Luo J, Tjahjadi T (2017) Robust arbitrary-view gait recognition based on 3D partial similarity matching. IEEE Trans Image Process 26 (1):7–23

    Article  MathSciNet  MATH  Google Scholar 

  21. Tao D, Li X, Wu X, Maybank S (2007) General tensor discriminant analysis and Gabor features for gait recognition. EEE Trans Pattern Anal Mach Intell 29(10):1700–1715

    Article  Google Scholar 

  22. Tong M, Li M, He B, Ma L, Zhao M (2020) DKD–DAD: a novel framework with discriminative kinematic descriptor and deep attention-pooled descriptor for action recognition. Neural Computing and Applications volume 32:5285–5302

    Article  Google Scholar 

  23. Tong M, Zhao M, Chen Y, Houyi W (2019) D3-LND: A two-stream framework with discriminant deep descriptor, linear cmdt and nonlinear kcmdt descriptors for action recognition. Neurocomputing 325:90–100

    Article  Google Scholar 

  24. Uddin MZ, Ngo TT, Makihara Y, Takemura N, Li X, Muramatsu D, Yagi Y (2018) The ou-isir large population gait database with real-life carried object and its performance evaluation. IPSJ Trans on Computer Vis Appl 10 (1):1–8

    Article  Google Scholar 

  25. Wang X, Feng S, Yan WQ (2019) Human gait recognition based on self-adaptive hidden markov model. IEEE Transactions on Computational Biology and Bioinformatics 1(1):1–10

    Article  Google Scholar 

  26. Wang X, Wang J, Yan K (2018) Gait recognition based on gabor wavelets and (2D)2PCA. Multimedia Tools and Applications 77(10):12545–12561

    Article  Google Scholar 

  27. Wang X, Yan K (2016) Human gait recognition using continuous density hidden Markov models. Pattern Recognit Artif Intell 29(8):709–717

    Google Scholar 

  28. Wang X, Yan WQ (2020) Cross-view gait recognition through ensemble learning. Neural Comput and Applic 32:7275–7287

    Article  Google Scholar 

  29. Wang X, Yan WQ (2020) Human gait recognition based on frame-by-frame gait energy images and convolutional long short term memory. International Journal of Neural Systems, 30(1)

  30. Wolf T, Babaee M, Rigoll G (2016) Multi-view gait recognition using 3D convolutional neural networks. In: IEEE International Conference on Image Processing, pp 4165–4169, Phoenix, USA

  31. Wu Z, Huang Y, Wang L, Wang X, Tan T (2017) A comprehensive study on cross-view gait based human identification with deep cnns. IEEE Trans Pattern Anal Mach Intell 39(02):209–226

    Article  Google Scholar 

  32. Yu S, Tan D, Tan T (2006) A framework for evaluating the effect of view angle, clothing and carrying condition on gait recognition. In: International Conference on Pattern Recognition, pp 441–444, Hong Kong, China

  33. Zhang R, Xu Z, Huang G, Wang D (2012) Global convergence of online BP training with dynamic learning rate. IEEE Transactions on Neural Networks and Learning Systems 23(2):330–341

    Article  Google Scholar 

  34. Zhao G, Liu G, Li H, Pietikainen M (2006) 3D gait recognition using multiple cameras. In: International Conference on Automatic Face and Gesture Recognition. Southampton, UK

Download references

Acknowledgements

This work is supported by the National Natural Science Foundation of China (No. 61303146 and 61602431) and the Natural Science Foundation of Zhejiang Province (No.Y20F020113).

Author information

Authors and Affiliations

  1. Key Laboratory of Electromagnetic Wave Information Technology and Metrology of Zhejiang Province, College of Information Engineering, China Jiliang University, No. 258, Xueyuan Street, Hangzhou, 310018, China

    Xiuhui Wang & Ke Yan

Authors
  1. Xiuhui Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Ke Yan
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Xiuhui Wang.

Ethics declarations

Conflict of interests

We declare that we have not financial and personal relationships with other people or organizations that can inappropriately influence our work, there is no professional or other personal interest of any nature or kind in any product, service and/or company that could be construed as influencing the position presented in, or the review of, the manuscript entitled “Gait Classification Through CNN-based Ensemble Learning”.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Wang, X., Yan, K. Gait classification through CNN-based ensemble learning. Multimed Tools Appl 80, 1565–1581 (2021). https://doi.org/10.1007/s11042-020-09777-7

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-020-09777-7

Keywords

Search

Navigation