Skip to main content
Springer Nature Link
Log in

Prosperous Human Gait Recognition: an end-to-end system based on pre-trained CNN features selection

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

Human Gait Recognition (HGR) is a biometric approach, widely used for security purposes from the past few decades. In HGR, the change in an individual walk along with wearing clothes and carrying bag are major covariant controls which impact the performance of a system. Moreover, recognition under various view angles is another key challenge in HGR. In this work, a novel fully automated method is proposed for HGR under various view angles using deep learning. Four primary steps are involved such as: preprocessing of original video frames, exploiting pre-trained Densenet-201 CNN model for features extraction, reduction of additional features from extracted vector based on a hybrid selection method, and finally recognition using supervised learning methods. The extraction of CNN features is a key step in which our target is to extract the most active features. To achieve this goal, we fuse the features of both second last and third last layers in a parallel process. At a later stage, best features are selected by the Firefly algorithm and Skewness based approach. These selected features are serially combined and fed to One against All Multi Support Vector Machine (OAMSVM) for final recognition. Three different angles 180, 360 and 540 of the CASIA B dataset are selected for the evaluation process and accuracy of 94.3%, 93.8% and 94.7% is achieved respectively. Results show significant improvement in accuracy and recall rate as compared to the existing state-of-the-art techniques.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11

Similar content being viewed by others

References

  1. Alghamdi A, Hammad M, Ugail H, Abdel-Raheem A, Muhammad K, Khalifa HS et al. (2019) Detection of Myocardial Infarction Based on Novel Deep Transfer Learning Methods for Urban Healthcare in Smart Cities, arXiv, p. arXiv: 1906.09358

  2. Arora P, Hanmandlu M, Srivastava S (2015) Gait based authentication using gait information image features. Pattern Recogn Lett 68:336–342

    Article  Google Scholar 

  3. Arshad H, Khan MA, Sharif M, Yasmin M, Javed MY (2019) Multi-level features fusion and selection for human gait recognition: an optimized framework of Bayesian model and binomial distribution. Int J Mach Learn Cybern 10:3601–3618

    Article  Google Scholar 

  4. Arshad H, Khan MA, Sharif MI, Yasmin M, Tavares JMR, Zhang YD, et al., A multilevel paradigm for deep convolutional neural network features selection with an application to human gait recognition, Expert Systems, p. e12541.

  5. Bansode SS, Hiremath RB, Kolgiri S, Deshmukh RA, Biomimetics and Its Applications-A Review

  6. Batool FE, Attique M, Sharif M, Javed K, Nazir M, Abbasi AA, Iqbal Z, Riaz N (2020) Offline signature verification system: a novel technique of fusion of GLCM and geometric features using SVM. Multimed Tools Appl 1–20

  7. Ben X, Zhang P, Lai Z, Yan R, Zhai X, Meng W (2019) A general tensor representation framework for cross-view gait recognition. Pattern Recogn 90:87–98

    Article  Google Scholar 

  8. Ben X, Gong C, Zhang P, Yan R, Wu Q, Meng W (2019) Coupled bilinear discriminant projection for cross-view gait recognition, IEEE Transactions on Circuits and Systems for Video Technology

  9. Castro FM, Marín-Jiménez MJ, Guil N, De La Blanca NP (2017) Automatic learning of gait signatures for people identification, in International Work-Conference on Artificial Neural Networks, 257–270

  10. Choudhury SD, Tjahjadi T (2015) Robust view-invariant multiscale gait recognition. Pattern Recogn 48:798–811

    Article  Google Scholar 

  11. Deng M, Yang H, Cao J, Feng X (2019) View-Invariant Gait Recognition Based on Deterministic Learning and Knowledge Fusion, in 2019 International Joint Conference on Neural Networks (IJCNN), 1–8

  12. El-Rahiem BA, Ahmed MAO, Reyad O, El-Rahaman HA, Amin M, El-Samie FA (2019) An efficient deep convolutional neural network for visual image classification," in International Conference on Advanced Machine Learning Technologies and Applications, 23–31.

  13. Gad R, El-Latif AAA, Elseuofi S, Ibrahim HM, Elmezain M, Said W (2019) IoT Security Based on Iris Verification Using Multi-Algorithm Feature Level Fusion Scheme, in 2019 2nd International Conference on Computer Applications & Information Security (ICCAIS), 1–6

  14. Goffredo M, Carter JN, Nixon MS (2008) Front-view gait recognition, in 2008 IEEE Second International Conference on Biometrics: Theory, Applications and Systems, 1–6

  15. Gu J, Wang Z, Kuen J, Ma L, Shahroudy A, Shuai B, Liu T, Wang X, Wang G, Cai J, Chen T (2018) Recent advances in convolutional neural networks. Pattern Recogn 77:354–377

    Article  Google Scholar 

  16. He K, Zhang X, Ren S, Sun J (2016) Identity mappings in deep residual networks, in European conference on computer vision, 630–645

  17. Huang C-C, Hsu C-C, Liao H-Y, Yang S-H, Wang L-L, Chen S-Y (2016) Frontal gait recognition based on spatio-temporal interest points. J Chin Inst Eng 39:997–1002

    Article  Google Scholar 

  18. Huang G, Liu Z, Van Der Maaten L, Weinberger KQ (2017) Densely connected convolutional networks, in Proceedings of the IEEE conference on computer vision and pattern recognition, 4700–4708

  19. Hussain N, Khan MA, Sharif M, Khan SA, Albesher AA, Saba T, Armaghan A (2020) A deep neural network and classical features based scheme for objects recognition: an application for machine inspection. Multimed Tools Appl. https://doi.org/10.1007/s11042-020-08852-3

  20. Khan MH, Li F, Farid MS, Grzegorzek M (2017) Gait recognition using motion trajectory analysis, in International Conference on Computer Recognition Systems, 73–82

  21. Khan MA, Sharif M, Javed MY, Akram T, Yasmin M, Saba T (2017) License number plate recognition system using entropy-based features selection approach with SVM. IET Image Process 12:200–209

    Article  Google Scholar 

  22. Khan MH, Farid MS, Grzegorzek M (2019) "spatiotemporal features of human motion for gait recognition," signal. Image and Video Processing 13:369–377

    Article  Google Scholar 

  23. Khan MA, Akram T, Sharif M, Muhammad N, Javed MY, Naqvi SR (2019) Improved strategy for human action recognition; experiencing a cascaded design, IET Image Process

  24. Khan MA, Javed K, Khan SA, Saba T, Habib U, Khan JA, Abbasi AA (2020) Human action recognition using fusion of multiview and deep features: an application to video surveillance. Multimed Tools Appl 1–27

  25. Khan MA, Sharif M, Akram T, Raza M, Saba T, Rehman A (2020) Hand-crafted and deep convolutional neural network features fusion and selection strategy: an application to intelligent human action recognition. Applied Soft Computing 87:105986

  26. Khan MA, Rubab S, Kashif A, Sharif MI, Muhammad N, Shah JH, Zhang Y-D, Satapathy SC (2020) Lungs cancer classification from CT images: an integrated design of contrast based classical features fusion and selection. Pattern Recognit Lett 129:77–85

  27. Kovač J, Štruc V, Peer P (2019) Frame–based classification for cross-speed gait recognition. Multimed Tools Appl 78:5621–5643

    Article  Google Scholar 

  28. Kumar V, Minz S (2014) Feature selection: a literature review. SmartCR 4:211–229

    Article  Google Scholar 

  29. Kusakunniran W, Wu Q, Zhang J, Li H (2012) Gait recognition under various viewing angles based on correlated motion regression. IEEE transactions on circuits and systems for video technology 22:966–980

    Article  Google Scholar 

  30. Kusakunniran W, Wu Q, Zhang J, Li H, Wang L (2013) Recognizing gaits across views through correlated motion co-clustering. IEEE Trans Image Process 23:696–709

    Article  MathSciNet  Google Scholar 

  31. Li C, Min X, Sun S, Lin W, Tang Z (2017) DeepGait: a learning deep convolutional representation for view-invariant gait recognition using joint Bayesian. Appl Sci 7:210

    Article  Google Scholar 

  32. Li X, Makihara Y, Xu C, Yagi Y, Ren M (2019) Joint intensity transformer network for gait recognition robust against clothing and carrying status. IEEE Transactions on Information Forensics and Security 14:3102–3115

    Article  Google Scholar 

  33. Lishani AO, Boubchir L, Khalifa E, Bouridane A (2019) Human gait recognition using GEI-based local multi-scale feature descriptors. Multimed Tools Appl 78:5715–5730

    Article  Google Scholar 

  34. Liu Y, Zheng YF (2005) One-against-all multi-class SVM classification using reliability measures, in Proceedings. 2005 IEEE International Joint Conference on Neural Networks, 2005., 849–854

  35. Liu G, Zhong S, Li T (2019) Gait recognition method of temporal–spatial HOG features in critical separation of Fourier correction points. Futur Gener Comput Syst 94:11–15

    Article  Google Scholar 

  36. Lynnerup N, Vedel J (2005) Person identification by gait analysis and photogrammetry. Journal of Forensic science 50:JFS2004054–JFS2004057

    Article  Google Scholar 

  37. Mogan JN, Lee CP, Lim KM, Tan AW (2017) Gait recognition using binarized statistical image features and histograms of oriented gradients, in 2017 International Conference on Robotics, Automation and Sciences (ICORAS), 1–6

  38. Mortazavi M (2019) An improved human skin detection and localization by using machine learning techniques in RGB and YCbCr color spaces, PeerJ Preprints 2167-9843

  39. Rawat W, Wang Z (2017) Deep convolutional neural networks for image classification: a comprehensive review. Neural Comput 29:2352–2449

    Article  MathSciNet  Google Scholar 

  40. Rehman A, Khan MA, Mehmood Z, Saba T, Sardaraz M, Rashid M (2020) Microscopic melanoma detection and classification: a framework of pixel‐based fusion and multilevel features reduction. Microscopy Research and Technique 83(4):410–423

  41. Saba T, Khan MA, Rehman A, Marie-Sainte SL (2019) Region extraction and classification of skin cancer: a heterogeneous framework of deep CNN features fusion and reduction. J Med Syst 43(9):289

  42. Saeys Y, Inza I, Larrañaga P (2007) A review of feature selection techniques in bioinformatics. Bioinformatics 23:2507–2517

  43. Sharif M, Attique M, Tahir MZ, Yasmim M, Saba T, Tanik UJ (2020) A Machine Learning Method with Threshold Based Parallel Feature Fusion and Feature Selection for Automated Gait Recognition. Journal of Organizational and End User Computing (JOEUC) 32:67–92

    Article  Google Scholar 

  44. Sharif MI, Li JP, Khan MA, Saleem MA (2020) Active deep neural network features selection for segmentation and recognition of brain tumors using MRI images. Pattern Recognit Lett 129:181–189

  45. Shin H-C, Roth HR, Gao M, Lu L, Xu Z, Nogues I, Yao J, Mollura D, Summers RM (2016) Deep convolutional neural networks for computer-aided detection: CNN architectures, dataset characteristics and transfer learning. IEEE Trans Med Imaging 35:1285–1298

    Article  Google Scholar 

  46. Shirke S, Pawar S, Shah K (2014) Literature review: Model free human gait recognition, in 2014 Fourth International Conference on Communication Systems and Network Technologies, 891–895

  47. Song C, Huang Y, Huang Y, Jia N, Wang L (2019) GaitNet: an end-to-end network for gait based human identification. Pattern Recogn 96:106988

    Article  Google Scholar 

  48. Sugandhi K, Raju G (2019) An Efficient HOG-Centroid Descriptor for Human Gait Recognition," in 2019 Amity International Conference on Artificial Intelligence (AICAI), 355–360

  49. Szegedy C, Vanhoucke V, Ioffe S, Shlens J, Wojna Z (2016) Rethinking the inception architecture for computer vision, in Proceedings of the IEEE conference on computer vision and pattern recognition, 2818–2826

  50. Szegedy C, Ioffe S, Vanhoucke V, Alemi AA (2017) Inception-v4, inception-resnet and the impact of residual connections on learning, in thirty-first AAAI conference on artificial intelligence

  51. Tafazzoli F, Safabakhsh R (2010) Model-based human gait recognition using leg and arm movements. Eng Appl Artif Intell 23:1237–1246

    Article  Google Scholar 

  52. Tian Y, Wei L, Lu S, Huang T (2019) Free-view gait recognition. PLoS One 14:e0214389

    Article  Google Scholar 

  53. Tong S-b, Fu Y-z, Ling H-f (2019) Cross-view gait recognition based on a restrictive triplet network. Pattern Recogn Lett 125:212–219

    Article  Google Scholar 

  54. Vedaldi A, Lenc K (2015) Matconvnet: Convolutional neural networks for matlab, in Proceedings of the 23rd ACM international conference on Multimedia, 689–692

  55. Wang X, Yan WQ (2019) Human gait recognition based on frame-by-frame gait energy images and convolutional long short-term memory, Int J Neural Syst, pp. 1950027–1950027

  56. Wang N, Li Q, El-Latif AAA, Peng J, Niu X (2014) An enhanced thermal face recognition method based on multiscale complex fusion for Gabor coefficients. Multimed Tools Appl 72:2339–2358

    Article  Google Scholar 

  57. Wang X, Feng S, Yan WQ (2019) Human gait recognition based on self-adaptive hidden Markov model, IEEE/ACM transactions on computational biology and bioinformatics, Human Gait Recognition Based on Self-adaptive Hidden Markov Model.

  58. Wang F, Yan L, Xiao J (2019) Human gait recognition system based on support vector machine algorithm and using wearable sensors. Sensors and Materials 31:1335–1349

    Article  Google Scholar 

  59. Wang M, Yong S, He C, Chen H, Zhang S, Peng C et al. (2019) Research on Abnormal Gait Recognition Algorithms for Stroke Patients Based on Array Pressure Sensing System, in 2019 IEEE 3rd information technology, networking, Electronic and Automation Control Conference (ITNEC), 1560–1563

  60. Yang X-S (2010) Firefly algorithm, stochastic test functions and design optimisation, arXiv preprint arXiv:1003.1409

  61. Yao L, Kusakunniran W, Wu Q, Zhang J, Tang Z, Yang W (2019) Robust gait recognition using hybrid descriptors based on skeleton gait energy image. Pattern Recogn Lett

  62. Zheng S, Zhang J, Huang K, He R, Tan T (2011) "robust view transformation model for gait recognition," in 2011 18th IEEE international conference on image processing, 2073–2076

Download references

Author information

Authors and Affiliations

  1. Department of Computer Science, COMSATS University Islamabad, Wah Campus, Islamabad, Pakistan

    Asif Mehmood & Muhammad Sharif

  2. Department of Computer Science and Engineering, HITEC University, Museum Road Taxila, Rawalpindi, Pakistan

    Muhammad Attique Khan & Imran Ashraf

  3. Department of Software Engineering, Foundation University Islamabad, Islamabad, Pakistan

    Sajid Ali Khan & Muhammad Shaheen

  4. Department of Computer and Information Sciences, Prince Sultan University, Riyadh, Saudi Arabia

    Tanzila Saba

  5. SEECS, National University of Science and Technology, Islamabad, Pakistan

    Naveed Riaz

Authors
  1. Asif Mehmood
    View author publications

    You can also search for this author inPubMed Google Scholar

  2. Muhammad Attique Khan
    View author publications

    You can also search for this author inPubMed Google Scholar

  3. Muhammad Sharif
    View author publications

    You can also search for this author inPubMed Google Scholar

  4. Sajid Ali Khan
    View author publications

    You can also search for this author inPubMed Google Scholar

  5. Muhammad Shaheen
    View author publications

    You can also search for this author inPubMed Google Scholar

  6. Tanzila Saba
    View author publications

    You can also search for this author inPubMed Google Scholar

  7. Naveed Riaz
    View author publications

    You can also search for this author inPubMed Google Scholar

  8. Imran Ashraf
    View author publications

    You can also search for this author inPubMed Google Scholar

Corresponding author

Correspondence to Muhammad Attique Khan.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Mehmood, A., Khan, M.A., Sharif, M. et al. Prosperous Human Gait Recognition: an end-to-end system based on pre-trained CNN features selection. Multimed Tools Appl 83, 14979–14999 (2024). https://doi.org/10.1007/s11042-020-08928-0

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-020-08928-0

Keywords