Skip to main content
SpringerLink
Log in

Hybrid neural network model for reconstruction of occluded regions in multi-gait scenario

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

In real-time situations such as airports, railway stations, and shopping complexes, etc. people walk in a group, and such a group of walking persons termed as multi-gait (MG). In these situations, occlusion is a serious issue that affects gait recognition performance. This issue of occlusion of body regions affects the extraction of gait features for the correct recognition of an object. The objective of this article is to reconstruct occluded regions at the preprocessing stage, which can be used for human recognition in the MG scenario. The article is divided into two folds. Firstly, we segment five regions of interest such as ankle, knee, wrist, elbow, and shoulder. We propose a particle swarm optimization (PSO) based neural network (NN) called hybrid NN to solve this problem. The performance of the proposed model is validated on our constructed dataset (SMVDU-MG), considering two view directions i.e. lateral (left to right) and oblique (left to right diagonal). Experimental results show that the proposed model gives better performance compared to an artificial neural network and alternating least square (ALS) method based on mean square error (MSE) and mean absolute percentage error (MAPE) as a performance measure function.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13

Similar content being viewed by others

References

  1. Aristidou A, Cameron J, Lasenby J (2008) Real-time estimation of missing markers in human motion capture. 2nd International Conference on Bioinformatics and Biomedical Engineering pp 1343–1346. https://doi.org/10.1109/ICBBE.2008.665.

  2. Arora P, Hanmandlub M, Srivastava S (2015) Gait based authentication using gait information image features. Pattern Recogn Lett 68(2):336–342. https://doi.org/10.1016/j.patrec.2015.05.016

    Article  Google Scholar 

  3. Bashir K, Xiang T, Gong S (2010) Gait recognition without subject cooperation. Pattern Recognition Letters 31(13):2052–2060. https://doi.org/10.1016/j.patrec.2010.05.027

    Article  Google Scholar 

  4. Begg R, Kamruzzaman J (2006) Neural network for detection and classification of walking pattern changes due to ageing. Aust Phys Eng Med 29(2):188–195. https://doi.org/10.1007/BF03178892

    Article  Google Scholar 

  5. Chen J, Fang J, Liu W, Tang T, Yang C (2018) clMF: A fine-grained and portable alternating least squares algorithm for parallel matrix factorization, Future Generation Computer Systems

  6. Chen X, Weng J, Lu W, Xu J (2017) Multi-gait Recognition based on Attribute Discovery. IEEE Trans Pattern Anal Mach Intell PP(99):1. https://doi.org/10.1109/TPAMI.2017.2726061

    Article  Google Scholar 

  7. Chen X, Xu J, Weng J (2017) Multi-gait recognition using hypergraph partition. Mach Vis Appl 28(1–2):117–127. https://doi.org/10.1007/s00138-016-0810-6

    Article  Google Scholar 

  8. Chen X, Yang T, Xu J (2016) Multi-gait identification based on multilinear analysis and multi-target tracking. Multimed Tools Appl 75(11):6505–6532. https://doi.org/10.1007/s11042-015-2585-6

    Article  Google Scholar 

  9. Federolf PA (2013) A novel approach to solve the missing marker problem in marker-based motion analysis that exploits the segment coordination patterns in multi-limb motion data. PLoS One 8(10):1–13. https://doi.org/10.1371/journal.pone.0078689

    Article  Google Scholar 

  10. Gloersen O, Federolf P (2016) Predicting missing marker trajectories in human motion data using marker interconnections. Plos one 11(3):1–14. https://doi.org/10.1371/journal.pone.0152616

    Article  Google Scholar 

  11. Hofman M, Sural S, Rigoll G (2011) Gait recognition in the presence of occlusion: a new dataset and baseline algorithms. In Proceedings of the 19th International Conference in Central Europe on Computer Graphics, Visualization and Computer Vision pp 99–104

  12. Hu Q, Yang J, Win KT, Huang X (2019) An alternating least square based algorithm for predicting patient survivability. In: Islam R et al (eds) Data mining. AusDM 2018. Communications in Computer and Information Science, vol 996. Springer, Singapore https://doi.org/10.1007/978-981-13-6661-1_24

  13. Iwashita Y, Sakano H, Kurazume R (2015) Gait recognition robust to speed transition using mutual subspace method. Int Conf Image Anal Process (ICIAP) 2015:141–149. https://doi.org/10.1007/978-3-319-23231-713

    Article  MathSciNet  Google Scholar 

  14. Jia S, Wang L, Li X (2015) View-invariant gait authentication based on Silhouette contours analysis and view estimation. IEEE/CAA J Autom Sin 2(2):226–232. https://doi.org/10.1109/JAS.2015.7081662

    Article  MathSciNet  Google Scholar 

  15. Kale A, Sundaresan A, Rajagopalan AN (2004) Identification of humans using gait. IEEE Trans Image Process 13(9):1163–1173. https://doi.org/10.1109/TIP.2004.832865

    Article  Google Scholar 

  16. Kennedy J, Eberhart R (1995) Particle swarm optimization. Proc IEEE Int Conf Neural Netw 4:1942–1948

    Article  Google Scholar 

  17. Kovac J, Struc V, Peer P (2017) Frame–based classification for cross-speed gait recognition. Multimed Tools Appl 78:1–23. https://doi.org/10.1007/s11042-017-5469-0

    Article  Google Scholar 

  18. Liang W, Tan T, Hu W, Ning H (2003) Automatic gait recognition based on statistical shape analysis. IEEE Trans Image Process 12(9):1120–1131. https://doi.org/10.1109/TIP.2003.815251

    Article  MathSciNet  Google Scholar 

  19. Lishani AO, Boubchir L, Khalifa E, Bouridane A (2017) Human gait recognition based on Haralick features. Signal Image Video Process 11(6):1123–1130. https://doi.org/10.1007/s11760-017-1066-y

    Article  Google Scholar 

  20. Liu G, McMillan L (2006) Estimation of missing markers in human motion capture. Vis Comput 22(9–11):721–728

    Article  Google Scholar 

  21. Makihara Y, Mannami H, Tsuji A, Hossain MA, Sugiura K, Mori A, Yagi Y (2012) The OU-ISIR gait database comprising the treadmill dataset. IPSJ Trans Comput Vis Appl 4:53–62

    Article  Google Scholar 

  22. Man J, Bhanu B (2006) Individual recognition using gait energy image. IEEE Trans Pattern Anal Mach Intell 28(2):316–322. https://doi.org/10.1109/TPAMI.2006.38

    Article  Google Scholar 

  23. Masood H, Farooq H (2017) “A proposed framework for vision based gait biometric system against spoofing attacks”, international conference on communication. Comput Digit Syst (C-CODE):357–362. https://doi.org/10.1109/C-CODE.2017.7918957

  24. Nandy A, Chakraborty R, Chakraborty P (2016) Cloth invariant gait recognition using pooled segmented statistical features. Neurocomputing 191:117–140. https://doi.org/10.1016/j.neucom.2016.01.002

    Article  Google Scholar 

  25. Rajasekaran S, Pai GAV (2017) Introduction to artificial intelligence system. In Neural Networks, Fuzzy Systems and Evolutionary Algorithms: Synthesis and Applications, PHI India, Ed.2nd, ch.1, pp 1–7

  26. Rida I, Almaadeed S, Bouridane A (2016) Gait recognition based on modified phase-only correlation. SIViP 10(3):463–470. https://doi.org/10.1007/s11760-015-0766-4

    Article  Google Scholar 

  27. Riedmiller M, Braun H (1993) A direct adaptive method for faster backpropagation learning: The RPROP algorithm. Proceedings of the IEEE International Conference on Neural Networks pp 586–591

  28. Roy A, Sural S, Mukherjee J, Rigoll G (2011) Occlusion detection and gait silhouette reconstruction from degraded scenes. Signal Image Video Process 5:415–430. https://doi.org/10.1007/s11760-011-0245-5

    Article  Google Scholar 

  29. Sarkar S, Phillips PJ, Liu Z, Vega IR, Grother P, Bowyer KW (2005) The HumanID gait challenge problem: data sets, performance, and analysis. IEEE Trans Pattern Anal Mach Intell 27(2):162–177. https://doi.org/10.1109/TPAMI.2005.39

    Article  Google Scholar 

  30. Shi Y, Eberhart R (1998) Parameter selection in particle swarm optimization. In: The proceedings of the international conference on evolutionary programming pp 591–601

  31. Singh JP, Arora S, Jain S, Singh UP (2019) A multi-gait dataset for human recognition under occlusion scenario. 2019 International conference on issues and challenges in intelligent computing techniques (ICICT), GHAZIABAD, India, pp 1–6

  32. Singh JP, Jain S, Arora S, Singh UP A survey of behavioral biometric gait recognition: current success and future perspectives. Arch Comput Methods Eng. https://doi.org/10.1007/s11831-019-09375-3

  33. Singh JP, Jain S, Arora S, Singh UP Reconstruction of occluded ROI in multi-person gait based on numerical methods. Multimed Syst. https://doi.org/10.1007/s00530-019-00641-9

  34. Tafazzoli F, Safabakhsh R (2010) Model-based human gait recognition using leg and arm movements. Eng Appl Artif Intell 23(8):1237–1246

    Article  Google Scholar 

  35. Takemura N, Makihara Y, Muramatsu D, Echigo T, Yagi Y (2018) Multi-view large population gait dataset and its performance evaluation for cross-view gait recognition. IPSJ Trans Comput Vis Appl 10(4):1–14

    Google Scholar 

  36. Yoo J-H, Nixon MS (2011) Automated Markerless analysis of human gait motion for recognition and classification. ETRI J 33(2):259–266

    Article  Google Scholar 

  37. Yu S, Tan D, Tan T (2006) A framework for evaluating the effect of view angle, clothing and carrying condition on gait recognition. 18th International Conference on Pattern Recognition (ICPR), pp 441–444

  38. Zeng W, Wang C, Li Y (2014) Model-based human gait recognition via deterministic learning. Cogn Comput 6(2):218–229. https://doi.org/10.1007/s12559-013-9221-4

    Article  Google Scholar 

  39. Zhang JR, Zhang J, Lok TM, Lyu MR (2007) A hybrid particle swarm optimization–back-propagation algorithm for feedforward neural network training. Appl Math Comput 185(2):1026–1037. https://doi.org/10.1016/j.amc.2006.07.025

    Article  MATH  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. School of Computer Science & Engineering, Shri Mata Vaishno Devi University, Katra, 182320, India

    Jasvinder Pal Singh & Sakshi Arora

  2. School of Computer Science & Engineering, IIITDM, Jabalpur, 482005, India

    Sanjeev Jain

  3. School of Mathematics, Shri Mata Vaishno Devi University, Katra, 182320, India

    Uday Pratap Singh

Authors
  1. Jasvinder Pal Singh
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Sanjeev Jain
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Uday Pratap Singh
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Sakshi Arora
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Jasvinder Pal Singh.

Ethics declarations

Conflict of interest

The authors declare that they have no conflict of interest.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Singh, J.P., Jain, S., Singh, U.P. et al. Hybrid neural network model for reconstruction of occluded regions in multi-gait scenario. Multimed Tools Appl 81, 9607–9629 (2022). https://doi.org/10.1007/s11042-022-11964-7

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-022-11964-7

Keywords

Search

Navigation