Skip to main content
SpringerLink
Log in

Deep mutual learning network for gait recognition

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

Human identification plays a significant role in ensuring social security. However, face-based and appearance-based retrieval methods are not effective in monitoring due to the long distance and low camera resolution. Compared with other biological characteristics, the gait of humans has a strong discriminating ability even at long distance and low resolution. In this paper, the deep mutual learning strategy is applied to gait recognition, and by training collaboratively with other networks, the generalization ability of the network is improved simply and effectively. We use a set of independent frames of gait as input to two convolutional neural networks. This method is unaffected by frame alignment and can naturally integrate video frames of different walking conditions (e.g. different viewing angles, different clothing/carrying conditions). At the same time, the set can extract gait features from incomplete gait cycles due to occlusion. A mutual learning strategy can improve the running speed appropriately and realize the compactness and accuracy of the model. Two convolutional networks learn simultaneously and solve problems together. To evaluate the method’s performance, we compare it to several methods on the CASIA and OU-ISIR gait databases, and construct different sets of gaits with incomplete periods to compare the accuracy of identification with them and the complete gait set. Experimental results show that the method is effective.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 5
Fig. 6

Similar content being viewed by others

References

  1. Ben X, Gong C, Zhang P, Yan R, Wu Q, Meng W (2019) Coupled bilinear discriminant projection for cross-view gait recognition. In: IEEE Transactions on Circuits and Systems for Video Technology, pp 1–1

    Google Scholar 

  2. Buciluˇa C, Caruana R, Niculescu-Mizil A (2006) Model compression. In: Proceedings of the 12th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, KDD ‘06. ACM, New York, NY, pp 535–541

    Chapter  Google Scholar 

  3. Chao H, He Y, Zhang J, Feng J (2019) GaitSet: Regarding gait as a set for cross-view gait recognition. In: AAAI, vol 33, pp 8126–8133

    Google Scholar 

  4. Connie T, Goh MKO, Teoh ABJ (2018) “Human gait recognition using localized grassmann mean representatives with partial least squares regression,” Multimedia Tools and Applications, Human gait recognition using localized Grassmann mean representatives with partial least squares regression.

  5. Dahl JV, Koch KC, Kleinhans E, Ostwald E, Schulz G, Buell U, Hanrath P (2011) Convolutional networks and applications in vision. In: IEEE international symposium on circuits and systems

  6. Fu Y, Wei Y, Zhou Y, Shi H, Huang G, Wang X, Yao Z, Huang TS (2018) Horizontal pyramid matching for person re-identification. CoRR, vol. abs/1804.05275

  7. Goffredo M, Bouchrika I, Carter JN, Nixon MS (2009) Self- calibrating view-invariant gait biometrics. IEEE Trans Syst Man Cybern B Cybern 40(4):997–1008

    Article  Google Scholar 

  8. Hinton G, Vinyals O, Dean J (2015) Distilling the knowledge in a neural network. Comput. Sci. 14(7):38–39

    Google Scholar 

  9. M. Hu, Y. Wang, Z. Zhang, D. Zhang JJ. Little 2013, “Incremental learn-ing for video-based gait recognition with lbp flow,” IEEE Transactions on Systems Man and Cybernetics Part B Cybernetics A Publication of the IEEE Systems Man and Cybernetics Society, vol. 43, no. 1, pp. 77–89

  10. Imed B, Michaela G, John C, Mark N (2011) On using gait in forensic biometrics. J Forensic Sci 56(4):882–889

    Article  Google Scholar 

  11. Iwama H, Okumura M, Makihara Y, Yagi Y (2012) The ou-isir gait database comprising the large population dataset and performance evaluation of gait recognition. IEEE Trans. Inf. Forensics Secur. 7:1511–1521

    Article  Google Scholar 

  12. Iwama H, Okumura M, Makihara Y, Yagi Y (2012) The ou-isir gait database comprising the large population dataset and performance eval- uation of gait recognition. IEEE Trans. Inf. Forensics Secur. 7(5):1511–1521

    Article  Google Scholar 

  13. Kaiming H, Xiangyu Z, Shaoqing R, Jian S (2014) Spatial pyramid pooling in deep convolutional networks for visual recognition. IEEE Trans. Pattern Anal. Mach. Intell. 37(9):1904–1916

    Google Scholar 

  14. Kusakunniran W, Wu Q, Li H, Zhang J (2010) Multiple views gait recognition using view transformation model based on optimized gait energy image

  15. Kusakunniran W, Wu Q, Zhang J, Li H (2010) Support vector re- gression for multi-view gait recognition based on local motion feature selection. In: Computer vision and pattern recognition

    Google Scholar 

  16. Kusakunniran W, Wu Q, Zhang J, Ma Y, Li H (2013) A new view- invariant feature for cross-view gait recognition. IEEE Trans. Inf. Forensics Secur. 8(10):1642–1653

    Article  Google Scholar 

  17. Lei JB, Caurana R (2013) Do deep nets really need to be deep? Adv Neural Inf Proces Syst:2654–2662

  18. Liang W, Tan T, Member S, Ning H, Hu W (2003) Silhouette analysisbased gait recognition. IEEE Trans. Pattern Anal. Mach. Intell. 25(12):1505–1518

    Article  Google Scholar 

  19. Makihara Y, Sagawa R, Mukaigawa Y, Echigo T, Yagi Y (2006) Gait recognition using a view transformation model in the frequency domain. Proceuropean Confcomputer Vision Graz Austria May 2006(5):151–163

  20. Muramatsu D, Shiraishi A, Makihara Y, Uddin MZ, Yagi Y (2014) Gait-based person recognition using arbitrary view transformation model. In: Image Processing IEEE Transactions on, vol. 24, no. 1

    Google Scholar 

  21. Muramatsu D, Makihara Y, Yagi Y (2016) View transformation model incorporating quality measures for cross-view gait recognition. IEEE Transactions on Cybernetics 46:1602–1615

    Article  Google Scholar 

  22. Schroff F, Kalenichenko D, Philbin J (2015) Facenet: A unified embed- ding for face recognition and clustering. In: 2015 IEEE conference on computer vision and pattern recognition (CVPR), pp 815–823

    Chapter  Google Scholar 

  23. Souri Y, Fayyaz M, Gall J (2019) Weakly Supervised Action Segmentation Using Mutual Consistency. arXiv:1904.03116

  24. Takemura N, Makihara Y, Muramatsu D (2017) On input/output architectures for convolutional neural network-based cross-view gait recognition. IEEE Transactions on Circuits and Systems for Video Technology:1–1

  25. Takemura N, Makihara Y, Muramatsu D, Echigo T, Yagi Y (2018) Multi-view large population gait dataset and its performance evaluation for cross-view gait recognition. IPSJ Transactions on Computer Vision and Applications 10:4

    Article  Google Scholar 

  26. Tang J, Luo J, Tjahjadi T, Guo F (2017) Robust arbitrary-view gait recognition based on 3d partial similarity matching. IEEE Trans Image Process 26(1):7–22

    Article  MathSciNet  Google Scholar 

  27. Wang X, Girshick RB, Gupta A, He K (2017) Non-local neural networks. CoRR, vol. abs/1711.07971

  28. Wang X, Wang J, Yan K (2018) Gait recognition based on gabor wavelets and (2d)2pca. Multimed Tools Appl 77:12545–12561

    Article  Google Scholar 

  29. Wolf T, Babaee M, Rigoll G (2016) Multi-view gait recognition using 3d convolutional neural networks. In: International Conference on Image Processing (ICIP). IEEE, Phoenix, pp 4165–4169

    Google Scholar 

  30. Wolf T, Babaee M, Rigoll G (2016) Multi-view gait recognition using 3d convolutional neural networks. In: IEEE international conference on image processing

  31. Worapan K, Qiang W, Jian Z, Hongdong L, Liang W (2014) Recognizing gaits across views through correlated motion co-clustering. IEEE Trans Image Process 23(2):696–709

    Article  MathSciNet  Google Scholar 

  32. Wu, Runmin 2019, et al. “A Mutual Learning Method for Salient Object Detection with Intertwined Multi-Supervision.” Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition

  33. Wu Z, Huang Y, Wang L, Wang X, Tan T (2016) A comprehensive study on cross-view gait based human identification with deep cnns. IEEE Trans Pattern Anal Mach Intell 39(2):209–226

    Article  Google Scholar 

  34. Wu H, Jian W, Xin C, Wei L (2018) Feedback weight convolutional neural network for gait recognition. J. Vis. Commun. Image Represent:S1047320318301445

  35. Wu D, Yang H, Huang D (2018) Omni-directional feature learning for person re-identification. CoRR, vol. abs/1812.05319

  36. Xing X, Wang K, Yan T, Lv Z (2016) Complete canonical correlation analysis with application to multi-view gait recognition. Pattern Recogn 50:107–117

    Article  Google Scholar 

  37. Yu S, Tan D, Tan T (2006) A framework for evaluating the effect of view angle, clothing and carrying condition on gait recognition. In: International conference on pattern recognition

    Google Scholar 

  38. Zhang Y, Xiang T, Hospedales TM, Lu H (2017) Deep mutual learning. CoRR, vol. abs/1706.00384

  39. Zheng S, Zhang J, Huang K, He R, Tan T (2011) Robust view transformation model for gait recognition. In: International Conference on Image Processing(ICIP) (Brussels)

Download references

Acknowledgments

This work was supported in part by the National Key R&D Program of China (2018YFB1307403 and 2019YFB1311001), in part by the National Natural Science Foundation of China (61876099), in part by the Scientific and Technological Development Project of Shandong Province (2019GSF111002). We are very grateful to the CASIA-B Database from Institute of Automation, Chinese Academy of Sciences and the OU-ISIR Gait Database from Institute of Scientific and Industrial Research (ISIR), Osaka University (OU).

Author information

Authors and Affiliations

  1. School of Control Science and Engineering, Shandong University, Jinan, 250061, China

    Yanchun Wang, Zhenxue Chen & Xuewen Rong

  2. Department of Electrical and Computer Engineering, University of Windsor, Windsor, N9B 3P4, Canada

    Q. M. Jonathan Wu

Authors
  1. Yanchun Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Zhenxue Chen
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Q. M. Jonathan Wu
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Xuewen Rong
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Zhenxue Chen.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Wang, Y., Chen, Z., Wu, Q.M.J. et al. Deep mutual learning network for gait recognition. Multimed Tools Appl 79, 22653–22672 (2020). https://doi.org/10.1007/s11042-020-09003-4

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-020-09003-4

Keywords

Search

Navigation