Lingxiang Yao
Abstract
For decades, very few methods were proposed for cross-mode (i.e., walking vs. running) gait recognition. Thus, it remains largely unexplored regarding how to recognize persons by the way they walk and run. Existing cross-mode methods handle the walking-versus-running problem in two ways, either by exploring the generic mapping relation between walking and running modes or by extracting gait features which are non-/less vulnerable to the changes across these two modes. However, for the first approach, a mapping relation fit for one person may not be applicable to another person. There is no generic mapping relation given that walking and running are two highly self-related motions. The second approach does not give more attention to the disparity between walking and running modes, since mode labels are not involved in their feature learning processes. Distinct from these existing cross-mode methods, in our method, mode labels are used in the feature learning process, and a mode-invariant gait descriptor is hybridized for cross-mode gait recognition to handle this walking-versus-running problem. Further research is organized in this article to investigate the disparity between walking and running. Running is different from walking not only in the speed variances but also, more significantly, in prominent gesture/motion changes. According to these rationales, in our proposed method, we give more attention to the differences between walking and running modes, and a robust gait descriptor is developed to hybridize the mode-invariant spatial and temporal features. Two multi-task learning-based networks are proposed in this method to explore these mode-invariant features. Spatial features describe the body parts non-/less affected by mode changes, and temporal features depict the instinct motion relation of each person. Mode labels are also adopted in the training phase to guide the network to give more attention to the disparity across walking and running modes. In addition, relevant experiments on OU-ISIR Treadmill Dataset A have affirmed the effectiveness and feasibility of the proposed method. A state-of-the-art result can be achieved by our proposed method on this dataset.
- [1] Nick Carey. 2005. Establishing Pedestrian Walking Speeds. https://www.westernite.org/datacollectionfund/2005/psu_ped_summary.pdf.Google Scholar
- [2] TranSafety, Inc. 1997. Study Compares Older and Younger Pedestrian Walking Speeds. https://en.wikipedia.org/wiki/Walking.Google Scholar
- [3] . 2018. Gaitset: Regarding gait as a set for cross-view gait recognition. arXiv:1811.06186.Google Scholar
- [4] . [n.d.]. Deformgait: Gait recognition under posture changes using deformation patterns between gait feature pairs. In IEEE International Joint Conference on Biometrics (IJCB’20), Houston. IEEE, 1–10.Google Scholar
- [5] . 2020. Gaitpart: Temporal part-based model for gait recognition. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Seattle. IEEE, 14225–14233.Google ScholarCross Ref
- [6] . 2019. Slowfast networks for video recognition. In Proceedings of the IEEE/CVF International Conference on Computer Vision, Seoul. IEEE, 6202–6211.Google ScholarCross Ref
- [7] . 2016. Learning effective gait features using LSTM. In 23rd International Conference on Pattern Recognition (ICPR’16), Cancun. IEEE, 325–330.Google ScholarCross Ref
- [8] . 2013. A robust speed-invariant gait recognition system for walker and runner identification. In International Conference on Biometrics (ICB’13), Madrid. IEEE, 1–8.Google ScholarCross Ref
- [9] . 2005. Individual recognition using gait energy image. IEEE Transactions on Pattern Analysis and Machine Intelligence 28, 2 (2005), 316–322.Google ScholarDigital Library
- [10] . 2017. In defense of the triplet loss for person re-identification. arXiv:1703.07737.Google Scholar
- [11] . 2019. Compare more nuanced: Pairwise alignment bilinear network for few-shot fine-grained learning. arXiv:1904.03580.Google Scholar
- [12] . 2019. SBSGAN: Suppression of inter-domain background shift for person re-identification. In IEEE International Conference on Computer Vision (ICCV’19), Seoul. IEEE, 9526–9535.Google ScholarCross Ref
- [13] . 2018. Multi-pseudo regularized label for generated data in person re-identification. IEEE Transactions on Image Processing 28, 3 (2018), 1391–1403.Google ScholarDigital Library
- [14] . 2014. Adam: A method for stochastic optimization. arXiv:1412.6980.Google Scholar
- [15] . 2020. Gait recognition via semi-supervised disentangled representation learning to identity and covariate features. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Seattle. IEEE, 13309–13319.Google ScholarCross Ref
- [16] . 2006. Improved gait recognition by gait dynamics normalization. IEEE Transactions on Pattern Analysis and Machine Intelligence6 (2006), 863–876.Google Scholar
- [17] . 2012. Can gait fluctuations improve gait recognition?. In Proceedings of the 21st International Conference on Pattern Recognition (ICPR’12), Tsukuba. IEEE, 3276–3279.Google Scholar
- [18] . 2012. The OU-ISIR gait database comprising the treadmill dataset. IPSJ Transactions on Computer Vision and Applications 4 (2012), 53–62.Google ScholarCross Ref
- [19] . 1999. Gait recognition: Databases, representations, and applications. Wiley Encyclopedia of Electrical and Electronics Engineering (1999), 1–15.Google Scholar
- [20] . 2014. Cross-view gait recognition using view-dependent discriminative analysis. In IEEE International Joint Conference on Biometrics, Clearwater. IEEE, 1–8.Google ScholarCross Ref
- [21] . 2016. Geinet: View-invariant gait recognition using a convolutional neural network. In International Conference on Biometrics (ICB’16), Halmstad. IEEE, 1–8.Google ScholarCross Ref
- [22] . 2014. Two-stream convolutional networks for action recognition in videos. arXiv:1406.2199.Google Scholar
- [23] . 2019. Deep high-resolution representation learning for human pose estimation. arXiv:1902.09212.Google Scholar
- [24] . 2003. A hidden Markov model based framework for recognition of humans from gait sequences. In Proceedings of the 2003 International Conference on Image Processing (Cat. No. 03CH37429), Vol. 2, Barcelona. IEEE, II–93.Google ScholarCross Ref
- [25] . 2017. On input/output architectures for convolutional neural network-based cross-view gait recognition. IEEE Transactions on Circuits and Systems for Video Technology 29 (2019), 2708–2719.Google Scholar
- [26] . 2018. A closer look at spatiotemporal convolutions for action recognition. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 6450–6459.Google ScholarCross Ref
- [27] . 2011. Action recognition by dense trajectories. In CVPR’11, Colorado Springs. IEEE, 3169–3176.Google Scholar
- [28] . 2018. Orthogonal deep features decomposition for age-invariant face recognition. In Proceedings of the European Conference on Computer Vision (ECCV’18), Munich. 738–753.Google ScholarCross Ref
- [29] . 2009. Biomechanics and Motor Control of Human Movement. John Wiley & Sons.Google ScholarCross Ref
- [30] . 2016. Multi-view gait recognition using 3D convolutional neural networks. In IEEE International Conference on Image Processing (ICIP’16), Phoenix. IEEE, 4165–4169.Google ScholarCross Ref
- [31] . 2015. Learning representative deep features for image set analysis. IEEE Transactions on Multimedia 17, 11 (2015), 1960–1968.Google ScholarDigital Library
- [32] . 2016. A comprehensive study on cross-view gait based human identification with deep cnns. IEEE Transactions on Pattern Analysis and Machine Intelligence 39, 2 (2016), 209–226.Google ScholarDigital Library
- [33] . 2019. Speed-invariant gait recognition using single-support gait energy image. Multimedia Tools and Applications (2019), 1–28.Google Scholar
- [34] . 2020. Cross-view gait recognition using pairwise spatial transformer networks. IEEE Transactions on Circuits and Systems for Video Technology 31 (2021), 260–274.Google Scholar
- [35] . 2001. Extended model-based automatic gait recognition of walking and running. In International Conference on Audio-and Video-based Biometric Person Authentication, Halmstad. Springer, 278–283.Google ScholarCross Ref
- [36] . 2002. Gait recognition by walking and running: A model-based approach.Google Scholar
- [37] . 2002. On the relationship of human walking and running: Automatic person identification by gait. In Object Recognition Supported by User Interaction for Service Robots, Vol. 1. IEEE, 287–290.Google Scholar
- [38] . 2016. Relative distance features for gait recognition with Kinect. Journal of Visual Communication and Image Representation 39 (2016), 209–217.Google ScholarDigital Library
- [39] . 2017. Robust gait recognition under unconstrained environments using hybrid descriptions. In International Conference on Digital Image Computing: Techniques and Applications (DICTA’17), Sydney. IEEE, 1–7.Google ScholarCross Ref
- [40] . 2018. Robust CNN-based gait verification and identification using skeleton gait energy image. In Digital Image Computing: Techniques and Applications (DICTA’18), Canberra. IEEE, 1–7.Google Scholar
- [41] . 2021. Robust gait recognition using hybrid descriptors based on skeleton gait energy image. Pattern Recognition Letters 150 (2021), 289–296.Google ScholarDigital Library
- [42] . 2016. Siamese neural network based gait recognition for human identification. In IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP’16), Shanghai. IEEE, 2832–2836.Google ScholarDigital Library
- [43] . 2019. VT-GAN: View transformation GAN for gait recognition across views. In International Joint Conference on Neural Networks (IJCNN’1), Budapest. IEEE, 1–8.Google ScholarCross Ref
- [44] . 2019. Cross-view gait recognition by discriminative feature learning. IEEE Transactions on Image Processing 29 (2019), 1001–1015.Google ScholarCross Ref
Index Terms
- Recognizing Gaits Across Walking and Running Speeds
Recommendations
Gait recognition across different walking speeds via deterministic learning
Deformation of gait silhouettes caused by objects under different walking speeds has a significant effect on the performance of gait recognition. In this paper, we present an algorithm via deterministic learning theory to eliminate the effect of walking ...
Two-phase discontinuous gaits for quadruped walking machines with a failed leg
A fault-tolerant gait of multi-legged robots with static walking is a gait which can maintain gait stability and continue its walking against an occurrence of a leg failure. This paper proposes two-phase discontinuous gaits as a new fault-tolerant gait ...
Arm swing effects on walking bipedal gaits composed of impact, single and double support phases
In human walking, it is often assumed that the arms have a passive movement which reduces the energy consumption of walking. The issue addressed in this work is the influence of the arms on the walking of a humanoid robot. The study has two objectives: ...
Comments