ABSTRACT
Gait recognition is an emerging biometric technology for identifying pedestrians based on their unique walking patterns. In past gait recognition, global-based methods are inadequate to meet the growing demand for accuracy, while commonly used part-based methods provided coarse and inaccurate feature representation for specific body parts. Human parsing appears to be a better option for accurately representing specific and complete body parts in gait recognition. However, its practical application in gait recognition is often hindered by missing RGB modality, lack of annotated body parts, and difficulty in balancing parsing quantity and quality. To address this issue, we propose LandmarkGait, an accessible and alternative parsing-based solution for gait recognition. LandmarkGait introduces an unsupervised landmark discovery network to transform the dense silhouette into a finite set of landmarks with remarkable consistency across various conditions. By grouping landmarks subsets corresponding to distinct body part regions, following a reconstruction task and further refinement from high-quality input silhouettes, we can directly obtain fine-grained parsing results from original binary silhouettes in an unsupervised manner. Moreover, we also develop a multi-scale feature extractor that simultaneously captures global and parsing feature representations based on the integrity and flexibility of specific body parts. Extensive experiments demonstrate that our LandmarkGait can extract more stable features and exhibit significant performance improvement under all conditions, especially in various dressing conditions. Code is available at https://github.com/wzb-bupt/LandmarkGait.
- Hanqing Chao, Yiwei He, Junping Zhang, and Jianfeng Feng. 2019. Gaitset: Regarding gait as a set for cross-view gait recognition. In Proceedings of the AAAI Conference on Artificial Intelligence. 8126--8133.Google ScholarDigital Library
- Bowen Cheng, Omkar Parkhi, and Alexander Kirillov. 2022. Pointly-supervised instance segmentation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2617--2626.Google ScholarCross Ref
- Xiaohan Ding, Xiangyu Zhang, Jungong Han, and Guiguang Ding. 2021. Diverse branch block: Building a convolution as an inception-like unit. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 10886--10895.Google ScholarCross Ref
- Chao Fan, Junhao Liang, Chuanfu Shen, Saihui Hou, Yongzhen Huang, and Shiqi Yu. 2023. OpenGait: Revisiting Gait Recognition Towards Better Practicality. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 9707--9716.Google ScholarCross Ref
- Chao Fan, Yunjie Peng, Chunshui Cao, Xu Liu, Saihui Hou, Jiannan Chi, Yongzhen Huang, Qing Li, and Zhiqiang He. 2020. Gaitpart: Temporal part-based model for gait recognition. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 14225--14233.Google ScholarCross Ref
- Junsong Fan, Zhaoxiang Zhang, and Tieniu Tan. 2022. Pointly-supervised panoptic segmentation. In Proceedings of the European Conference on Computer Vision. Springer, 319--336.Google ScholarDigital Library
- Yang Fu, Shibei Meng, Saihui Hou, Xuecai Hu, and Yongzhen Huang. 2023. GPGait: Generalized Pose-based Gait Recognition. arXiv preprint arXiv:2303.05234 (2023).Google Scholar
- Ke Gong, Xiaodan Liang, Dongyu Zhang, Xiaohui Shen, and Liang Lin. 2017. Look into person: Self-supervised structure-sensitive learning and a new benchmark for human parsing. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 932--940.Google ScholarCross Ref
- Xinqian Gu, Hong Chang, Bingpeng Ma, Shutao Bai, Shiguang Shan, and Xilin Chen. 2022. Clothes-Changing person re-identification with RGB modality only. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 1060--1069.Google ScholarCross Ref
- Xiao Han, Peishan Cong, Lan Xu, Jingya Wang, Jingyi Yu, and Yuexin Ma. 2022. LiCamGait: Gait recognition in the wild by using LiDAR and camera multi-modal visual sensors. arXiv preprint arXiv:2211.12371 (2022).Google Scholar
- Kaiming He, Georgia Gkioxari, Piotr Dollár, and Ross Girshick. 2017. Mask r-cnn. In Proceedings of the IEEE International Conference on Computer Vision. 2961--2969.Google ScholarCross Ref
- Saihui Hou, Chunshui Cao, Xu Liu, and Yongzhen Huang. 2020. Gait lateral network: Learning discriminative and compact representations for gait recognition. In Proceedings of the European Conference on Computer Vision. Springer, 382--398.Google ScholarDigital Library
- Saihui Hou, Xu Liu, Chunshui Cao, and Yongzhen Huang. 2021. Set residual network for silhouette-based gait recognition. IEEE Transactions on Biometrics, Behavior, and Identity Science, Vol. 3, 3 (2021), 384--393.Google ScholarCross Ref
- Saihui Hou, Xu Liu, Chunshui Cao, and Yongzhen Huang. 2022. Gait quality aware network: toward the interpretability of silhouette-based gait recognition. IEEE Transactions on Neural Networks and Learning Systems (2022).Google Scholar
- Hung-Min Hsu, Yizhou Wang, Cheng-Yen Yang, Jenq-Neng Hwang, Hoang Le Uyen Thuc, and Kwang-Ju Kim. 2022. GAITTAKE: Gait recognition by temporal attention and keypoint-guided embedding. In IEEE International Conference on Image Processing. 2546--2550.Google ScholarCross Ref
- Jie Hu, Li Shen, and Gang Sun. 2018. Squeeze-and-excitation networks. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 7132--7141.Google ScholarCross Ref
- Xiaohu Huang, Duowang Zhu, Hao Wang, Xinggang Wang, Bo Yang, Botao He, Wenyu Liu, and Bin Feng. 2021b. Context-sensitive temporal feature learning for gait recognition. In Proceedings of the IEEE/CVF International Conference on Computer Vision. 12909--12918.Google ScholarCross Ref
- Zhen Huang, Dixiu Xue, Xu Shen, Xinmei Tian, Houqiang Li, Jianqiang Huang, and Xian-Sheng Hua. 2021a. 3D local convolutional neural networks for gait recognition. In Proceedings of the IEEE/CVF International Conference on Computer Vision. 14920--14929.Google ScholarCross Ref
- Tomas Jakab, Ankush Gupta, Hakan Bilen, and Andrea Vedaldi. 2018. Unsupervised learning of object landmarks through conditional image generation. Advances in Neural Information Processing Systems, Vol. 31 (2018).Google Scholar
- Zhenchao Jin, Bin Liu, Qi Chu, and Nenghai Yu. 2021. ISNet: Integrate image-level and semantic-level context for semantic segmentation. In Proceedings of the IEEE/CVF International Conference on Computer Vision. 7189--7198.Google ScholarCross Ref
- Alexander Kirillov, Eric Mintun, Nikhila Ravi, Hanzi Mao, Chloe Rolland, Laura Gustafson, Tete Xiao, Spencer Whitehead, Alexander C Berg, Wan-Yen Lo, et al. 2023. Segment anything. arXiv preprint arXiv:2304.02643 (2023).Google Scholar
- Tejas D Kulkarni, Ankush Gupta, Catalin Ionescu, Sebastian Borgeaud, Malcolm Reynolds, Andrew Zisserman, and Volodymyr Mnih. 2019a. Unsupervised learning of object keypoints for perception and control. Advances in Neural Information Processing Systems, Vol. 32 (2019).Google Scholar
- Tejas D Kulkarni, Ankush Gupta, Catalin Ionescu, Sebastian Borgeaud, Malcolm Reynolds, Andrew Zisserman, and Volodymyr Mnih. 2019b. Unsupervised learning of object keypoints for perception and control. Advances in Neural Information Processing Systems, Vol. 32 (2019).Google Scholar
- Guodong Li, Lijun Guo, Rong Zhang, Jiangbo Qian, and Shangce Gao. 2023 a. TransGait: Multimodal-based gait recognition with set transformer. Applied Intelligence, Vol. 53, 2 (2023), 1535--1547.Google ScholarDigital Library
- Peike Li, Yunqiu Xu, Yunchao Wei, and Yi Yang. 2020. Self-correction for human parsing. IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol. 44, 6 (2020), 3260--3271.Google ScholarCross Ref
- Weijia Li, Saihui Hou, Chunjie Zhang, Chunshui Cao, Xu Liu, Yongzhen Huang, and Yao Zhao. 2023 b. An in-depth exploration of person re-identification and gait recognition in cloth-changing conditions. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition.Google ScholarCross Ref
- Xiaodan Liang, Si Liu, Xiaohui Shen, Jianchao Yang, Luoqi Liu, Jian Dong, Liang Lin, and Shuicheng Yan. 2015. Deep human parsing with active template regression. IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol. 37, 12 (2015), 2402--2414.Google ScholarDigital Library
- Rijun Liao, Shiqi Yu, Weizhi An, and Yongzhen Huang. 2020. A model-based gait recognition method with body pose and human prior knowledge. Pattern Recognition, Vol. 98 (2020), 107069.Google ScholarDigital Library
- Beibei Lin, Shunli Zhang, and Feng Bao. 2020. Gait recognition with multiple-temporal-scale 3d convolutional neural network. In Proceedings of the 28th ACM International Conference on Multimedia. 3054--3062.Google ScholarDigital Library
- Beibei Lin, Shunli Zhang, and Xin Yu. 2021. Gait recognition via effective global-local feature representation and local temporal aggregation. In Proceedings of the IEEE/CVF International Conference on Computer Vision. 14648--14656.Google ScholarCross Ref
- Kunliang Liu, Ouk Choi, Jianming Wang, and Wonjun Hwang. 2022. Cdgnet: Class distribution guided network for human parsing. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 4473--4482.Google ScholarCross Ref
- Xin Liu, Chen Zhao, Bin Zheng, Qinwei Guo, Xiaoqin Duan, Aziguli Wulamu, and Dezheng Zhang. 2021. Wearable devices for gait analysis in intelligent healthcare. Frontiers in Computer Science, Vol. 3 (2021), 661676.Google ScholarCross Ref
- Hao Luo, Youzhi Gu, Xingyu Liao, Shenqi Lai, and Wei Jiang. 2019. Bag of tricks and a strong baseline for deep person re-identification. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops.Google ScholarCross Ref
- Kang Ma, Ying Fu, Dezhi Zheng, Chunshui Cao, Xuecai Hu, and Yongzhen Huang. 2023. Dynamic Aggregated Network for Gait Recognition. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 22076--22085.Google ScholarCross Ref
- Yasushi Makihara, Hidetoshi Mannami, Akira Tsuji, Md Altab Hossain, Kazushige Sugiura, Atsushi Mori, and Yasushi Yagi. 2012. The OU-ISIR gait database comprising the treadmill dataset. IPSJ Transactions on Computer Vision and Applications, Vol. 4 (2012), 53--62.Google ScholarCross Ref
- Dimitrios Mallis, Enrique Sanchez, Matthew Bell, and Georgios Tzimiropoulos. 2020. Unsupervised learning of object landmarks via self-training correspondence. Advances in Neural Information Processing Systems, Vol. 33 (2020), 4709--4720.Google Scholar
- Honghu Pan, Yongyong Chen, Tingyang Xu, Yunqi He, and Zhenyu He. 2023. Toward complete-view and high-level pose-based gait recognition. IEEE Transactions on Information Forensics and Security, Vol. 18 (2023), 2104--2118.Google ScholarDigital Library
- Yunjie Peng, Kang Ma, Yang Zhang, and Zhiqiang He. 2023. Learning rich features for gait recognition by integrating skeletons and silhouettes. Multimedia Tools and Applications (2023), 1--22.Google Scholar
- Xuqian Ren, Saihui Hou, Chunshui Cao, Xu Liu, and Yongzhen Huang. 2022. Progressive feature learning for realistic cloth-changing gait recognition. arXiv preprint arXiv:2207.11720 (2022).Google Scholar
- Imad Rida, Noor Almaadeed, and Somaya Almaadeed. 2019. Robust gait recognition: a comprehensive survey. IET Biometrics, Vol. 8, 1 (2019), 14--28.Google ScholarCross Ref
- Enrique Sanchez and Georgios Tzimiropoulos. 2019. Object landmark discovery through unsupervised adaptation. Advances in Neural Information Processing Systems, Vol. 32 (2019).Google Scholar
- Alireza Sepas-Moghaddam and Ali Etemad. 2022. Deep gait recognition: A survey. IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol. 45, 1 (2022), 264--284.Google ScholarCross Ref
- Chuanfu Shen, Chao Fan, Wei Wu, Rui Wang, George Q Huang, and Shiqi Yu. 2023. LidarGait: Benchmarking 3D Gait Recognition With Point Clouds. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 1054--1063.Google ScholarCross Ref
- Zhixin Shu, Mihir Sahasrabudhe, Riza Alp Guler, Dimitris Samaras, Nikos Paragios, and Iasonas Kokkinos. 2018. Deforming autoencoders: Unsupervised disentangling of shape and appearance. In Proceedings of the European Conference on Computer Vision. 650--665.Google ScholarDigital Library
- Jamie D Shutler, Michael G Grant, Mark S Nixon, and John N Carter. 2004. On a large sequence-based human gait database. In Applications and Science in Soft Computing. Springer, 339--346.Google Scholar
- Chunfeng Song, Yongzhen Huang, Yan Huang, Ning Jia, and Liang Wang. 2019. Gaitnet: An end-to-end network for gait based human identification. Pattern recognition, Vol. 96 (2019), 106988.Google Scholar
- Christian Szegedy, Sergey Ioffe, Vincent Vanhoucke, and Alexander Alemi. 2017. Inception-v4, inception-resnet and the impact of residual connections on learning. In Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 31.Google ScholarCross Ref
- Noriko Takemura, Yasushi Makihara, Daigo Muramatsu, Tomio Echigo, and Yasushi Yagi. 2018. Multi-view large population gait dataset and its performance evaluation for cross-view gait recognition. IPSJ transactions on Computer Vision and Applications, Vol. 10 (2018), 1--14.Google Scholar
- Torben Teepe, Johannes Gilg, Fabian Herzog, Stefan Hörmann, and Gerhard Rigoll. 2022. Towards a deeper understanding of skeleton-based gait recognition. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 1569--1577.Google ScholarCross Ref
- Torben Teepe, Ali Khan, Johannes Gilg, Fabian Herzog, Stefan Hörmann, and Gerhard Rigoll. 2021. Gaitgraph: Graph convolutional network for skeleton-based gait recognition. In IEEE International Conference on Image Processing. 2314--2318.Google ScholarCross Ref
- James Thewlis, Hakan Bilen, and Andrea Vedaldi. 2017. Unsupervised learning of object landmarks by factorized spatial embeddings. In Proceedings of the IEEE International Conference on Computer Vision. 5916--5925.Google ScholarCross Ref
- Changsheng Wan, Li Wang, and Vir V Phoha. 2018. A survey on gait recognition. Comput. Surveys, Vol. 51, 5 (2018), 1--35.Google ScholarDigital Library
- Ming Wang, Xianda Guo, Beibei Lin, Tian Yang, Zheng Zhu, Lincheng Li, Shunli Zhang, and Xin Yu. 2023. DyGait: Exploiting Dynamic Representations for High-performance Gait Recognition. arXiv preprint arXiv:2303.14953 (2023).Google Scholar
- Meng Ye, Mikael Kanski, Dong Yang, Qi Chang, Zhennan Yan, Qiaoying Huang, Leon Axel, and Dimitris Metaxas. 2021. Deeptag: An unsupervised deep learning method for motion tracking on cardiac tagging magnetic resonance images. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 7261--7271.Google ScholarCross Ref
- Shiqi Yu, Daoliang Tan, and Tieniu Tan. 2006. A framework for evaluating the effect of view angle, clothing and carrying condition on gait recognition. In International Conference on Pattern Recognition, Vol. 4. IEEE, 441--444.Google Scholar
- Lazaros Zafeiriou, Epameinondas Antonakos, Stefanos Zafeiriou, and Maja Pantic. 2014. Joint unsupervised face alignment and behaviour analysis. In Proceedings of the European Conference on Computer Vision. Springer, 167--183.Google ScholarCross Ref
- Dingwen Zhang, Junwei Han, and Yu Zhang. 2017. Supervision by fusion: Towards unsupervised learning of deep salient object detector. In Proceedings of the IEEE International Conference on Computer Vision. 4048--4056.Google ScholarCross Ref
- Yuting Zhang, Yijie Guo, Yixin Jin, Yijun Luo, Zhiyuan He, and Honglak Lee. 2018. Unsupervised discovery of object landmarks as structural representations. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2694--2703.Google ScholarCross Ref
- Ziyuan Zhang, Luan Tran, Feng Liu, and Xiaoming Liu. 2020. On learning disentangled representations for gait recognition. IEEE Transactions on Pattern Analysis and Machine Intelligence (2020).Google Scholar
- Jinkai Zheng, Xinchen Liu, Xiaoyan Gu, Yaoqi Sun, Chuang Gan, Jiyong Zhang, Wu Liu, and Chenggang Yan. 2022a. Gait recognition in the wild with multi-hop temporal switch. In Proceedings of the 30th ACM International Conference on Multimedia. 6136--6145.Google ScholarDigital Library
- Jinkai Zheng, Xinchen Liu, Wu Liu, Lingxiao He, Chenggang Yan, and Tao Mei. 2022b. Gait recognition in the wild with dense 3d representations and a benchmark. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 20228--20237.Google ScholarCross Ref
- Yongsheng Zhu, Fengxin Sun, Changjun Jia, Chaorui Huang, Kuo Wang, Ying Li, Liping Chou, and Yupeng Mao. 2022. A 3D printing triboelectric sensor for gait analysis and virtual control based on human-computer interaction and the internet of things. Sustainability, Vol. 14, 17 (2022), 10875.Google ScholarCross Ref
- Zheng Zhu, Xianda Guo, Tian Yang, Junjie Huang, Jiankang Deng, Guan Huang, Dalong Du, Jiwen Lu, and Jie Zhou. 2021. Gait recognition in the wild: A benchmark. In Proceedings of the IEEE/CVF International Conference on Computer Vision. 14789--14799.Google Scholar
Index Terms
- LandmarkGait: Intrinsic Human Parsing for Gait Recognition
Recommendations
Learning hierarchical poselets for human parsing
CVPR '11: Proceedings of the 2011 IEEE Conference on Computer Vision and Pattern RecognitionWe consider the problem of human parsing with part-based models. Most previous work in part-based models only considers rigid parts (e.g. torso, head, half limbs) guided by human anatomy. We argue that this representation of parts is not necessarily ...
Gait Recognition for Human Identification using Kinect
RACS '17: Proceedings of the International Conference on Research in Adaptive and Convergent SystemsGait is a pattern of biometric movement for human identification. Unlike other biometrics such as fingerprint, iris, face, and voice recognition, human gait can be captured with unobtrusive method. In this paper, several measurements are proposed which ...
A model-based gait recognition method with body pose and human prior knowledge
Highlights- We propose a novel model-based gait recognition method, PoseGait, which exploits human pose as feature. The method can achieve high recognition rate despite ...
AbstractWe propose in this paper a novel model-based gait recognition method, PoseGait. Gait recognition is a challenging and attractive task in biometrics. Early approaches to gait recognition were mainly appearance-based. The appearance-...
Comments