Ke Chen
ABSTRACT
Human motion recognition is a prominent research topic in the field of computer vision and pattern recognition. This technology has found numerous applications in various fields, such as human-computer interaction, intelligent video surveillance, motion analysis, and medical assistance. Graph convolutional network (GCNs) have achieved outstanding performance in skeleton-based action recognition. However existing GCN-based methods still have some issues. First the adjacency matrix neglecting the dependencies between the non-adjacent joints. Second they rely entirely on human joints ignore modeling of the surrounding environment for modeling complicated dependencies. In this paper, we present a novel approach to skeleton-based motion recognition using a combination of GCNs and deep neural network (DNNs). Our proposed method utilizes GCN to process skeleton data and DNN to process human image data. Subsequently, we fuse the extracted features to make predictions for the final label. We have conducted experiments on the fall detection dataset, and our results indicate that the proposed approach achieves state-of-the-art accuracy.
- Mostafa A Abdelrazik, Abdelhaliem Zekry, and Wael A Mohamed. 2023. Efficient hybrid algorithm for human action recognition. Journal of Image and Graphics 11, 1 (2023), 72–81.Google Scholar
Cross Ref
- Kripesh Adhikari, Hamid Bouchachia, and Hammadi Nait-Charif. 2017. Activity recognition for indoor fall detection using convolutional neural network. In 2017 Fifteenth IAPR International Conference on Machine Vision Applications (MVA). IEEE, 81–84.Google ScholarCross Ref
- Joan Bruna, Wojciech Zaremba, Arthur Szlam, and Yann LeCun. 2013. Spectral networks and locally connected networks on graphs. arXiv preprint arXiv:1312.6203 (2013).Google Scholar
- Ke Cheng, Yifan Zhang, Xiangyu He, Weihan Chen, Jian Cheng, and Hanqing Lu. 2020. Skeleton-based action recognition with shift graph convolutional network. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 183–192.Google ScholarCross Ref
- Michaël Defferrard, Xavier Bresson, and Pierre Vandergheynst. 2016. Convolutional neural networks on graphs with fast localized spectral filtering. Advances in neural information processing systems 29 (2016).Google Scholar
- Bardia Doosti, Shujon Naha, Majid Mirbagheri, and David J Crandall. 2020. Hope-net: A graph-based model for hand-object pose estimation. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. 6608–6617.Google ScholarCross Ref
- Yong Du, Yun Fu, and Liang Wang. 2015. Skeleton based action recognition with convolutional neural network. In 2015 3rd IAPR Asian conference on pattern recognition (ACPR). IEEE, 579–583.Google ScholarCross Ref
- Yong Du, Wei Wang, and Liang Wang. 2015. Hierarchical recurrent neural network for skeleton based action recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition. 1110–1118.Google Scholar
- Muhammad Hassan, Tasweer Ahmad, Nudrat Liaqat, Ali Farooq, Syed Asghar Ali, and Syed Rizwan Hassan. 2014. A review on human actions recognition using vision based techniques. Journal of Image and Graphics 2, 1 (2014), 28–32.Google ScholarCross Ref
- Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. 2016. Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition. 770–778.Google ScholarCross Ref
- Gao Huang, Zhuang Liu, Laurens Van Der Maaten, and Kilian Q Weinberger. 2017. Densely connected convolutional networks. In Proceedings of the IEEE conference on computer vision and pattern recognition. 4700–4708.Google ScholarCross Ref
- Hamid Reza Vaezi Joze, Amirreza Shaban, Michael L Iuzzolino, and Kazuhito Koishida. 2020. MMTM: Multimodal transfer module for CNN fusion. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 13289–13299.Google ScholarCross Ref
- Qiuhong Ke, Mohammed Bennamoun, Senjian An, Ferdous Sohel, and Farid Boussaid. 2017. A new representation of skeleton sequences for 3d action recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition. 3288–3297.Google ScholarCross Ref
- Thomas N Kipf and Max Welling. 2016. Semi-supervised classification with graph convolutional networks. arXiv preprint arXiv:1609.02907 (2016).Google Scholar
- Thomas N Kipf and Max Welling. 2016. Semi-supervised classification with graph convolutional networks. arXiv preprint arXiv:1609.02907 (2016).Google Scholar
- Alex Krizhevsky, Ilya Sutskever, and Geoffrey E Hinton. 2017. Imagenet classification with deep convolutional neural networks. Commun. ACM 60, 6 (2017), 84–90.Google ScholarDigital Library
- Naresh Kumar and Nagarajan Sukavanam. 2018. Motion trajectory for human action recognition using fourier temporal features of skeleton joints. Journal of Image and Graphics 6, 2 (2018), 174–180.Google ScholarCross Ref
- Maosen Li, Siheng Chen, Yangheng Zhao, Ya Zhang, Yanfeng Wang, and Qi Tian. 2020. Dynamic multiscale graph neural networks for 3d skeleton based human motion prediction. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 214–223.Google ScholarCross Ref
- Mark Sandler, Andrew Howard, Menglong Zhu, Andrey Zhmoginov, and Liang-Chieh Chen. 2018. Mobilenetv2: Inverted residuals and linear bottlenecks. In Proceedings of the IEEE conference on computer vision and pattern recognition. 4510–4520.Google ScholarCross Ref
- Lei Shi, Yifan Zhang, Jian Cheng, and Hanqing Lu. 2019. Skeleton-based action recognition with directed graph neural networks. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 7912–7921.Google ScholarCross Ref
- Lei Shi, Yifan Zhang, Jian Cheng, and Hanqing Lu. 2019. Two-stream adaptive graph convolutional networks for skeleton-based action recognition. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. 12026–12035.Google ScholarCross Ref
- Karen Simonyan and Andrew Zisserman. 2014. Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556 (2014).Google Scholar
- Tae Soo Kim and Austin Reiter. 2017. Interpretable 3d human action analysis with temporal convolutional networks. In Proceedings of the IEEE conference on computer vision and pattern recognition workshops. 20–28.Google Scholar
- Christian Szegedy, Wei Liu, Yangqing Jia, Pierre Sermanet, Scott Reed, Dragomir Anguelov, Dumitru Erhan, Vincent Vanhoucke, and Andrew Rabinovich. 2015. Going deeper with convolutions. In Proceedings of the IEEE conference on computer vision and pattern recognition. 1–9.Google ScholarCross Ref
- Mingxing Tan and Quoc Le. 2019. Efficientnet: Rethinking model scaling for convolutional neural networks. In International conference on machine learning. PMLR, 6105–6114.Google Scholar
- Jingdong Wang, Ke Sun, Tianheng Cheng, Borui Jiang, Chaorui Deng, Yang Zhao, Dong Liu, Yadong Mu, Mingkui Tan, Xinggang Wang, 2020. Deep high-resolution representation learning for visual recognition. IEEE transactions on pattern analysis and machine intelligence 43, 10 (2020), 3349–3364.Google Scholar
- Tianhan Xu and Wataru Takano. 2021. Graph stacked hourglass networks for 3d human pose estimation. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. 16105–16114.Google ScholarCross Ref
- Sijie Yan, Yuanjun Xiong, and Dahua Lin. 2018. Spatial temporal graph convolutional networks for skeleton-based action recognition. In Thirty-second AAAI conference on artificial intelligence.Google ScholarCross Ref
- Zhizhi Yu, Di Jin, Ziyang Liu, Dongxiao He, Xiao Wang, Hanghang Tong, and Jiawei Han. 2021. AS-GCN: Adaptive semantic architecture of graph convolutional networks for text-rich networks. In 2021 IEEE International Conference on Data Mining (ICDM). IEEE, 837–846.Google ScholarCross Ref
- Seyma Yucer and Yusuf Sinan Akgul. 2018. 3D human action recognition with siamese-LSTM based deep metric learning. arXiv preprint arXiv:1807.02131 (2018).Google Scholar
- Wojciech Zaremba, Ilya Sutskever, and Oriol Vinyals. 2014. Recurrent neural network regularization. arXiv preprint arXiv:1409.2329 (2014).Google Scholar
- Songyang Zhang, Xiaoming Liu, and Jun Xiao. 2017. On geometric features for skeleton-based action recognition using multilayer lstm networks. In 2017 IEEE Winter Conference on Applications of Computer Vision (WACV). IEEE, 148–157.Google ScholarCross Ref
- Shaobo Zhang, Wanqing Zhao, Ziyu Guan, Xianlin Peng, and Jinye Peng. 2021. Keypoint-graph-driven learning framework for object pose estimation. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. 1065–1073.Google ScholarCross Ref
- Wu Zheng, Lin Li, Zhaoxiang Zhang, Yan Huang, and Liang Wang. 2019. Relational network for skeleton-based action recognition. In 2019 IEEE International Conference on Multimedia and Expo (ICME). IEEE, 826–831.Google ScholarCross Ref
Index Terms
- Skeleton-based Motion Recognition Model by using Graph Convolutional Network and Deep Neural Network
Recommendations
Facial Expression Intensity Estimation using Deep Convolutional Neural Network
ACIT '21: Proceedings of the the 8th International Virtual Conference on Applied Computing & Information TechnologyFacial expression recognition has been applied in various fields, but it is not enough to express complex emotions. It is required to detect the intensity of facial expression because the intensity of facial expression varies depending on the intensity ...
Unconstrained face recognition using deep convolution neural network
Different methods have been proposed for face recognition during the past decades that differ essentially on how to determine discriminant facial features for better recognition. Recently, very deep neural networks achieved great success on general object ...
Facial Expression Recognition Using Convolutional Neural Network
ICVISP 2018: Proceedings of the 2nd International Conference on Vision, Image and Signal ProcessingFacial expressions are part of human language and are often used to convey emotions. With the development of human-computer interaction technology, people pay more and more attention to facial expression recognition (FER) technology. Besides, in the ...
Comments