MingDa Li
ABSTRACT
Skeleton data has been widely used in action recognition tasks to achieve effective recognition because it can stably adapt to dynamic environments and complex backgrounds. In those existing methods, two kinds of information, joint and bone segments, have been proven to be very useful. However, the inexplicit motion information, which is between bones, is always be ignored. Therefore, how to combine these three kinds of information, how to make full use of the joint information of bones and the relationship between bone segments and bone movement information, and how to represent bone movement information are still problems that need to be solved urgently. This paper designs a triple-stream undirected graph neural network, named 3s-AGCN, to extract motion information features of joints, bone segments, and bones for action recognition. Our final model has been tested on the large data set NTU-RGBD and has reached the latest level of the current model.
- Lin Li, Wu Zheng, Zhaoxiang Zhang, Yan Huang, and Liang Wang. Skeleton-Based Relational Modeling for Action Recognition. arXiv:1805.02556 [cs], 2018.Google Scholar
- Shuai Li, Wanqing Li, Chris Cook, Ce Zhu, and Yanbo Gao. Independently recurrent neural network (indrnn): Building A longer and deeper RNN. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 5457--5466, 2018.Google ScholarCross Ref
- Qiuhong Ke, Mohammed Bennamoun, Senjian An, Ferdous Ahmed Sohel, and Farid Broussard. A New Representation of Skeleton Sequences for 3d Action Recognition. 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pages 4570--4579, 2017.Google ScholarCross Ref
- Chao Li, Qiaoyong Zhong, Di Xie, and Shiliang Pu. Skeleton-based action recognition with convolutional neural networks. In Multimedia & Expo Workshops (ICMEW), 2017 IEEE International Conference on, pages 597--600. IEEE, 2017.Google Scholar
- Sijie Yan, Yuanjun Xiong, and Dahua Lin. Spatial-Temporal Graph Convolutional Networks for Skeleton-Based Action Recognition. In AAAI, 2018.Google ScholarCross Ref
- Amir Shahroudy, Jun Liu, Tian-Tsong Ng, and Gang Wang. NTU RGB+D: A Large Scale Dataset for 3d Human Activity Analysis. The IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2016.Google ScholarCross Ref
- Lei Shi, Yifan Zhang, Jian Cheng, and Hanqing Lu. Non Local Graph Convolutional Networks for Skeleton-Based Action Recognition. arXiv:1805.07694 [cs], May 2018.Google Scholar
- Raviteja Vemulapalli, Felipe Arrate, and Rama Chellappa. Human action recognition by representing 3d skeletons as points in a lie group. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 588--595, 2014.Google ScholarDigital Library
- Pengfei Zhang, Curling Lan, Junliang Xing, Wenjun Zeng, Jianru Xue, and Nanning Zheng. View Adaptive Recurrent Neural Networks for High-Performance Human Action Recognition From Skeleton Data. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 2117--2126, 2017.Google ScholarCross Ref
- Mengyuan Liu, Hong Liu, and Chen Chen. Enhanced skeleton visualization for view-invariant human action recognition. Pattern Recognition, 68:346--362, 2017. 1, 2, 8Google ScholarDigital Library
- Bo Li, Yuchao Dai, Xuelian Cheng, Huahui Chen, Yi Lin, and Mingyi He. Skeleton based action recognition using translation-scale invariant image mapping and multi-scale deep CNN. In Multimedia & Expo Workshops (ICMEW), 2017 IEEE International Conference on, pages 601--604. IEEE, 2017.Google Scholar
- Yong Du, Wei Wang, and Liang Wang. Hierarchical recurrent neural network for skeleton-based action recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 1110--1118, 2015.Google Scholar
- Jun Liu, Amir Shahroudy, Dong Xu, and Gang Wang. Spatial-Temporal LSTM with Trust Gates for 3d Human Action Recognition. In Computer Vision ECCV 2016, volume 9907, pages 816--833. SpringerGoogle Scholar
- Hong Liu, Juanhui Tu, and Mengyuan Liu. Two-Stream 3d Convolutional Neural Network for Skeleton-Based Action Recognition. arXiv:1705.08106 [cs], May 2017..Google Scholar
- Tae Soo Kim and Austin Reiter. Interpretable 3d human action analysis with temporal convolutional networks. In Computer Vision and Pattern Recognition Workshops (CVPRW), 2017 IEEE Conference on, pages 1623--1631, 2017.Google ScholarCross Ref
- Karen Simonyan and Andrew Zisserman. Two-stream convolutional networks for action recognition in videos. In Advances in neural information processing systems, pages 568-- 576, 2014.Google ScholarDigital Library
- C. Cao, C. Lan, Y. Zhang, W. Zeng, H. Lu, and Y. Zhang. Skeleton-Based Action Recognition with Gated Convolutional Neural Networks. IEEE Transactions on Circuits and Systems for Video Technology pages 1--1, 2018.Google Scholar
- Evangelidis, G., Singh, G., Horaud, R.: Skeletal quads: Human action recognition using joint quadruples. In: IEEE International Conference on Pattern Recognition. pp. 4513--4518 (2014)Google Scholar
- Vemulapalli, R., Arrate, F., Chellappa, R.: Human action recognition by representing 3D skeletons as points in a lie group. In: IEEE Conference on Computer Vision and Pattern Recognition. pp. 588--595 (2014)Google Scholar
- Luo, J., Wang, W., Qi, H.: Group sparsity and geometry constrained dictionary learning for action recognition from depth maps. In: IEEE International Conference on Computer Vision. pp. 1809--1816 (2013)Google Scholar
- Rahmani, H., Mian, A.: Learning a non-linear knowledge transfer model for crossview action recognition. In: IEEE Conference on Computer Vision and Pattern Recognition. pp. 2458--2466 (2015)Google Scholar
- Du, Y., Wang, W., Wang, L.: Hierarchical recurrent neural network for skeleton based action recognition. In: IEEE Conference on Computer Vision and Pattern Recognition. pp. 1110--1118 (2015)Google Scholar
- Song, S., Lan, C., Xing, J., Zeng, W., Liu, J.: An end-to-end spatio-temporal attention model for human action recognition from skeleton data. In: AAAI Conference on Artificial Intelligence. pp. 4263--4270 (2017)Google Scholar
- Ke, Q., Bennamoun, M., An, S., Sohel, F., Boussaid, F.: A new representation of skeleton sequences for 3D action recognition. In: IEEE Conference on Computer Vision and Pattern Recognition. pp. 3288--3297 (2017)Google Scholar
- Zhao, R., Ali, H., Van der Smagt, P.: Two-stream RNN/CNN for action recognition in 3D videos. In: IEEE International Conference on Intelligent Robots and Systems. pp. 4260--4267 (2017)Google Scholar
- Bruna, J., Zaremba, W., Szlam, A., Lecun, Y.: Spectral networks and locally connected networks on graphs. In: International Conference on Learning Representations (2014)Google Scholar
- Niepert, M., Ahmed, M., Kutzkov, K.: Learning convolutional neural networks for graphs. In: IEEE International Conference on Machine Learning. pp. 2014--2023 (2016)Google Scholar
- Duvenaud, D.K., Maclaurin, D., Iparraguirre, J., Bombarell, R., Hirzel, T., AspuruGuzik, A., Adams, R.P.: Convolutional networks on graphs for learning molecular fingerprints. In: Advances in Neural Information Processing Systems. pp. 2224-- 2232 (2015)Google Scholar
- Henaff, M., Bruna, J., Lecun, Y.: Deep convolutional networks on graph-structured data. Computer Science (2015)Google Scholar
- Shi, L., Zhang, Y., Cheng, J., Lu, H.: Skeleton-based action recognition with directed graph neural networks. In: IEEE Conference on Computer Vision and Pattern Recognition. pp. 7912--7921 (2019)Google Scholar
- Li, M., Chen, S., Chen, X., Zhang, Y., Wang, Y., Tian, Q.: Actional-structural graph convolutional networks for skeleton-based action recognition. In: IEEE Conference on Computer Vision and Pattern Recognition. pp. 3595--3603Google Scholar
Index Terms
- skeleton-based Action recognition with a triple-stream graph convolutional network
Recommendations
Direction-guided two-stream convolutional neural networks for skeleton-based action recognition
AbstractIn skeleton-based action recognition, treating skeleton data as pseudoimages using convolutional neural networks (CNNs) has proven to be effective. However, among existing CNN-based approaches, most focus on modeling information at the joint-level ...
A comparative review of graph convolutional networks for human skeleton-based action recognition
AbstractHuman action recognition is one of the hottest topics in the research field, so there are many relevant review papers illustrating the multi-modality of data, the selection of feature vectors, and the pros and cons of classification networks. With ...
Joint-Bone Fusion Graph Convolutional Network for Semi-Supervised Skeleton Action Recognition
In recent years, graph convolutional networks (GCNs) play an increasingly critical role in skeleton-based human action recognition. However, most GCN-based methods still have two main limitations: 1) They only consider the motion information of the joints ...
Comments