Abstract
Action recognition is to classify the spatio-temporal changes of the human body as a qualitative pattern, so an efficient representation method that can reflect the structural characteristics of the human body is required. There-fore, in deep learning-based action recognition, graph convolutional network (GCN) algorithms with skeleton data as input were mainly used. However, these methods are difficult to use in the real situation without first obtaining accurate skeleton data. In this paper, we propose a multi-region based radial graph convolutional network (MRGCN) capable of end-to-end action recognition using only optical flow and gradient of the image. This method uses the optical flow and gradient as an oriented histogram, compresses it into a 6-dimensional feature vector, and uses it as an input to the network. Since the network that learns this feature vector has a radial hierarchical structure, it can learn the structural deformation of the human body. As a result of applying a performance experiment on 30 actions, MRGCN obtained Top-1 accuracy of 84.78%, which is higher than that of the existing GCN-based action recognition method. These results show that MRGCN is a high-performance action recognition algorithm suitable for using in the field of surveillance systems where skeleton data can-not be used.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Yan, S., Xiong, Y., Lin, D.: Spatial temporal graph convolutional networks for skeleton-based action recognition. In: Thirty-Second AAAI Conference on Artificial Intelligence, pp. 7444–7452 (2018)
Jang, H.-B., Kim, D.J., Lee, C.W.: Human action recognition based on ST-GCN using optical flow and image gradient. In: The 9th International Conference on Smart Media and Applications, pp. 255–260 (2020)
Jang, H.-B., Lee, C.-W.: ST-GCN based human action recognition with abstracted three features of optical flow and image gradient. In: Jeong, H., Sumi, K. (eds.) IW-FCV 2021. CCIS, vol. 1405, pp. 203–217. Springer, Cham (2021). https://doi.org/10.1007/978-3-030-81638-4_17
Jang, H.-B., Lee, C.W.: A human action recognition based on MRGCN using overlapped data acquisition regions. In: The 10th International Conference on Smart Media and Applications, pp. 10–15 (2021)
Raviteja, V., Arrate, F., Chellappa, R.: Human action recognition by representing 3d skeletons as points in a lie group. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (2014)
Mohamed, E.H., et al.: Human action recognition using a temporal hierarchy of covariance descriptors on 3D joint locations. In: Twenty-Third International Joint Conference on Artificial Intelligence, pp. 2466–2472 (2013)
Li, C., et al.: Skeleton-based action recognition using LSTM and CNN. In: 2017 IEEE International Conference on Multimedia and Expo Workshops (ICMEW), pp. 585–590. IEEE (2017)
Liu, J., Shahroudy, A., Dong, X., Wang, G.: Spatio-temporal LSTM with trust gates for 3D human action recognition. In: Leibe, B., Matas, J., Sebe, N., Welling, M. (eds.) ECCV 2016. LNCS, vol. 9907, pp. 816–833. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-46487-9_50
Du, Y., Wang, W., Wang, L.: Hierarchical recurrent neural network for skeleton based action recognition. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 1110–1118 (2015)
Kalpit, T., Narayanan, P.J.: Part-based graph convolutional network for action recognition. arXiv:1809.04983 (2018)
Li, M., et al.: Actional-structural graph convolutional networks for skeleton-based action recognition. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3590–3598 (2019)
Gao, X., et al.: Optimized skeleton-based action recognition via sparsified graph regression. In: Proceedings of the 27th ACM International Conference on Multimedia, pp. 601–610 (2019)
Amir, S., et al.: NTU RGB+D: A large scale dataset for 3D human activity analysis. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 1010–1019 (2016)
Acknowledgments
This research is supported by Ministry of Culture, Sports, and Tourism (MCST) and Korea Creative Content Agency (KOCCA) in the Culture Technology (CT) Research & Development Program (R2020060002) 2020.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2022 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Jang, HB., Lee, CW. (2022). Multi-region Based Radial GCN Algorithm for Human Action Recognition. In: Sumi, K., Na, I.S., Kaneko, N. (eds) Frontiers of Computer Vision. IW-FCV 2022. Communications in Computer and Information Science, vol 1578. Springer, Cham. https://doi.org/10.1007/978-3-031-06381-7_23
Download citation
DOI: https://doi.org/10.1007/978-3-031-06381-7_23
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-06380-0
Online ISBN: 978-3-031-06381-7
eBook Packages: Computer ScienceComputer Science (R0)