Abstract
Industrial Metaverse plays an increasingly important role in Intelligent Manufacturing Training, improving the user immersion as well as reducing the cost and risks in training. Action recognition technology can effectively detect human actions, help construct the industrial metaverse space of Human-in-the-loop, and empower the low-intrusion, personalized intelligent manufacturing training system. Intelligent Manufacturing Training-Worker Action Recognition and Evaluation (IMTware) is researched to improve the human-manchine interaction and intelligent guidance in industrial training. In terms of action recognition, IMTware extracts the skeletal point sequences of human practice training actions in the video via OpenPose network and inputs them into ST-GCN network for action recognition. In terms of action evaluation, a method combining Dynamic Time Warping (DTW) keyframe matching with cosine similarity solving is proposed to compute the action similarity between the skeletal point sequences and the standard skeletal point sequences, and evaluate them according to the similarity to give personalized correction suggestions. Subsequently, the performance of the system is validated using overhead crane command actions training in converter steelmaking as an example, achieving satisfactory results.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Zhou, Y., et al.: Research on intelligent manufacturing training system based on industrial metaverse. In: Hassan, F., Sunar, N., Mohd Basri, M.A., Mahmud, M.S.A., Ishak, M.H.I., Mohamed Ali, M.S. (eds.) AsiaSim 2023. CCIS, vol. 1911, pp. 28–43. Springer, Singapore (2024). https://doi.org/10.1007/978-981-99-7240-1_3
Zheng, Z., et al.: Industrial metaverse: connotation, features, technologies, applications and challenges. In: Fan, W., Zhang, L., Li, N., Song, X. (eds.) AsiaSim 2022. CCIS, vol. 1712, pp. 239–263. Springer, Singapore (2022). https://doi.org/10.1007/978-981-19-9198-1_19
Yan, S., Xiong, Y., Lin, D.: Spatial temporal graph convolutional networks for skeleton-based action recognition. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol. 32, no. 1 (2018)
Li, M., Chen, S., Chen, X., et al.: Actional-structural graph convolutional networks for skeleton-based action recognition. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3595–3603 (2019)
Chen, X.: Action evaluation system based on human three-dimensional posture. Zhejiang University (2018). (in Chinese)
Qian, C., Zhang, X., Tao, J., et al.: Design and research of Kinect-based rehabilitation training system. J. Jilin Univ. (Inf. Sci. Ed.) 38(01), 92–98 (2020). https://doi.org/10.19292/j.cnki.jdxxp.2020.01.013. (in Chinese)
Shi, L., Zhang, Y., Cheng, J., et al.: Two-stream adaptive graph convolutional networks for skeleton-based action recognition. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 12026–12035 (2019)
Shi, L., Zhang, Y., Cheng, J., et al.: Skeleton-based action recognition with directed graph neural networks. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 7912–7921 (2019)
Parmar, P., Tran Morris, B.: Learning to score Olympic events. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops, pp. 20–28 (2017)
Bruce, X.B., Liu, Y., Chan, K.C.C., et al.: Skeleton-based human action evaluation using graph convolutional network for monitoring Alzheimer’s progression. Pattern Recogn. 119, 108095 (2021)
Wang, J., Du, Z., Li, A., Wang, Y.: Assessing action quality via attentive spatio-temporal convolutional networks. In: Peng, Y., et al. (eds.) PRCV 2020, pp. 3–16. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-60639-8_1
Pan, J.H., Gao, J., Zheng, W.S.: Action assessment by joint relation graphs. In: Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 6331–6340 (2019)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2024 The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd.
About this paper
Cite this paper
Wu, G. et al. (2024). Research on Worker Action Recognition and Evaluation in Intelligent Manufacturing Training Based on Industrial Metaverse. In: Saito, S., Tanaka, S., Li, L., Takatori, S., Tamura, Y. (eds) Methods and Applications for Modeling and Simulation of Complex Systems. AsiaSim 2024. Communications in Computer and Information Science, vol 2170. Springer, Singapore. https://doi.org/10.1007/978-981-97-7225-4_28
Download citation
DOI: https://doi.org/10.1007/978-981-97-7225-4_28
Published:
Publisher Name: Springer, Singapore
Print ISBN: 978-981-97-7224-7
Online ISBN: 978-981-97-7225-4
eBook Packages: Computer ScienceComputer Science (R0)