Abstract
Manual assembly task analysis is essential for optimizing work instructions, improving tasks, and scheduling assembly lines in the context of Industry 5.0’s emphasis on human-centric, sustainable, and resilient manufacturing processes. The current paper outlines a comprehensive approach for data preparation for AI-assisted video analysis, aiming to simplify manual assembly task analysis, alleviate the workload of assembly operators and time setting experts, and advance Industry 5.0 principles. The paper focuses on setting up processes for recording videos of assembly tasks and converting the operator movements into skeleton models for subsequent analysis. Landmark points extracted from these models provide a numerical basis for task analysis. This data preparation process prepares the ground for future machine learning-based time setting prediction, considering companies’ unique time settings. The paper also addresses the ethical implications of video recording and data anonymization. Future work will delve into machine learning applications for time setting prediction and task-to-landmark correlations.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Bragança, S., Costa, E.: An application of the lean production tool standard work. Jurnal Teknologi (Sci. Eng.) 76, 47–53 (2015). https://doi.org/10.11113/jt.v76.3659
Torres, Y., Nadeau, S., Landau, K.: Classification and quantification of human error in manufacturing: a case study in complex manual assembly. Appl. Sci. 11, 749 (2021). https://doi.org/10.3390/app11020749
Ani, M.C., Abdul Azid, I.: Solving the production bottleneck through minimizing the waste of motion for manual assembly processes. Progress Eng. Technol. II, 185–197 (2020). https://doi.org/10.1007/978-3-030-46036-5_17
Ani, M.C., Abdul Hamid, S.: Analysis and reduction of the waste in the work process using time study analysis: a case study. Appl. Mech. Mater. 660, 971–975 (2014). https://doi.org/10.4028/www.scientific.net/AMM.660.971
Judt, D., Lawson, C., Lockett, H.: Experimental investigation into aircraft system manual assembly performance under varying structural component orientations. Proc. Inst. Mech. Eng. Part B J. Eng. Manuf. 234, 840–855 (2020). https://doi.org/10.1177/0954405419883047
Ding, Z., Hon, B.: Constraints analysis and evaluation of manual assembly. CIRP Ann. 62, 1–4 (2013). https://doi.org/10.1016/j.cirp.2013.03.003
Cohen, Y., Naseraldin, H., Chaudhuri, A., Pilati, F.: Assembly systems in Industry 4.0 era: a road map to understand assembly 4.0. Int. J. Adv. Manuf. Technol. 105(9), 4037–4054 (2019). https://doi.org/10.1007/s00170-019-04203-1
Lu, Y., et al.: Outlook on human-centric manufacturing towards industry 5.0. J. Manuf. Syst. 62, 612–627 (2022). https://doi.org/10.1016/j.jmsy.2022.02.001
Nahavandi, S.: Industry 5.0-a human-centric solution. Sustainability 11, 4371 (2019). https://doi.org/10.3390/su11164371
Chander, B., Pal, S., De, D., Buyya, R.: Artificial intelligence-based internet of things for industry 5.0. Artif. Intell.-Based Internet Things Syst., 3–45 (2022). https://doi.org/10.1007/978-3-030-87059-1_1
John, K., Adarsh, S., Pattali, V.: Workers to super workers: a brief discussion on important technologies for industry 5.0 manufacturing systems. AIP Conf. Proc. 2311, 070025 (2020). https://doi.org/10.1063/5.0034521
Favi, C., Germani, M., Marconi, M.: A 4M approach for a comprehensive analysis and improvement of manual assembly lines. Procedia Manuf. 11, 1510–1518 (2017). https://doi.org/10.1016/j.promfg.2017.07.283
Hedman, R., Almström, P.: A state of the art system for managing time data in manual assembly. Int. J. Comput. Integr. Manuf. 30, 1060–1071 (2017). https://doi.org/10.1080/0951192X.2017.1305501
Pimminger, S., et al.: Assembly task analysis using the General Assembly Task Model (GATM) on the shop floor. Procedia CIRP. 93, 1109–1114 (2020). https://doi.org/10.1016/j.procir.2020.04.007
Elnekave, M., Gilad, I.: Rapid video-based analysis system for advanced work measurement. Int. J. Prod. Res. 44, 271–290 (2006). https://doi.org/10.1080/00207540500160920
Paquet, V., Mathiassen, S., Dempsey, P.: Video-based ergonomic job analysis. Prof. Saf. 51, 27–35 (2006)
Peltokorpi, J., Niemi, E.: Analysis of the effects of group size and learning on manual assembly performance. Procedia Manuf. 39, 964–973 (2019). https://doi.org/10.1016/j.promfg.2020.02.001
Kazmierczak, K., Mathiassen, S., Neumann, P., Winkel, J.: Observer reliability of industrial activity analysis based on video recordings. Int. J. Ind. Ergon. 36, 275–282 (2006). https://doi.org/10.1016/j.ergon.2005.12.006
Aganian, D., Stephan, B., Eisenbach, M., Stretz, C., Gross, H.: ATTACH dataset: annotated two-handed assembly actions for human action understanding. ArXiv Preprint ArXiv:2304.08210 (2023). http://dx.doi.org/10.48550/arXiv.2304.08210
Yang, C., Li, W., Hsu, S.: Skeleton-based hand gesture recognition for assembly line operation. In: 2020 International Conference On Advanced Robotics And Intelligent Systems (ARIS), pp. 1–6 (2020). https://doi.org/10.1109/ARIS50834.2020.9205781
Zhang, D., Wu, Y., Guo, M., Chen, Y.: Deep learning methods for 3D human pose estimation under different supervision paradigms: a survey. Electronics 10, 2267 (2021). https://doi.org/10.3390/electronics10182267
Wang, J., et al.: Deep 3D human pose estimation: a review. Comput. Vis. Image Underst. 210, 103225 (2021). https://doi.org/10.1016/j.cviu.2021.103225
Lan, G., Wu, Y., Hu, F., Hao, Q.: Vision-based human pose estimation via deep learning: a survey. IEEE Trans. Hum.-Mach. Syst. (2022). https://doi.org/10.1109/THMS.2022.3219242
Tian, Y., Li, H., Cui, H., Chen, J.: Construction motion data library: an integrated motion dataset for on-site activity recognition. Sci. Data 9, 726 (2022). https://doi.org/10.1038/s41597-022-01841-1
Lin, P., Chen, Y., Chen, W., Lee, Y.: Automatic real-time occupational posture evaluation and select corresponding ergonomic assessments. Sci. Rep. 12, 2139 (2022). https://doi.org/10.1038/s41598-022-05812-9
Chen, C., Zhao, X., Wang, J., Li, D., Guan, Y., Hong, J.: Dynamic graph convolutional network for assembly behavior recognition based on attention mechanism and multi-scale feature fusion. Sci. Rep. 12, 7394 (2022). https://doi.org/10.1038/s41598-022-11206-8
Tan, H., Zhu, H., Lim, J., Tan, C.: A comprehensive survey of procedural video datasets. Comput. Vis. Image Understand. 202, 103107 (2021). https://doi.org/10.1016/j.cviu.2020.103107
Cicirelli, G., et al.: The HA4M dataset: multi-modal monitoring of an assembly task for human action recognition in manufacturing. Sci. Data 9, 745 (2022). https://doi.org/10.1038/s41597-022-01843-z
Martınez, G.: OpenPose: Whole-Body Pose Estimation. Carnegie Mellon University Pittsburgh, PA, USA (2019)
Lugaresi, C., et al.: MediaPipe: a framework for perceiving and processing reality. In: Third Workshop on Computer Vision for AR/VR at IEEE Computer Vision And Pattern Recognition (CVPR) (2019). https://doi.org/10.48550/arXiv.1906.08172
Acknowledgment
The authors would like to acknowledge the support of the Swedish Innovation Agency (Vinnova). This study is part of the Time Data Management Automation for Manual Assembly (TIMEBLY) project.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2023 IFIP International Federation for Information Processing
About this paper
Cite this paper
Jeong, Y., Wiktorsson, M., Park, D., Gans, J., Svensson, L. (2023). Data Preparation for AI-Assisted Video Analysis in Manual Assembly Task: A Step Towards Industry 5.0. In: Alfnes, E., Romsdal, A., Strandhagen, J.O., von Cieminski, G., Romero, D. (eds) Advances in Production Management Systems. Production Management Systems for Responsible Manufacturing, Service, and Logistics Futures. APMS 2023. IFIP Advances in Information and Communication Technology, vol 691. Springer, Cham. https://doi.org/10.1007/978-3-031-43670-3_43
Download citation
DOI: https://doi.org/10.1007/978-3-031-43670-3_43
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-43669-7
Online ISBN: 978-3-031-43670-3
eBook Packages: Computer ScienceComputer Science (R0)