Skip to main content

Abstract

Manual assembly task analysis is essential for optimizing work instructions, improving tasks, and scheduling assembly lines in the context of Industry 5.0’s emphasis on human-centric, sustainable, and resilient manufacturing processes. The current paper outlines a comprehensive approach for data preparation for AI-assisted video analysis, aiming to simplify manual assembly task analysis, alleviate the workload of assembly operators and time setting experts, and advance Industry 5.0 principles. The paper focuses on setting up processes for recording videos of assembly tasks and converting the operator movements into skeleton models for subsequent analysis. Landmark points extracted from these models provide a numerical basis for task analysis. This data preparation process prepares the ground for future machine learning-based time setting prediction, considering companies’ unique time settings. The paper also addresses the ethical implications of video recording and data anonymization. Future work will delve into machine learning applications for time setting prediction and task-to-landmark correlations.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Bragança, S., Costa, E.: An application of the lean production tool standard work. Jurnal Teknologi (Sci. Eng.) 76, 47–53 (2015). https://doi.org/10.11113/jt.v76.3659

  2. Torres, Y., Nadeau, S., Landau, K.: Classification and quantification of human error in manufacturing: a case study in complex manual assembly. Appl. Sci. 11, 749 (2021). https://doi.org/10.3390/app11020749

    Article  Google Scholar 

  3. Ani, M.C., Abdul Azid, I.: Solving the production bottleneck through minimizing the waste of motion for manual assembly processes. Progress Eng. Technol. II, 185–197 (2020). https://doi.org/10.1007/978-3-030-46036-5_17

  4. Ani, M.C., Abdul Hamid, S.: Analysis and reduction of the waste in the work process using time study analysis: a case study. Appl. Mech. Mater. 660, 971–975 (2014). https://doi.org/10.4028/www.scientific.net/AMM.660.971

  5. Judt, D., Lawson, C., Lockett, H.: Experimental investigation into aircraft system manual assembly performance under varying structural component orientations. Proc. Inst. Mech. Eng. Part B J. Eng. Manuf. 234, 840–855 (2020). https://doi.org/10.1177/0954405419883047

  6. Ding, Z., Hon, B.: Constraints analysis and evaluation of manual assembly. CIRP Ann. 62, 1–4 (2013). https://doi.org/10.1016/j.cirp.2013.03.003

  7. Cohen, Y., Naseraldin, H., Chaudhuri, A., Pilati, F.: Assembly systems in Industry 4.0 era: a road map to understand assembly 4.0. Int. J. Adv. Manuf. Technol. 105(9), 4037–4054 (2019). https://doi.org/10.1007/s00170-019-04203-1

    Article  Google Scholar 

  8. Lu, Y., et al.: Outlook on human-centric manufacturing towards industry 5.0. J. Manuf. Syst. 62, 612–627 (2022). https://doi.org/10.1016/j.jmsy.2022.02.001

  9. Nahavandi, S.: Industry 5.0-a human-centric solution. Sustainability 11, 4371 (2019). https://doi.org/10.3390/su11164371

  10. Chander, B., Pal, S., De, D., Buyya, R.: Artificial intelligence-based internet of things for industry 5.0. Artif. Intell.-Based Internet Things Syst., 3–45 (2022). https://doi.org/10.1007/978-3-030-87059-1_1

  11. John, K., Adarsh, S., Pattali, V.: Workers to super workers: a brief discussion on important technologies for industry 5.0 manufacturing systems. AIP Conf. Proc. 2311, 070025 (2020). https://doi.org/10.1063/5.0034521

  12. Favi, C., Germani, M., Marconi, M.: A 4M approach for a comprehensive analysis and improvement of manual assembly lines. Procedia Manuf. 11, 1510–1518 (2017). https://doi.org/10.1016/j.promfg.2017.07.283

  13. Hedman, R., Almström, P.: A state of the art system for managing time data in manual assembly. Int. J. Comput. Integr. Manuf. 30, 1060–1071 (2017). https://doi.org/10.1080/0951192X.2017.1305501

  14. Pimminger, S., et al.: Assembly task analysis using the General Assembly Task Model (GATM) on the shop floor. Procedia CIRP. 93, 1109–1114 (2020). https://doi.org/10.1016/j.procir.2020.04.007

  15. Elnekave, M., Gilad, I.: Rapid video-based analysis system for advanced work measurement. Int. J. Prod. Res. 44, 271–290 (2006). https://doi.org/10.1080/00207540500160920

  16. Paquet, V., Mathiassen, S., Dempsey, P.: Video-based ergonomic job analysis. Prof. Saf. 51, 27–35 (2006)

    Google Scholar 

  17. Peltokorpi, J., Niemi, E.: Analysis of the effects of group size and learning on manual assembly performance. Procedia Manuf. 39, 964–973 (2019). https://doi.org/10.1016/j.promfg.2020.02.001

  18. Kazmierczak, K., Mathiassen, S., Neumann, P., Winkel, J.: Observer reliability of industrial activity analysis based on video recordings. Int. J. Ind. Ergon. 36, 275–282 (2006). https://doi.org/10.1016/j.ergon.2005.12.006

  19. Aganian, D., Stephan, B., Eisenbach, M., Stretz, C., Gross, H.: ATTACH dataset: annotated two-handed assembly actions for human action understanding. ArXiv Preprint ArXiv:2304.08210 (2023). http://dx.doi.org/10.48550/arXiv.2304.08210

  20. Yang, C., Li, W., Hsu, S.: Skeleton-based hand gesture recognition for assembly line operation. In: 2020 International Conference On Advanced Robotics And Intelligent Systems (ARIS), pp. 1–6 (2020). https://doi.org/10.1109/ARIS50834.2020.9205781

  21. Zhang, D., Wu, Y., Guo, M., Chen, Y.: Deep learning methods for 3D human pose estimation under different supervision paradigms: a survey. Electronics 10, 2267 (2021). https://doi.org/10.3390/electronics10182267

  22. Wang, J., et al.: Deep 3D human pose estimation: a review. Comput. Vis. Image Underst. 210, 103225 (2021). https://doi.org/10.1016/j.cviu.2021.103225

  23. Lan, G., Wu, Y., Hu, F., Hao, Q.: Vision-based human pose estimation via deep learning: a survey. IEEE Trans. Hum.-Mach. Syst. (2022). https://doi.org/10.1109/THMS.2022.3219242

  24. Tian, Y., Li, H., Cui, H., Chen, J.: Construction motion data library: an integrated motion dataset for on-site activity recognition. Sci. Data 9, 726 (2022). https://doi.org/10.1038/s41597-022-01841-1

  25. Lin, P., Chen, Y., Chen, W., Lee, Y.: Automatic real-time occupational posture evaluation and select corresponding ergonomic assessments. Sci. Rep. 12, 2139 (2022). https://doi.org/10.1038/s41598-022-05812-9

  26. Chen, C., Zhao, X., Wang, J., Li, D., Guan, Y., Hong, J.: Dynamic graph convolutional network for assembly behavior recognition based on attention mechanism and multi-scale feature fusion. Sci. Rep. 12, 7394 (2022). https://doi.org/10.1038/s41598-022-11206-8

  27. Tan, H., Zhu, H., Lim, J., Tan, C.: A comprehensive survey of procedural video datasets. Comput. Vis. Image Understand. 202, 103107 (2021). https://doi.org/10.1016/j.cviu.2020.103107

  28. Cicirelli, G., et al.: The HA4M dataset: multi-modal monitoring of an assembly task for human action recognition in manufacturing. Sci. Data 9, 745 (2022). https://doi.org/10.1038/s41597-022-01843-z

  29. Martınez, G.: OpenPose: Whole-Body Pose Estimation. Carnegie Mellon University Pittsburgh, PA, USA (2019)

    Google Scholar 

  30. Lugaresi, C., et al.: MediaPipe: a framework for perceiving and processing reality. In: Third Workshop on Computer Vision for AR/VR at IEEE Computer Vision And Pattern Recognition (CVPR) (2019). https://doi.org/10.48550/arXiv.1906.08172

Download references

Acknowledgment

The authors would like to acknowledge the support of the Swedish Innovation Agency (Vinnova). This study is part of the Time Data Management Automation for Manual Assembly (TIMEBLY) project.

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Yongkuk Jeong .

Editor information

Editors and Affiliations

Rights and permissions

Reprints and permissions

Copyright information

© 2023 IFIP International Federation for Information Processing

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Jeong, Y., Wiktorsson, M., Park, D., Gans, J., Svensson, L. (2023). Data Preparation for AI-Assisted Video Analysis in Manual Assembly Task: A Step Towards Industry 5.0. In: Alfnes, E., Romsdal, A., Strandhagen, J.O., von Cieminski, G., Romero, D. (eds) Advances in Production Management Systems. Production Management Systems for Responsible Manufacturing, Service, and Logistics Futures. APMS 2023. IFIP Advances in Information and Communication Technology, vol 691. Springer, Cham. https://doi.org/10.1007/978-3-031-43670-3_43

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-43670-3_43

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-43669-7

  • Online ISBN: 978-3-031-43670-3

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics