Skip to main content
Springer Nature Link
Log in

A Motion Analysis System for Pointing and Calling Considering Safety Checks for Soldering Work

  • Conference paper
  • First Online:
Advances on P2P, Parallel, Grid, Cloud and Internet Computing (3PGCIC 2023)

Part of the book series: Lecture Notes on Data Engineering and Communications Technologies ((LNDECT,volume 189))

  • 332 Accesses

Abstract

In Japan, pointing and calling are carried out to improve the safety, reduce human errors and prevent accidents. Via pointing and calling concept the safety is improved by pointing to the work object or tool and voicing the situation in order to predict the accidents during the work. In factories, soldering is part of the handicraft industry and one of the career options for people with disabilities. During soldering skill, it is necessary to memorize and repeat the same work, which takes a long time. Also, there are human errors caused by accidents and lack of safety checks or experience. In addition, ensuring worker safety requires continuous monitoring of worker motion, which is a significant burden for instructors. In this paper, we propose a motion analysis system for pointing and calling. The proposed system uses a depth camera to capture images of workers during pointing and calling. Also, the proposed system considers beforehand safety checks for soldering operations to prevent accidents and injuries. The experimental results of the pointing orientation show that the proposed system is effective for safety checks and can support beginners and people with disabilities to continue soldering work safely.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Similar content being viewed by others

References

  1. Fu, D., et al.: Research on safety helmet detection algorithm of power workers based on improved YOLOv5. J. Phys. Conf. Ser. 2171(1), 12006 (2022)

    Article  Google Scholar 

  2. Hyeonju, L., et al.: Virtual reality metaverse system supplementing remote education methods: based on aircraft maintenance simulation. Appl. Sci. 12(5), 2667 (2021)

    Google Scholar 

  3. Xiang, X., et al.: A four-stage product appearance defect detection method with small samples. IEEE Access 10, 83740–83754 (2022)

    Article  Google Scholar 

  4. Yasunaga, T., et al.: Object detection and pose estimation approaches for soldering danger detection. In: Proceedings of the IEEE 10-th Global Conference on Consumer Electronics, pp. 776–777 (2021)

    Google Scholar 

  5. Yasunaga, T., et al.: A soldering motion analysis system for danger detection considering object detection and attitude estimation. In: Proceedings of the 10-th International Conference on Emerging Internet, Data & Web Technologies, pp. 301–307 (2022)

    Google Scholar 

  6. Toyoshima, K., et al.: Analysis of a soldering motion for dozing state and attention posture detection. In: Barolli, L. (ed.) Advances on P2P, Parallel, Grid, Cloud and Internet Computing. 3PGCIC 2022. LNNS, vol. 571, pp. 146–153 . Springer, Cham (2022). https://doi.org/10.1007/978-3-031-19945-5_14

  7. Toyoshima, K., et al.: Proposal of a haptics and LSTM based soldering motion analysis system. In: Proceedings of the IEEE 10-th Global Conference on Consumer Electronics, pp. 1–2 (2021)

    Google Scholar 

  8. Toyoshima, K., et al.: Design and implementation of a haptics based soldering education system. In: Barolli, L. (eds.) Innovative Mobile and Internet Services in Ubiquitous Computing. IMIS 2022. LNNS, vol. 496, pp. 54–64. Springer, Cham (2022). https://doi.org/10.1007/978-3-031-08819-3_6

  9. Toyoshima, K., et al.: Experimental results of a haptics based soldering education system: a comparison study of RNN and LSTM for detection of dangerous movements. In: Barolli, L., Miwa, H. (eds.) Advances in Intelligent Networking and Collaborative Systems. INCoS 2022. LNNS, vol. 527, pp. 212–223. Springer, Cham (2022). https://doi.org/10.1007/978-3-031-14627-5_20

  10. Toyoshima, K., et al.: A soldering motion analysis system for monitoring whole body of people with developmental disabilities. In: Barolli, L. (ed.) Innovative Mobile and Internet Services in Ubiquitous Computing. IMIS 2023. LNDECT, vol. 177, pp. 38–46. Springer, Cham (2023). https://doi.org/10.1007/978-3-031-35836-4_5

  11. Obukata, R., et al.: Design and evaluation of an ambient intelligence testbed for improving quality of life. Int. J. Space-Based Situated Comput. 7(1), 8–15 (2017)

    Article  Google Scholar 

  12. Oda, T., Ueda, C., Ozaki, R., Katayama, K.: Design of a deep Q-network based simulation system for actuation decision in ambient intelligence. In: Barolli, L., Takizawa, M., Xhafa, F., Enokido, T. (eds.) WAINA 2019. AISC, vol. 927, pp. 362–370. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-15035-8_34

    Chapter  Google Scholar 

  13. Obukata, R., et al.: Performance evaluation of an AmI testbed for improving QoL: evaluation using clustering approach considering distributed concurrent processing. In: Proceedings of the IEEE AINA-2017, pp. 271–275 (2017)

    Google Scholar 

  14. Yamada, M., et al.: Evaluation of an IoT-based e-learning testbed: performance of OLSR protocol in a NLoS environment and mean-shift clustering approach considering electroencephalogram data. Int. J. Web Inf. Syst. 13(1), 2–13 (2017)

    Article  Google Scholar 

  15. Hirota, Y., Oda, T., Saito, N., Hirata, A., Hirota, M., Katatama, K.: Proposal and experimental results of an ambient intelligence for training on soldering iron holding. In: Barolli, L., Takizawa, M., Enokido, T., Chen, H.-C., Matsuo, K. (eds.) BWCCA 2020. LNNS, vol. 159, pp. 444–453. Springer, Cham (2021). https://doi.org/10.1007/978-3-030-61108-8_44

    Chapter  Google Scholar 

  16. Hirota, Y., et al.: Proposal and experimental results of a DNN based real-time recognition method for ohsone style fingerspelling in static characters environment. In: Proceedings of the IEEE 9-th Global Conference on Consumer Electronics, pp. 476–477 (2020)

    Google Scholar 

  17. Oda, T., et al.: Design and implementation of an IoT-based e-learning testbed. Int. J. Web Grid Serv. 13(2), 228–241 (2017)

    Article  Google Scholar 

  18. Toshev, A., Szegedy, C.: Deeppose: human pose estimation via deep neural networks. In: Proceedings of the 27-th IEEE/CVF Conference on Computer Vision and Pattern Recognition (IEEE/CVF CVPR-2014), pp. 1653–1660 (2014)

    Google Scholar 

  19. Haralick, R., et al.: Pose estimation from corresponding point data. IEEE Trans. Syst. 19(6), 1426–1446 (1989)

    Google Scholar 

  20. Fang, H., et al.: RMPE: regional multi-person pose estimation. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2334–2343 (2017)

    Google Scholar 

  21. Xiao, B., Wu, H., Wei, Y.: Simple baselines for human pose estimation and tracking. In: Ferrari, V., Hebert, M., Sminchisescu, C., Weiss, Y. (eds.) ECCV 2018. LNCS, vol. 11210, pp. 472–487. Springer, Cham (2018). https://doi.org/10.1007/978-3-030-01231-1_29

    Chapter  Google Scholar 

  22. Martinez, J., et al.: A simple yet effective baseline for 3d human pose estimation. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2640–2649 (2017)

    Google Scholar 

  23. Lugaresi, C., et al.: MediaPipe: A Framework for Building Perception Pipelines, arXiv preprint arXiv:1906.08172 (2019)

  24. Micilotta, A.S., Ong, E.-J., Bowden, R.: Real-time upper body detection and 3D pose estimation in monoscopic images. In: Leonardis, A., Bischof, H., Pinz, A. (eds.) ECCV 2006. LNCS, vol. 3953, pp. 139–150. Springer, Heidelberg (2006). https://doi.org/10.1007/11744078_11

    Chapter  Google Scholar 

  25. Andriyanov, N., et al.: Intelligent system for estimation of the spatial position of apples based on YOLOv3 and real sense depth camera D415. Symmetry 14(1), 1–14 (2022)

    Google Scholar 

  26. Soukupova, T., et al.: Real-time eye blink detection using facial landmarks. In: Proceedings of the 21st Computer Vision Winter Workshop, Rimske Toplice, Slovenia (2016)

    Google Scholar 

  27. Zhang, F., et al.: MediaPipe Hands: On-device Real-time Hand Tracking, arXiv preprint arXiv:2006.10214 (2020)

  28. Shin, J., et al.: American sign language alphabet recognition by extracting feature from hand pose estimation. Sensors 21(17), 5856 (2021)

    Article  Google Scholar 

  29. Erol, A., et al.: Vision-based hand pose estimation: a review. Comput. Vis. Image Underst. 108(1–2), 52–73 (2007)

    Google Scholar 

Download references

Acknowledgement

This work was supported by JSPS KAKENHI Grant Number JP20K19793.

Author information

Authors and Affiliations

  1. Graduate School of Engineering, Okayama University of Science (OUS), 1-1 Ridaicho, Kita-ku, Okayama, 700–0005, Japan

    Kyohei Toyoshima, Chihiro Yukawa & Yuki Nagai

  2. Department of Information and Computer Engineering, Okayama University of Science (OUS), 1-1 Ridaicho, Kita-ku, Okayama, 700–0005, Japan

    Yuma Yamashita & Tetsuya Oda

  3. Department of Informention and Communication Engineering, Fukuoka Insitute of Technology, 3-30-1 Wajiro-Higashi-ku, Fukuoka, 811-0295, Japan

    Leonard Barolli

Authors
  1. Kyohei Toyoshima
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Chihiro Yukawa
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Yuki Nagai
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Yuma Yamashita
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Tetsuya Oda
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Leonard Barolli
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Tetsuya Oda .

Editor information

Editors and Affiliations

  1. Faculty of Information Engineering, Fukuoka Institute of Technology, Fukuoka, Japan

    Leonard Barolli

Rights and permissions

Reprints and permissions

Copyright information

© 2024 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Toyoshima, K., Yukawa, C., Nagai, Y., Yamashita, Y., Oda, T., Barolli, L. (2024). A Motion Analysis System for Pointing and Calling Considering Safety Checks for Soldering Work. In: Barolli, L. (eds) Advances on P2P, Parallel, Grid, Cloud and Internet Computing . 3PGCIC 2023. Lecture Notes on Data Engineering and Communications Technologies, vol 189. Springer, Cham. https://doi.org/10.1007/978-3-031-46970-1_9

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-46970-1_9

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-46969-5

  • Online ISBN: 978-3-031-46970-1

  • eBook Packages: EngineeringEngineering (R0)

Publish with us

Policies and ethics