Skip to main content
Springer Nature Link
Log in

Experimental Validation of an Interface for a Human-Robot Interaction Within a Collaborative Task

  • Conference paper
  • First Online:
Interactive Collaborative Robotics (ICR 2023)

Part of the book series: Lecture Notes in Computer Science ((LNAI,volume 14214))

Included in the following conference series:

  • 626 Accesses

Abstract

This paper presents a prototype of a non-contact UR robot based Virtual Control (UR-VC) system for collaborative robots of the UR family, which is based on computer vision techniques and a virtual interaction interface. A control method involved specific hand movements within a field of view of a web camera, which was connected to a laptop with the running UR-VC system. We present the UR-VC system and the results of an experimental validation. To inquire if the UR-VC system is comfortable and user-friendly for an interaction with collaborative robots and to study opportunities for a further development and expansion directions of the system, we designed a test case that simulates a joint product assembly in a collaborative workspace. The constructed collaborative workspace included the UR3e robot, the laptop with the running UR-VC system and assembly parts for a collaborative task. 24 participants were involved in the experiments. First, the participants learned how to control the robot using the UR-VC system. After the training, all participants successfully controlled the robot using the proposed interface for performing the collaborative task. Participants’ experience of operating the robot was analyzed via surveys, their unconstrained comments and video recordings of the experiments.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Similar content being viewed by others

References

  1. Chuengwa, T.Y., Swanepoel, J.A., Kurien, A.M., Kanakana-Katumba, M.G., Djouani, K.: Research perspectives in collaborative assembly: a review. Robotics 12(2), 37 (2023)

    Article  Google Scholar 

  2. Bisen, A., Payal, H.: Collaborative robots for industrial tasks: a review. Mater. Today: Proc. 52(Part 3), 500–504 (2022)

    Google Scholar 

  3. Rodríguez-Guerra, D., Sorrosal, G., Cabanes, I., Calleja, C.: Human-robot interaction review: challenges and solutions for modern industrial environments. IEEE Access 9, 108557–108578 (2021)

    Article  Google Scholar 

  4. Villani, V., Pini, F., Leali, F., Secchi, C.: Survey on human–robot collaboration in industrial settings: safety, intuitive interfaces, and applications. Mechatronics 55, 248–266 (2018)

    Article  Google Scholar 

  5. Galin, R., Meshcheryakov, R.: Review on human–robot interaction during collaboration in a shared workspace. In: Ronzhin, A., Rigoll, G., Meshcheryakov, R. (eds.) ICR 2019. LNCS, vol. 11659, pp. 63–74. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-26118-4_7

  6. Galin, R.R., Shiroky, A.A., Magid, E.A., Mescheriakov, R.V., Mamchenko, M.V.: Effective functioning of a mixed heterogeneous team in a collaborative robotic system. Inform. Autom. 20(6), 1224–1253 (2021)

    Google Scholar 

  7. Mavridis, N.: A review of verbal and non-verbal human–robot interactive communication. Robot. Auton. Syst. 63(1), 22–35 (2015)

    Article  MathSciNet  Google Scholar 

  8. Gustavsson, P., Holm, M., Syberfeldt, A., Wang, L.: Human-robot collaboration – towards new metrics for selection of communication technologies. Proc. CIRP 72, 123–128 (2018)

    Article  Google Scholar 

  9. Sultanov, R., Sulaiman, S., Li, H., Meshcheryakov, R., Magid, E.: A review on collaborative robots in industrial and service sectors. In: International Siberian Conference on Control and Communications (SIBCON), Tomsk, Russian Federation, pp. 1–7 (2022)

    Google Scholar 

  10. Dobrokvashina, A., Sulaiman, S., Zagirov, A., Chebotareva, E., Kuo-Hsien, H., Magid, E.: Human robot interaction in collaborative manufacturing scenarios: prospective cases. In: International Siberian Conference on Control and Communications (SIBCON), Tomsk, Russian Federation, pp. 1–6 (2022)

    Google Scholar 

  11. Cherubini, A., Navarro-Alarcon, D.: Sensor-based control for collaborative robots: fundamentals, challenges, and opportunities. Front. Neurorobot. 14, 113 (2021)

    Article  Google Scholar 

  12. Hameed, A., Ordys, A., Możaryn, J., Sibilska-Mroziewicz, A.: Control system design and methods for collaborative robots: review. Appl. Sci. 13(1), 675 (2023)

    Article  Google Scholar 

  13. Costa, G.M., Petry, M.R., Moreira, A.P.: Augmented reality for human–robot collaboration and cooperation in industrial applications: a systematic literature review. Sensors 22(7), 2725 (2022)

    Article  Google Scholar 

  14. Lindblom, J., Alenljung, B., Billing, E.: Evaluating the user experience of human–robot interaction. In: Jost, C., et al. (eds.) Human-Robot Interaction. SSBN, vol. 12, pp. 231–256. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-42307-0_9

  15. De Simone, V., Di Pasquale, V., Giubileo, V., Miranda, S.: Human-robot collaboration: an analysis of worker’s performance. Proc. Comput. Sci. 200, 1540–1549 (2022)

    Article  Google Scholar 

  16. Gualtieri, L., Rauch, E., Vidoni, R., Matt, D.T.: Safety, ergonomics and efficiency in human-robot collaborative assembly: design guidelines and requirements. Proc. CIRP 91, 367–372 (2020)

    Article  Google Scholar 

  17. Shukla, D., Erkent, Ö., Piater, J.: Learning semantics of gestural instructions for human-robot collaboration. Front. Neurorobot. 12, 7 (2018)

    Article  Google Scholar 

  18. Quek, F.K.H.: Eyes in the interface. Image Vis. Comput. 13, 511–525 (1995)

    Article  Google Scholar 

  19. Simão, M., Gibaru, O., Neto, P.: Online recognition of incomplete gesture data to interface collaborative robots. IEEE Trans. Ind. Electron. 66, 9372–9382 (2019)

    Article  Google Scholar 

  20. Vysocky, A., Poštulka, T., Chlebek, J., Kot, T., Maslowski, J., Grushko, S.: Hand gesture interface for robot path definition in collaborative applications: implementation and comparative study. Sensors 23, 4219 (2023)

    Article  Google Scholar 

  21. Hand Gesture Recognition Using Mediapipe. https://github.com/Kazuhito00/hand-gesture-recognition-using-mediapipe. Accessed 01 Apr 2023

  22. Mustafin, M., Chebotareva, E., Li, H., Martínez-García, E.A., Magid, E.: Features of interaction between a human and a gestures-controlled collaborative robot in an assembly task: pilot experiments. In: Proceedings of International Conference on Artificial Life and Robotics, pp. 158–162 (2023)

    Google Scholar 

Download references

Acknowledgment

The reported study was funded by the Russian Science Foundation (RSF) and the Cabinet of Ministers of the Republic of Tatarstan according to the research project No. 22-21-20033.

Author information

Authors and Affiliations

  1. Intelligent Robotics Department, Institute of Information Technology and Intelligent Systems, Kazan Federal University, 35, Kremlyovskaya st., Kazan, 420111, Russia

    Maksim Mustafin, Elvira Chebotareva & Evgeni Magid

  2. Department of Instrument Science and Engineering, Shanghai Jiao Tong University, Minhang, Shanghai, 200240, China

    Hongbing Li

  3. Tikhonov Moscow Institute of Electronics and Mathematics, HSE-University, 34, Tallinn st., Moscow, 123458, Russia

    Evgeni Magid

Authors
  1. Maksim Mustafin
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Elvira Chebotareva
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Hongbing Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Evgeni Magid
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Maksim Mustafin .

Editor information

Editors and Affiliations

  1. St. Petersburg Federal Research Center of the Russian Academy of Sciences, St. Petersburg, Russia

    Andrey Ronzhin

  2. Institute of Control Systems of Ministry of Science and Education of the Republic of Azerbaijan, Baku, Azerbaijan

    Aminagha Sadigov

  3. V.A. Trapeznikov Institute of Control Sciences of the Russian Academy of Sciences, Moscow, Russia

    Roman Meshcheryakov

Rights and permissions

Reprints and permissions

Copyright information

© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Mustafin, M., Chebotareva, E., Li, H., Magid, E. (2023). Experimental Validation of an Interface for a Human-Robot Interaction Within a Collaborative Task. In: Ronzhin, A., Sadigov, A., Meshcheryakov, R. (eds) Interactive Collaborative Robotics. ICR 2023. Lecture Notes in Computer Science(), vol 14214. Springer, Cham. https://doi.org/10.1007/978-3-031-43111-1_3

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-43111-1_3

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-43110-4

  • Online ISBN: 978-3-031-43111-1

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics