Skip to main content
Springer Nature Link
Log in

A Literature-Based Perspective on Human-Centered Design and Evaluation of Interfaces for Virtual Reality in Robotics

  • Conference paper
  • First Online:
Human-Friendly Robotics 2022 (HFR 2022)

Part of the book series: Springer Proceedings in Advanced Robotics ((SPAR,volume 26))

Included in the following conference series:

Abstract

Over the past decade, the use of virtual reality applications is increasing in the field of robotics, which is set to increase even further with the advent of “Metaverse”. Metaverse is considered as an “embodied internet” which will seamlessly integrate machines, robots, and computer applications including those in virtual and augmented reality. In this work, we review the current state of research and discuss the underlying problems with the current design and evaluation methods for the state-of-the-art virtual reality interfaces employed in the field of robotics. To overcome these issues, we present our perspective on the methods for design and evaluation of such interfaces. Our perspective takes into account human-centered design methods along with the functional measures of evaluating the performance of the interfaces. This interdisciplinary perspective emphasizes rigorously designed empirical experiments that require close collaborations among engineers, designers, human-computer interaction researchers and cognitive scientists.

Chenxu Hao and Anany Dwivedi authors contributed equally to this work.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Similar content being viewed by others

References

  1. Stephenson, N.: Snow Crash: A Aovel. Spectra (2003)

    Google Scholar 

  2. Hackl, C.: Defining the metaverse today. [Online]. Available: https://www.forbes.com/sites/cathyhackl/2021/05/02/defining-the-metaverse-today/?sh=433cc19a6448

  3. Iannuzzi, J.B, Faulds, Z.: Top companies chasing the metaverse. [Online]. Available: https://www.thestreet.com/technology/companies-making-money-metaverse

  4. Newton, C. Mark in the metaverse. [Online]. Available: https://www.theverge.com/22588022/mark-zuckerberg-facebook-ceo-metaverse-interview

  5. Alnajjar, M., Abutabikh, A., Issa, M., Debeljak, M., Cikajlo, I..: Development of 3d exergame for upper limbs rehabilitation using leap motion controller and unity. In: 2020 International Conference on Assistive and Rehabilitation Technologies (iCareTech), pp. 24–29. IEEE (2020)

    Google Scholar 

  6. Trombetta, M., Henrique, P.P.B., Brum, M.R., Colussi, E.L., De Marchi, A.C.B., Rieder, R.: Motion rehab ave 3d: A vr-based exergame for post-stroke rehabilitation. Comput. Methods Programs Biomed. 151, 15–20 (2017)

    Article  Google Scholar 

  7. Bui, J., Luauté, J., Farnè, A.: Enhancing upper limb rehabilitation of stroke patients with virtual reality: a mini review. Front. Virtual Reality, 146 (2021)

    Google Scholar 

  8. Zgonnikov, A., Abbink, D., Markkula, G.: Should i stay or should i go? evidence accumulation drives decision making in human drivers (2020)

    Google Scholar 

  9. Makhataeva, Z., Varol, H.A.: Augmented reality for robotics: a review. Robotics 9(2), 21 (2020)

    Article  Google Scholar 

  10. Schmidt, M. Köppinger, K., Fan, C., Kowalewski, K.-F., Schmidt, L., Vey, J., Proctor, T., Probst, P., Bintintan, V., Müller-Stich, B.-P., et al.: Virtual reality simulation in robot-assisted surgery: meta-analysis of skill transfer and predictability of skill. BJS open 5(2), zraa066 (2021)

    Google Scholar 

  11. Ambron, E., Miller, A., Kuchenbecker, K.J., Buxbaum, L.J., Coslett, H.B.: Immersive low-cost virtual reality treatment for phantom limb pain: evidence from two cases. Front. Neurol. 9, 67 (2018)

    Article  Google Scholar 

  12. Boschmann, A., Neuhaus, D., Vogt, S., Kaltschmidt, C., Platzner, M., Dosen, S.: Immersive augmented reality system for the training of pattern classification control with a myoelectric prosthesis. J. Neuroeng. Rehabil. 18(1), 1–15 (2021)

    Article  Google Scholar 

  13. Stadler, S., Kain, K., Giuliani, M., Mirnig, N., Stollnberger, G., Tscheligi, M.: Augmented reality for industrial robot programmers: workload analysis for task-based, augmented reality-supported robot control. In: 25th IEEE International Symposium on Robot and Human Interactive Communication (RO-MAN), pp. 179–184 . IEEE (2016)

    Google Scholar 

  14. Liu, O., Rakita, D., Mutlu, B., Gleicher, M.: Understanding human-robot interaction in virtual reality. In: 26th IEEE International Symposium on Robot and Human Interactive Communication (RO-MAN), pp. 751–757. IEEE (2017)

    Google Scholar 

  15. De Pace, F., Gorjup, G., Bai, H., Sanna, A., Liarokapis, M., Billinghurst, M.: Leveraging enhanced virtual reality methods and environments for efficient, intuitive, and immersive teleoperation of robots. In: 2021 IEEE International Conference on Robotics and Automation (ICRA), pp. 12 967–12 973. IEEE (2021)

    Google Scholar 

  16. Zhou, R., Wu, Y., Sareen, H.: Hextouch: a wearable haptic robot for complementary interactions to companion agents in virtual reality. In: SIGGRAPH Asia. Emerging Technologies, pp. 1–2 (2020)

    Google Scholar 

  17. Sutcliffe, A.G., Poullis, C., Gregoriades, A., Katsouri, I., Tzanavari, A., Herakleous, K.: Reflecting on the design process for virtual reality applications. Int. J. Hum. Comput. Interact. 35(2), 168–179 (2019)

    Article  Google Scholar 

  18. Beckerle, P., Castellini, C., Lenggenhager, B.: Robotic interfaces for cognitive psychology and embodiment research: a research roadmap. Wiley Interdisc. Rev.: Cogn. Sci. 10(2), e1486 (2019)

    Google Scholar 

  19. Dünser, A., Grasset, R., Seichter, H., Billinghurst, M.: Applying HCI principles to AR systems design. In: 2nd International Workshop at the IEEE Virtual Reality 2007 Conference, p. 2007. North Carolina, USA, Charlotte (2007)

    Google Scholar 

  20. Boletsis, C., Cedergren, J.E., Kongsvik, S.: Hci research in virtual reality: a discussion of problem-solving. In: International Conference on Interfaces and Human Computer Interaction. IHCI 2017, Portugal, 21–23 July 2017 (2017)

    Google Scholar 

  21. Kim, Y.M., Rhiu, I., Yun, M.H.: A systematic review of a virtual reality system from the perspective of user experience. Int. J. Hum. Comput. Interact. 36(10), 893–910 (2020)

    Article  Google Scholar 

  22. Oulasvirta, A., Jokinen, J.P., Howes, A.: Computational rationality as a theory of interaction. In: CHI Conference on Human Factors in Computing Systems, pp. 1–14 (2022)

    Google Scholar 

  23. Lewis, R.L., Howes, A., Singh, S.: Computational rationality: Linking mechanism and behavior through bounded utility maximization. Top. Cogn. Sci. 6(2), 279–311 (2014)

    Article  Google Scholar 

  24. Gershman, S.J., Horvitz, E.J., Tenenbaum, J.B.: Computational rationality: a converging paradigm for intelligence in brains, minds, and machines. Science 349(6245), 273–278 (2015)

    Article  MathSciNet  MATH  Google Scholar 

  25. Weistroffer, V., Paljic, A., Callebert, L., Fuchs, P.: A methodology to assess the acceptability of human-robot collaboration using virtual reality. In: Proceedings of the 19th ACM Symposium on Virtual Reality Software and Technology, pp. 39–48 (2013)

    Google Scholar 

  26. Badia, S.B.I., Silva, P.A., Branco, D., Pinto, A., Carvalho, C., Menezes, P., Almeida, J., Pilacinski, A.: Virtual reality for safe testing and development in collaborative robotics: challenges and perspectives. Electronics 11(11), 726 (2022)

    Google Scholar 

  27. Madhavan, K., Kolcun, J.P.G., Chieng, L.O., Wang, M.Y.: Augmented-reality integrated robotics in neurosurgery: are we there yet? Neurosurg. Focus 42(5), E3 (2017)

    Article  Google Scholar 

  28. Luciano, C.J., Banerjee, P.P., Sorenson, J.M., Foley, K.T., Ansari, S.A., Rizzi, S., Germanwala, A.V., Kranzler, L., Chittiboina, P., Roitberg, B.Z.: Percutaneous spinal fixation simulation with virtual reality and haptics. Neurosurgery 72(suppl_1), A89–A96 (2013)

    Google Scholar 

  29. Yudkowsky, R., Luciano, C., Banerjee, P., Schwartz, A., Alaraj, A., Lemole, G.M., Jr., Charbel, F., Smith, K., Rizzi, S., Byrne, R., et al.: Practice on an augmented reality/haptic simulator and library of virtual brains improves residents’ ability to perform a ventriculostomy. Simul. Healthc. 8(1), 25–31 (2013)

    Article  Google Scholar 

  30. Alaraj, A., Luciano, C.J., Bailey, D.P., Elsenousi, A., Roitberg, B.Z., Bernardo, A., Banerjee, P.P., Charbel, F.T.: Virtual reality cerebral aneurysm clipping simulation with real-time haptic feedback. Operative Neurosurg. 11(1), 52–58 (2015)

    Article  Google Scholar 

  31. Pessaux, P., Diana, M., Soler, L., Piardi, T., Mutter, D., Marescaux, J.: Towards cybernetic surgery: robotic and augmented reality-assisted liver segmentectomy. Langenbeck’s Arch. Surg. 400(3), 381–385 (2015)

    Article  Google Scholar 

  32. Makhataeva, Z., Zhakatayev, A., Varol, H.A.: Safety aura visualization for variable impedance actuated robot. In: IEEE/SICE International Symposium on System Integration (SII), 805–810. IEEE (2019)

    Google Scholar 

  33. Saponas, Z., Tan, D.S., Morris, D., Balakrishnan, R., Turner, J., Landay, J.A.: Enabling always-available input with muscle-computer interfaces. In: Proceedings of the 22nd annual ACM symposium on User interface software and technology, pp. 167–176 (2009)

    Google Scholar 

  34. Chapman, J., Dwivedi, A., Liarokapis, M.: A wearable, open-source, lightweight forcemyography armband: on intuitive, robust muscle-machine interfaces. In: 2021 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pp. 4138–4143. IEEE (2021)

    Google Scholar 

  35. Sutcliffe, A.G., Kaur, K.D.: Evaluating the usability of virtual reality user interfaces. Behav. Inf. Technol. 19(6), 415–426 (2000)

    Article  Google Scholar 

  36. Bowman, D.A., McMahan, R.P.: Virtual reality: how much immersion is enough? Computer 40(7), 36–43 (2007)

    Article  Google Scholar 

  37. Nostadt, N., Abbink, D.A., Christ, O., Beckerle, P.: Embodiment, presence, and their intersections: teleoperation and beyond. ACM Transa. Hum. Robot Inter. (THRI) 9(4), 1–19 (2020)

    Article  Google Scholar 

  38. Morra, L., Lamberti, F., Pratticó, F.G., La Rosa, S., Montuschi, P.: Building trust in autonomous vehicles: role of virtual reality driving simulators in HMI design. IEEE Trans. Veh. Technol. 68(10), 9438–9450 (2019)

    Article  Google Scholar 

  39. Moore, B.A., Urakami, J.: The impact of the physical and social embodiment of voice user interfaces on user distraction. Int. J. Hum. Comput. Stud. 102784 (2022)

    Google Scholar 

  40. Oviatt, S.: Human-centered design meets cognitive load theory: designing interfaces that help people think. In: Proceedings of the 14th ACM international conference on Multimedia, pp. 871–880 (2006)

    Google Scholar 

  41. Lindlbauer, D., Feit, A.M., Hilliges, O.: Context-aware online adaptation of mixed reality interfaces. In: Proceedings of the 32nd Annual ACM Symposium on User Interface Software and Technology, pp. 147–160 (2019)

    Google Scholar 

  42. Schürmann, T., Mohler, B.J., Peters, J., Beckerle, P.: How cognitive models of human body experience might push robotics. Front. Neurorobotics 13, 14 (2019)

    Article  Google Scholar 

  43. Rizzo, A.A., Kim, G.J., Yeh, S.-C., Thiebaux, M., Hwang, J., Buckwalter, J.G.: Development of a benchmarking scenario for testing 3d user interface devices and interaction methods. In Proceedings of the 11th International Conference on Human Computer Interaction. Las Vegas, Nevada (2005)

    Google Scholar 

  44. Isaacs, E., Walendowski, A.: Designing from both sides of the screen: How designers and engineers can collaborate to build cooperative technology. Sams Publishing (2002)

    Google Scholar 

  45. Oulasvirta, A, Hornbæk, K.: Hci research as problem-solving. In: Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems, pp. 4956–4967 (2016)

    Google Scholar 

  46. Sosa, R., Montiel, M., Sandoval, E.B., Mohan, R.E.: et al.: Robot ergonomics: Towards human-centred and robot-inclusive design. In: DS 92: Proceedings of the DESIGN 2018 15th International Design Conference, pp. 2323–2334 (2018)

    Google Scholar 

  47. Beckerle, P., Salvietti, G., Unal, R., Prattichizzo, D., Rossi, S., Castellini, C., Hirche, S., Endo, S., Amor, H.B., Ciocarlie, M., et al.: A human-robot interaction perspective on assistive and rehabilitation robotics. Front. Neurorobotics 11, 24 (2017)

    Article  Google Scholar 

  48. Beckerle, P., Christ, O., Schürmann, T., Vogt, J., von Stryk, O., Rinderknecht, S.: A human-machine-centered design method for (powered) lower limb prosthetics. Robot. Auton. Syst. 95, 1–12 (2017)

    Article  Google Scholar 

  49. Gorjup, G., Dwivedi, A., Elangovan, N., Liarokapis, M.: An intuitive, affordances oriented telemanipulation framework for a dual robot arm hand system: On the execution of bimanual tasks. In: 2019 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pp. 3611–3616. IEEE (2019)

    Google Scholar 

  50. Dwivedi, A., Shieff, D., Turner, A., Gorjup, G., Kwon, Y., Liarokapis, M.: A shared control framework for robotic telemanipulation combining electromyography based motion estimation and compliance control. In: 2021 IEEE International Conference on Robotics and Automation (ICRA), pp. 9467–9473. IEEE (2021)

    Google Scholar 

  51. Bianchi, M., Valenza, G., Greco, A., Nardelli, M., Battaglia, E., Bicchi, A., Scilingo, E.P.: Towards a novel generation of haptic and robotic interfaces: integrating affective physiology in human-robot interaction. In: 25th IEEE International Symposium on Robot and Human Interactive Communication (RO-MAN), pp. 125–131. IEEE (2016)

    Google Scholar 

  52. Aguilera, F., Alarcón, R.A., Guerrero, L.A., Collazos, A.: A cognitive model of user interaction as a guideline for designing novel interfaces. In: IFIP World Computer Congress, TC 2, pp. 62–76. Springer (2006)

    Google Scholar 

  53. Gonzalez-Franco, M., Lanier, J.: Model of illusions and virtual reality. Front. Psychol. 8, 1125 (2017)

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Chair of Autonomous Systems and Mechatronics, Department of Electrical Engineering, Friedrich-Alexander-Universität Erlangen-Nürnberg, Erlangen, Germany

    Chenxu Hao, Anany Dwivedi & Philipp Beckerle

  2. Department of Artificial Intelligence in Biomedical Engineering, Friedrich-Alexander-Universität Erlangen-Nürnberg, Erlangen, Germany

    Philipp Beckerle

Authors
  1. Chenxu Hao
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Anany Dwivedi
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Philipp Beckerle
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Chenxu Hao .

Editor information

Editors and Affiliations

  1. Faculty of Mechanical, Maritime and Materials Engineering, Delft University of Technology, Delft, The Netherlands

    Pablo Borja

  2. Faculty of Mechanical, Maritime and Materials Engineering, Delft University of Technology, Delft, The Netherlands

    Cosimo Della Santina

  3. Faculty of Mechanical, Maritime and Materials Engineering, Delft University of Technology, Delft, The Netherlands

    Luka Peternel

  4. Department of Mechanical Engineering, Eindhoven University of Technology, Eindhoven, The Netherlands

    Elena Torta

Rights and permissions

Reprints and permissions

Copyright information

© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Hao, C., Dwivedi, A., Beckerle, P. (2023). A Literature-Based Perspective on Human-Centered Design and Evaluation of Interfaces for Virtual Reality in Robotics. In: Borja, P., Della Santina, C., Peternel, L., Torta, E. (eds) Human-Friendly Robotics 2022. HFR 2022. Springer Proceedings in Advanced Robotics, vol 26. Springer, Cham. https://doi.org/10.1007/978-3-031-22731-8_1

Download citation

Publish with us

Policies and ethics