Skip to main content
SpringerLink
Log in

A Preliminary Investigation on a Multimodal Controller and Freehand Based Interaction in Virtual Reality

  • Conference paper
  • First Online:
Book cover Augmented Reality, Virtual Reality, and Computer Graphics (AVR 2021)

Part of the book series: Lecture Notes in Computer Science ((LNIP,volume 12980))

Abstract

In the last years, the synergy between VR and HCI has dramatically increased the user’s feeling of immersion within virtual scenes, improving VR applications’ user experience and usability. Two main aspects have emerged with the evolution of these technologies concerning immersion in the 3D scene: locomotion within the 3D scene and interaction with the components of the 3D scene. Locomotion with classical freehand approaches based on hand tracking can be stressful for the user due to the need to keep the hand still in a specific position for a long time to activate the locomotion gesture. Likewise, using a classic Head Mounted Display (HMD) controller for the interaction with the 3D scene components could be unnatural for the user, using the hand to pinch and grab the 3D objects. This paper proposes a multimodal approach, mixing the Leap Motion and 6-DOF controller to navigate and interact with the 3D scene to reduce the locomotion gesture stress based on the hand tracking and increase the immersion and interaction feeling using freehand to interact with the 3D scene objects.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 109.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 139.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Virtual Reality Applications: Guidelines to Design Natural User Interface, International Design Engineering Technical Conferences and Computers and Information in Engineering Conference, vol. Volume 1B: 38th Computers and Information in Engineering Conference (2018). https://doi.org/10.1115/DETC2018-85867, v01BT02A029

  2. Bastian, M., Heymann, S., Jacomy, M.: Gephi: an open source software for exploring and manipulating networks (2009). http://www.aaai.org/ocs/index.php/ICWSM/09/paper/view/154

  3. Caggianese, G., Capece, N., Erra, U., Gallo, L., Rinaldi, M.: Freehand-steering locomotion techniques for immersive virtual environments: a comparative evaluation. Int. J. Hum. Comput. Interact. 36(18), 1734–1755 (2020). https://doi.org/10.1080/10447318.2020.1785151

  4. Capece, N., Erra, U., Grippa, J.: GraphVR: a virtual reality tool for the exploration of graphs with HTC Vive system. In: 2018 22nd International Conference Information Visualisation (IV), pp. 448–453 (2018)

    Google Scholar 

  5. Capece, N., Erra, U., Gruosso, M., Anastasio, M.: Archaeo puzzle: an educational game using natural user interface for historical artifacts. In: Spagnuolo, M., Melero, F.J. (eds.) Eurographics Workshop on Graphics and Cultural Heritage. The Eurographics Association (2020)

    Google Scholar 

  6. Chen, D.L., Balakrishnan, R., Grossman, T.: Disambiguation techniques for freehand object manipulations in virtual reality. In: 2020 IEEE Conference on Virtual Reality and 3D User Interfaces (VR), pp. 285–292 (2020)

    Google Scholar 

  7. De Chiara, R., Di Santo, V., Erra, U., Scarano, V.: Real positioning in virtual environments using game engines, vol. 1, pp. 203–208 (2007), https://www.scopus.com/inward/record.uri?eid=2-s2.0-84878192833&partnerID=40&md5=83a6d6543f0358d6bd23a9882c42a65f, cited By 12

  8. Falcao, C., Lemos, A.C., Soares, M.: Evaluation of natural user interface: a usability study based on the leap motion device. Procedia Manuf. 3, 5490–5495 (2015), https://www.sciencedirect.com/science/article/pii/S2351978915006988, 6th International Conference on Applied Human Factors and Ergonomics (AHFE 2015) and the Affiliated Conferences, AHFE 2015

  9. Gruosso, M., Capece, N., Erra, U., Angiolillo, F.: A preliminary investigation into a deep learning implementation for hand tracking on mobile devices. In: 2020 IEEE International Conference on Artificial Intelligence and Virtual Reality (AIVR), pp. 380–385 (2020)

    Google Scholar 

  10. Hillmann, C.: Comparing the Gear VR, Oculus Go, and Oculus Quest, pp. 141–167. Apress, Berkeley, CA (2019). https://doi.org/10.1007/978-1-4842-4360-2_5

  11. Hincapié-Ramos, J.D., Guo, X., Moghadasian, P., Irani, P.: Consumed endurance: a metric to quantify arm fatigue of mid-air interactions, pp. 1063–1072. CHI 2014. Association for Computing Machinery, New York, NY, USA (2014). https://doi.org/10.1145/2556288.2557130

  12. Hooks, K., Ferguson, W., Morillo, P., Cruz-Neira, C.: Evaluating the user experience of omnidirectional VR walking simulators. Entertainment Comput. 34, 100352 (2020). https://www.sciencedirect.com/science/article/pii/S1875952119301284

  13. Iqbal, H., Latif, S., Yan, Y., Yu, C., Shi, Y.: Reducing arm fatigue in virtual reality by introducing 3D-spatial offset. IEEE Access 9, 64085–64104 (2021)

    Article  Google Scholar 

  14. Jacomy, M., Venturini, T., Heymann, S., Bastian, M.: ForceAtlas2, a continuous graph layout algorithm for handy network visualization designed for the Gephi software. PLOS ONE 9(6), 1–12 (2014). https://doi.org/10.1371/journal.pone.0098679

  15. Jang, S., Stuerzlinger, W., Ambike, S., Ramani, K.: Modeling cumulative arm fatigue in mid-air interaction based on perceived exertion and kinetics of arm motion. In: Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems, pp. 3328–3339. CHI 2017. Association for Computing Machinery, New York, NY, USA (2017). https://doi.org/10.1145/3025453.3025523

  16. Lee, P.W., Wang, H.Y., Tung, Y.C., Lin, J.W., Valstar, A.: Transection: hand-based interaction for playing a game within a virtual reality game. In: Proceedings of the 33rd Annual ACM Conference Extended Abstracts on Human Factors in Computing Systems, pp. 73–76. CHI EA 2015. Association for Computing Machinery, New York, NY, USA (2015). https://doi.org/10.1145/2702613.2728655

  17. Li, Y., Huang, J., Tian, F., Wang, H.A., Dai, G.Z.: Gesture interaction in virtual reality. Virtual Reality Intell. Hardware 1(1), 84–112 (2019). https://www.sciencedirect.com/science/article/pii/S2096579619300075

  18. McMahan, R.P., Lai, C., Pal, S.K.: Interaction fidelity: the uncanny valley of virtual reality interactions. In: Lackey, S., Shumaker, R. (eds.) VAMR 2016. LNCS, vol. 9740, pp. 59–70. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-39907-2_6

    Chapter  Google Scholar 

  19. Metsis, V., Smith, K.S., Gobert, D.: Integration of virtual reality with an omnidirectional treadmill system for multi-directional balance skills intervention. In: 2017 International Symposium on Wearable Robotics and Rehabilitation (WeRob), pp. 1–2 (2017)

    Google Scholar 

  20. Peer, A., Ponto, K.: Evaluating perceived distance measures in room-scale spaces using consumer-grade head mounted displays. In: 2017 IEEE Symposium on 3D User Interfaces (3DUI), pp. 83–86, March 2017

    Google Scholar 

  21. Pfeuffer, K., Mayer, B., Mardanbegi, D., Gellersen, H.: Gaze + pinch interaction in virtual reality. In: Proceedings of the 5th Symposium on Spatial User Interaction, pp. 99–108. SUI 2017. Association for Computing Machinery, New York, NY, USA (2017). https://doi.org/10.1145/3131277.3132180

  22. Piumsomboon, T., Lee, G., Lindeman, R.W., Billinghurst, M.: Exploring natural eye-gaze-based interaction for immersive virtual reality. In: 2017 IEEE Symposium on 3D User Interfaces (3DUI), pp. 36–39 (2017)

    Google Scholar 

  23. Pyo, S.H., Lee, H.S., Phu, B.M., Park, S.J., Yoon, J.W.: Development of an fast-omnidirectional treadmill (F-ODT) for immersive locomotion interface. In: 2018 IEEE International Conference on Robotics and Automation (ICRA), pp. 760–766 (2018)

    Google Scholar 

  24. Roth, S.D.: Ray casting for modeling solids. Comput. Graph. Image Process. 18(2), 109–144 (1982). https://www.sciencedirect.com/science/article/pii/0146664X82901691

  25. Sarupuri, B., Chipana, M.L., Lindeman, R.W.: Trigger walking: A low-fatigue travel technique for immersive virtual reality. In: 2017 IEEE Symposium on 3D User Interfaces (3DUI). pp. 227–228 (2017)

    Google Scholar 

  26. Satriadi, K.A., Ens, B., Cordeil, M., Jenny, B., Czauderna, T., Willett, W.: Augmented reality map navigation with freehand gestures. In: 2019 IEEE Conference on Virtual Reality and 3D User Interfaces (VR), pp. 593–603 (2019)

    Google Scholar 

  27. Szabó, B.K.: Interaction in an immersive virtual reality application. In: 2019 10th IEEE International Conference on Cognitive Infocommunications (CogInfoCom), pp. 35–40 (2019)

    Google Scholar 

  28. Voigt-Antons, J.N., Kojic, T., Ali, D., Möller, S.: Influence of hand tracking as a way of interaction in virtual reality on user experience. In: 2020 Twelfth International Conference on Quality of Multimedia Experience (QoMEX), pp. 1–4 (2020)

    Google Scholar 

  29. Wiedemann, D.P., Passmore, P.J., Moar, M.: An experiment design: investigating VR locomotion & virtual object interaction mechanics (2017)

    Google Scholar 

  30. Zhang, F., Chu, S., Pan, R., Ji, N., Xi, L.: Double hand-gesture interaction for walk-through in VR environment. In: 2017 IEEE/ACIS 16th International Conference on Computer and Information Science (ICIS), pp. 539–544 (2017)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Dipartimento di Matematica, Informatica ed Economia, Università degli Studi della Basilicata, Potenza, Italy

    Nicola Capece, Monica Gruosso, Ugo Erra, Rosario Catena & Gilda Manfredi

Authors
  1. Nicola Capece
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Monica Gruosso
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Ugo Erra
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Rosario Catena
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Gilda Manfredi
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Nicola Capece .

Editor information

Editors and Affiliations

  1. University of Salento, Lecce, Italy

    Lucio Tommaso De Paolis

  2. University of Naples Federico II, Naples, Italy

    Pasquale Arpaia

  3. University of Paris-Sud, Orsay, France

    Patrick Bourdot

Rights and permissions

Reprints and permissions

Copyright information

© 2021 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Capece, N., Gruosso, M., Erra, U., Catena, R., Manfredi, G. (2021). A Preliminary Investigation on a Multimodal Controller and Freehand Based Interaction in Virtual Reality. In: De Paolis, L.T., Arpaia, P., Bourdot, P. (eds) Augmented Reality, Virtual Reality, and Computer Graphics. AVR 2021. Lecture Notes in Computer Science(), vol 12980. Springer, Cham. https://doi.org/10.1007/978-3-030-87595-4_5

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-87595-4_5

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-87594-7

  • Online ISBN: 978-3-030-87595-4

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation