Skip to main content
SpringerLink
Log in

Operating Virtual Panels with Hand Gestures in Immersive VR Games

Experiences with the Leap Motion Controller

  • Conference paper
  • First Online:
Book cover Augmented Reality, Virtual Reality, and Computer Graphics (AVR 2017)

Part of the book series: Lecture Notes in Computer Science ((LNIP,volume 10324))

Abstract

Portable depth-sensing cameras allow users to control interfaces using hand gestures at a short range from the camera. These technologies are being combined with virtual reality (VR) headsets to produce immersive VR experiences that respond more naturally to user actions. In this research, we explore gesture-based interaction in immersive VR games by using the Unity game engine, the LeapMotion sensor, a laptop, a smartphone, and the Freefly VR headset. By avoiding Android deployment, this novel setup allowed for fast prototyping and testing of different ideas for immersive VR interaction, at an affordable cost. We implemented a system that allows users to play a game in a virtual world and compared placements of the leap motion sensor on the desk and on the headset. In this experimental setup, users interacted with a numeric dial panel and then played a Tetris game inside the VR environment by pressing the buttons of a virtual panel. The results suggest that, although the tracking quality of the Leap Motion sensor was rather limited when used in the head-mounted setup for pointing and selection tasks, its performance was much better in the desk-mounted setup, providing a novel platform for research and rapid application development.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 39.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Adhikarla, V.K., Sodnik, J., Szolgay, P., Jakus, G.: Exploring direct 3d interaction for full horizontal parallax light field displays using leap motion controller. Sensors 15(4), 8642–8663 (2015)

    Article  Google Scholar 

  2. Arthur, K.W.: Effects of field of view on performance with head-mounted displays. Ph.D. thesis, University of North Carolina at Chapel Hill (2000)

    Google Scholar 

  3. Bachmann, D., Weichert, F., Rinkenauer, G.: Evaluation of the leap motion controller as a new contact-free pointing device. Sensors 15(1), 214 (2015). http://www.mdpi.com/1424-8220/15/1/214

    Article  Google Scholar 

  4. Barret, J., Krueger, H.: Performance effects of reduced proprioceptive feedback on touch typists and casual users in a typing task. Behav. Inf. Technol. 13(6), 373–381 (1994). http://dx.doi.org/10.1080/01449299408914618

    Article  Google Scholar 

  5. Beattie, N., Horan, B., McKenzie, S.: Taking the leap with the oculus HMD and CAD - plucking at thin air? In: Proceedings of The 1st International Design Technology Conference, DESTECH2015, vol. 20, pp. 149–154. Elsevier (2015). http://www.sciencedirect.com/science/article/pii/S2212017315002029

  6. Besacier, G., Tournet, J., Goyal, N., Cento, F., Scott, S.D.: Object and arm shadows: Visual feedback for cross device transfer. In: CHI 2014 Extended Abstracts on Human Factors in Computing Systems, pp. 463–466, CHI EA 2014. ACM Press, New York (2014) http://doi.acm.org/10.1145/2559206.2574832

  7. Clark, T.: How Palmer Luckey created Oculus rift, November 2014. http://www.smithsonianmag.com/innovation/how-palmer-luckey-created-oculus-rift-180953049/?no-ist/

  8. Edward, B.: Unraveling the enigma of Nintendo’s virtual boy, 20 years later, August 2015. http://www.fastcompany.com/3050016/unraveling-the-enigma-of-nintendos-virtual-boy-20-years-later/

  9. FreeflyVR: Freefly VR: how does it work, the complete guide (2014). https://www.freeflyvr.com/freefly-vr-how-it-works/

  10. Google Inc.: Google cardboard Google VR (2016). https://www.google.com/get/cardboard/

  11. Greene, E.: Augmenting visual feedback using sensory substitution. Master’s thesis, University of Waterloo (2011). https://uwspace.uwaterloo.ca/bitstream/handle/10012/6161/Greene_Eugene.pdf

  12. Guna, J., Jakus, G., Poganik, M., Tomai, S., Sodnik, J.: An analysis of the precision and reliability of the leap motion sensor and its suitability for static and dynamic tracking. Sensors 14(2), 3702 (2014). http://www.mdpi.com/1424-8220/14/2/3702

    Article  Google Scholar 

  13. Higuchi, M., Komuro, T.: [Paper] robust finger tracking for gesture control of mobile devices using contour and interior information of a finger. ITE Trans. Media Technol. Appl. 1(3), 226–236 (2013)

    Article  Google Scholar 

  14. Hilfert, T., König, M.: Low-cost virtual reality environment for engineering and construction. Vis. Eng. 4(1), 1–18 (2016). http://dx.doi.org/10.1186/s40327-015-0031-5

    Article  Google Scholar 

  15. HTC Corporation: Vive - Home (2016). https://www.htcvive.com/ca/

  16. Hu, H.H., Gooch, A.A., Thompson, W.B., Smits, B.E., Rieser, J.J., Shirley, P.: Visual cues for imminent object contact in realistic virtual environment. In: Proceedings of the Conference on Visualization 2000, pp. 179–185, VIS 2000. IEEE Computer Society Press, Los Alamitos (2000). http://dl.acm.org/citation.cfm?id=375213.375238

  17. Intel Corp.: Intel RealSense Technology. http://www.intel.com/content/www/us/en/architecture-and-technology/realsense-shortrange.html (2016)

  18. Leap Motion Inc.: How Does the Leap Motion Controller Work? (2016). http://blog.leapmotion.com/hardware-to-software-how-does-the-leap-motion-controller-work/

  19. Leap Motion Inc: Leap motion developers (2016). https://developer.leapmotion.com/unity/

  20. Noobtuts.com: Unity 2D Tetris Tutorial (2012). http://noobtuts.com/unity/2d-tetris-game/

  21. Occipital, Inc.: Structure Sensor - 3D scanning, augmented reality, and more for mobile devices (2016). http://structure.io/

  22. OculusVR: Oculus (2016). https://www.oculus.com/en-us/

  23. Samsung Corporation: Samsung gear VR - the official Samsung galaxy site (2016). http://www.samsung.com/global/galaxy/wearables/gear-vr/

  24. Sharp, T., Keskin, C., Robertson, D., Taylor, J., Shotton, J., Kim, D., Rhemann, C., Leichter, I., Vinnikov, A., Wei, Y., Freedman, D., Kohli, P., Krupka, E., Fitzgibbon, A., Izadi, S.: Accurate, robust, and flexible real-time hand tracking. In: Proceedings of CHI, the 33rd Annual ACM Conference on Human Factors in Computing Systems. ACM Press, April 2015. http://research.microsoft.com/apps/pubs/default.aspx?id=238453

  25. Sony Interactive Entertainment: PlayStation VR (2016). https://www.playstation.com/en-ca/explore/playstation-vr/

  26. Splashtop Inc.: Top-performing remote desktop and remote support (2016). http://www.splashtop.com/

  27. Weichert, F., Bachmann, D., Rudak, B., Fisseler, D.: Analysis of the accuracy and robustness of the leap motion controller. Sensors 13(5), 6380 (2013). http://www.mdpi.com/1424-8220/13/5/6380

    Article  Google Scholar 

  28. Welch, G., Bishop, G.: Scaat: Incremental tracking with incomplete information. In: Proceedings of the 24th annual conference on Computer graphics and interactive techniques, pp. 333–344. ACM Press/Addison-Wesley Publishing Co. (1997)

    Google Scholar 

  29. Whitwam, R.: How Steve Jobs killed the stylus and made smartphones usable, October 2011. http://www.extremetech.com/computing/98923-how-steve-jobs-killed-the-stylus-and-made-smartphones-usable/

Download references

Author information

Authors and Affiliations

  1. Department of Computer Science, Memorial University of Newfoundland, St Johns, NL, A1B 3X7, Canada

    Yin Zhang & Oscar Meruvia-Pastor

Authors
  1. Yin Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Oscar Meruvia-Pastor
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Oscar Meruvia-Pastor .

Editor information

Editors and Affiliations

  1. University of Salento, Lecce, Italy

    Lucio Tommaso De Paolis

  2. University of Paris-Sud, Orsay, France

    Patrick Bourdot

  3. University of Salento, Lecce, Italy

    Antonio Mongelli

Rights and permissions

Reprints and permissions

Copyright information

© 2017 Springer International Publishing AG

About this paper

Cite this paper

Zhang, Y., Meruvia-Pastor, O. (2017). Operating Virtual Panels with Hand Gestures in Immersive VR Games. In: De Paolis, L., Bourdot, P., Mongelli, A. (eds) Augmented Reality, Virtual Reality, and Computer Graphics. AVR 2017. Lecture Notes in Computer Science(), vol 10324. Springer, Cham. https://doi.org/10.1007/978-3-319-60922-5_24

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-60922-5_24

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-60921-8

  • Online ISBN: 978-3-319-60922-5

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation