Abstract
The ability to interact with virtual objects using gestures would allow users to improve their experience in Mixed Reality (MR) environments, especially when they use AR headsets. Today, MR head-mounted displays like the HoloLens integrate hand gesture based interaction allowing users to take actions in MR environments. However, the proposed interactions remain limited. In this paper, we propose to combine a Leap Motion Controller (LMC) with a HoloLens in order to improve gesture interaction with virtual objects. Two main issues are presented: an interactive calibration procedure for the coupled HoloLens-LMC device and an intuitive hand-based interaction approach using LMC data in the HoloLens environment. A set of first experiments was carried out to evaluate the accuracy and the usability of the proposed approach.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Aruanno, B., Garzotto, F., Rodriguez, M.C.: Hololens-based mixed reality experiences for subjects with alzheimer’s disease. In: Proceedings of the 12th Biannual Conference on Italian SIGCHI Chapter, CHItaly (2017), pp. 15:1–15:9, (2017)
Benko, B., Jota, R., Wilson, A.: Miragetable: freehand interaction on a projected augmented reality tabletop. In: Proceedings of the 2012 ACM Annual Conference on Human Factors in Computing Systems, pp. 199–208 (2012)
Blaha, J., Gupta, M.: Diplopia: a virtual reality game designed to help amblyopics. In: 2014 IEEE Virtual Reality (VR), pp. 163–164 (2014)
Carceroni, R.L., Brown, C.M.: Numerical methods for model-based pose recovery. Technical Report 659, University of Rochester, Computer Science Department (1997)
Ens, B., Quigley, A., Yeo, H.-S., Irani, P., Piumsomboon, T., Billinghurst, M.: Counterpoint: exploring mixed-scale gesture interaction for AR applications. In: Proceedings of the 2018, CHI Conference on Human Factors in Computing Systems (2018)
Garon, M., Boulet, P., Doironz, J., Beaulieu, L., Lalonde, J.: Real-time high resolution 3d data on the hololens. In: 2016 IEEE International Symposium on Mixed and Augmented Reality (ISMAR-Adjunct), pp. 189–191 (2016)
Khademi, M., Mousavi Hondori, H., McKenzie, A., Dodakian, L., Lopes, C.V., Cramer, S.C.: Free-hand interaction with leap motion controller for stroke rehabilitation. In: Proceedings of the Extended Abstracts of the 32nd Annual ACM Conference on Human Factors in Computing Systems, CHI EA 2014, pp. 1663–1668 (2014)
Köhler, N.: Integration of a leap motion controller with the hololens to enable improved gesture interactions. Master’s thesis, Aalto University (2017)
Lyons, D.M.: System and method for permitting three-dimensional navigation through a virtual reality environment using camera-based gesture inputs (2001)
Umeyama, S.: Least-squares estimation of transformation parameters between two point patterns. IEEE Trans. Patt. Anal. Mach. Intell. 13(4), 376–380 (1991)
Weichert, F., Bachmann, D., Rudak, B., Fisseler, D.: Analysis of the accuracy and robustness of the leap motion controller. Sensors 13(5), 6380–6393 (2013)
Zhang, Z.: Iterative point matching for registration of free-form curves and surfaces. Int. J. Comput. Vision 13(2), 119–152 (1994)
Author information
Authors and Affiliations
Corresponding authors
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2020 Springer Nature Switzerland AG
About this paper
Cite this paper
Ababsa, F., He, J., Chardonnet, JR. (2020). Combining HoloLens and Leap-Motion for Free Hand-Based 3D Interaction in MR Environments. In: De Paolis, L., Bourdot, P. (eds) Augmented Reality, Virtual Reality, and Computer Graphics. AVR 2020. Lecture Notes in Computer Science(), vol 12242. Springer, Cham. https://doi.org/10.1007/978-3-030-58465-8_24
Download citation
DOI: https://doi.org/10.1007/978-3-030-58465-8_24
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-58464-1
Online ISBN: 978-3-030-58465-8
eBook Packages: Computer ScienceComputer Science (R0)