MRTouch: Adding Touch Input to Head-Mounted Mixed Reality | IEEE Journals & Magazine | IEEE Xplore

MRTouch: Adding Touch Input to Head-Mounted Mixed Reality


Abstract:

We present MRTouch, a novel multitouch input solution for head-mounted mixed reality systems. Our system enables users to reach out and directly manipulate virtual interf...Show More

Abstract:

We present MRTouch, a novel multitouch input solution for head-mounted mixed reality systems. Our system enables users to reach out and directly manipulate virtual interfaces affixed to surfaces in their environment, as though they were touchscreens. Touch input offers precise, tactile and comfortable user input, and naturally complements existing popular modalities, such as voice and hand gesture. Our research prototype combines both depth and infrared camera streams together with real-time detection and tracking of surface planes to enable robust finger-tracking even when both the hand and head are in motion. Our technique is implemented on a commercial Microsoft HoloLens without requiring any additional hardware nor any user or environmental calibration. Through our performance evaluation, we demonstrate high input accuracy with an average positional error of 5.4 mm and 95% button size of 16 mm, across 17 participants, 2 surface orientations and 4 surface materials. Finally, we demonstrate the potential of our technique to enable on-world touch interactions through 5 example applications.
Published in: IEEE Transactions on Visualization and Computer Graphics ( Volume: 24, Issue: 4, April 2018)
Page(s): 1653 - 1660
Date of Publication: 18 January 2018

ISSN Information:

PubMed ID: 29543181

References

References is not available for this document.