research-article

Head-mounted display with mid-air tactile feedback

Authors:

Ismo Rakkolainen,

Poika Isokoski,

Karri PalovuoriAuthors Info & Claims

VRST '15: Proceedings of the 21st ACM Symposium on Virtual Reality Software and Technology

Pages 51 - 58

https://doi.org/10.1145/2821592.2821593

Published: 13 November 2015 Publication History

Abstract

Virtual and physical worlds are merging. Currently users of head-mounted displays cannot have unobtrusive tactile feedback while touching virtual objects. We present a mid-air tactile feedback system for head-mounted displays. Our prototype uses the focus of a modulated ultrasonic phased array for unobtrusive mid-air tactile feedback generation. The array and the hand position sensor are mounted on the front surface of a head-mounted virtual reality display. The presented system can enhance 3D user interfaces and virtual reality in a new way.

To evaluate the tactile feedback together with visuals on an Oculus Rift VR headset, we had 13 participants do a simple virtual keypad tapping task with and without tactile feedback. The results indicate that while the measured speed and accuracy differed only a little, the subjects were nearly unanimous in that they preferred to use the tactile feedback. The "raw" NASA TLX questionnaires conducted after use revealed that the participants felt slightly less mental, physical and temporal demand with the tactile feedback. The participants' self-assessment of their performance was also higher with the tactile feedback.

References

[1]

Billinghurst, M., Bowskill, J., Dyer, N. and Morphett, J. 1998. Spatial Information Displays on a Wearable Computer. IEEE Computer Graphics and Applications 18, 6, 24--31.

Digital Library

[2]

Bowman, D., Kruijff, E., LaViola, J. and Poupyrev, I. 2004. 3D User Interfaces: Theory and Practice. Addison-Wesley, Boston, MA, USA.

Digital Library

[3]

Carter, T., Seah, S., Long, B., Drinkwater, B. and Subramanian, S. 2013. UltraHaptics: Multi-Point Mid-Air Haptic Feedback for Touch Surfaces. In Proceedings of the UIST 2013 Conference, ACM, 505--514.

Digital Library

[4]

Foulsham, T., Walker, E. and Kingtone, A. 2011. The where, what and when of gaze allocation in the lab and in the natural environment. Vision Research, 51, 1920--1931.

[5]

Freeman, E., Brewster, S. and Lantz, V. 2014. Tactile Feedback for Above-Device Gesture Interfaces: Adding Touch to Touchless Interactions. In Proceedings of the ICMI 2014 Conference, ACM, 419--426.

Digital Library

[6]

Gupta, S., Morris, D., Patel, S. and Tan, D. 2013. AirWave: non-contact haptic feedback using air vortex rings. In Proceedings of the UbiComp 2013 Conference, ACM, 419--428.

Digital Library

[7]

Hart, S. G. 2006. NASA-Task Load Index (NASA-TLX); 20 Years Later. In Proceedings of the Human Factors and Ergonomics Society 50th Annual Meeting. HFES, 904--908.

[8]

Hasegawa, K. and Shinoda, H. 2013. Aerial display of vibrotactile sensation with high spatial-temporal resolution using large-aperture airborne ultrasound phased array. In Proceedings of the World Haptics Conference 2013, IEEE, 31--36.

[9]

Hoshi, T., Abe, D. and Shinoda, H. 2009. Adding Tactile Reaction to Hologram. In Proceedings of the RO-MAN 2009 Conference, IEEE, 7--11.

[10]

Inoue, S., Kobayashi-Kirschvink, K., Monnai, Y., Hasegawa, K., Makino, Y. and Shinoda, H. 2014. HORN: the hapt-optic reconstruction. In Proceedings of the SIGGRAPH 2014 Conference, Emerging Technologies, ACM, Article 11.

Digital Library

[11]

Iwamoto, T., Tatezono, M. and Shinoda, H. 2008. Non-contact method for producing tactile sensation using airborne ultrasound. In Proceedings of the EuroHaptics 2008 Conference, Springer-Verlag, 504--513.

Digital Library

[12]

Ku, J., Mraz, R., Baker, N., Zakzanis, K., and Lee, J. H. 2003. A Data Glove with Tactile Feedback for fMRI of Virtual Reality Experiments. CyberPsychology & Behavior. October 2003, 6(5), 497--508.

[13]

Lenhardt, M. 2008. Airborne ultrasonic standards for hearing protection. In Proceedings of the 9th International Congress on Noise as a Public Health Problem, ICBEN.

[14]

Long, B., Seah, S., Carter, T. and Subramanian, S. 2014. Rendering Volumetric Haptic Shapes in Mid-Air using Ultrasound. ACM Transactions on Graphics 33, 6, Article 181.

Digital Library

[15]

Monnai, Y., Hasegawa, K., Fujiwara, M., Yoshino, K., Inoue, S. and Shinoda, H. 2014. HaptoMime: mid-air haptic interaction with a floating virtual screen. In Proceedings of the UIST 2014 Conference, ACM, 663--667.

Digital Library

[16]

Palovuori, K., Rakkolainen, I. and Sand, A. 2014. Bidirectional Touch Interaction for Immaterial Displays. In Proceedings of the MindTrek 2014 Conference, ACM, 76--78.

Digital Library

[17]

Pont, S., Kappers, A. and Koenderink, J. 1997. Haptic curvature discrimination at several regions of the hand. Perception & Psychophysics 59, 8, 1225--1240.

[18]

Ryu, J., Jung, J., Park, G. and Choi, S. 2010. Psycho-physical Model for Vibrotactile Rendering in Mobile Devices. Presence 19, 4, 364--387.

Digital Library

[19]

Sand, A., Rakkolainen, I., Isokoski, P., Raisamo, R., and Palovuori, K. 2015. Lightweight Immaterial Particle Displays with Mid-Air Tactile Feedback. In Proceedings of the IEEE International Workshop on Haptic Audio-Visual Environments and Games 2015, IEEE.

[20]

Sodhi, R., Poupyrev, I., Glisson, M. and Israr, A. 2013. AIREAL: Interactive Tactile Experiences in Free Air. In Proceedings of the ACM SIGGRAPH 2013 Conference, ACM, Article 134.

Digital Library

[21]

Smagowska, B. and Pawlaczyk-Łuszczyńska, M. 2015. Effects of Ultrasonic Noise on the Human Body --- A Bibliographic Review. International Journal of Occupational Safety and Ergonomics, 19:2, 195--202.

[22]

Sutherland, I. 1965. The Ultimate Display. In Proceedings of the. IFIP Congress 1965, Vol. 2, 506--508.

[23]

Sutherland, I. 1968. A Head-mounted Three-dimensional Display. In Proceedings of the AFIPS Fall Joint Computer Conference 1968, Thompson Books, Vol. 3, 757--764.

Digital Library

[24]

Takahashi M. and Shinoda, H. 2010. Large aperture Airborne Ultrasound Tactile Display using distributed array units. In Proceedings of the SICE 2010 Conference, IEEE, 359--362.

[25]

Wilson, G., Carter, T., Subramanian, S. and Brewster, S. 2014. Perception of ultrasonic haptic feedback on the hand: localisation and apparent motion. In Proceedings of the CHI 2014 Conference, ACM, 1133--1142.

Digital Library

[26]

Yoshino, K., Hasegawa, K. and Shinoda, H. 2012. Measuring Visio-Tactile threshold for Visio-Tactile Projector. In Proceedings of the SICE 2012 Conference, IEEE, 1996--2000.

Cited By

Mo WDechant MMarquardt NAyobi ASingh AHolloway C(2024)Exploring the Design Space of Input Modalities for Working in Mixed Reality on Long-haul FlightsProceedings of the 2024 ACM Designing Interactive Systems Conference10.1145/3643834.3661560(2267-2285)Online publication date: 1-Jul-2024
https://dl.acm.org/doi/10.1145/3643834.3661560
Shen VHarrison CShultz C(2024)Expressive, Scalable, Mid-air Haptics with Synthetic JetsACM Transactions on Computer-Human Interaction10.1145/363515031:2(1-28)Online publication date: 29-Jan-2024
https://dl.acm.org/doi/10.1145/3635150
Neuhaus RRingfort-Felner RCourtney DKneile MHassenzahl M(2024)Virtual Unreality: Augmentation-Oriented Ideation Through Design CardsProceedings of the 2024 CHI Conference on Human Factors in Computing Systems10.1145/3613904.3642364(1-23)Online publication date: 11-May-2024
https://dl.acm.org/doi/10.1145/3613904.3642364
Show More Cited By

Index Terms

Head-mounted display with mid-air tactile feedback

Recommendations

Wearable head-mounted 3D tactile display application scenarios
MobileHCI '16: Proceedings of the 18th International Conference on Human-Computer Interaction with Mobile Devices and Services Adjunct

Current generation virtual reality (VR) and augmented reality (AR) head-mounted displays (HMDs) usually include no or only a single vibration motor for haptic feedback and do not use it for guidance. In a previous work, we presented HapticHead, a ...
Infrared bundle adjusting and clustering method for head-mounted display and Leap Motion calibration

Leap Motion has become widely used due to its ability to recognize intuitive hand gestures or accurate finger positions. Attaching a Leap Motion to a virtual reality head-mounted display (VR HMD) is highly interoperable with virtual objects in virtual ...
Differences in virtual and physical head orientation predict sickness during active head-mounted display-based virtual reality
Abstract
During head-mounted display (HMD)-based virtual reality (VR), head movements and motion-to-photon-based display lag generate differences in our virtual and physical head pose (referred to as DVP). We propose that large-amplitude, time-varying ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

VRST '15: Proceedings of the 21st ACM Symposium on Virtual Reality Software and Technology

November 2015

237 pages

ISBN:9781450339902

DOI:10.1145/2821592

Conference Chairs:
Qinping Zhao
Beihang Univerisity
,
Daniel Thalmann
Nanyang Technological University
,
Editor:
Stephen N. Spencer
University of Washington
,
Program Chairs:
Enhua Wu
University of Macau & Institute of Software, Chinese Academy of Sciences
,
Ming C. Lin
University of North Carolina at Chapel Hill
,
Lili Wang
Beihang University

Copyright © 2015 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 13 November 2015

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Funding Sources

Suomen Akatemia

Conference

VRST '15

Sponsor:

VRST '15: 21th ACM Symposium on Virtual Reality Software and Technology

November 13 - 15, 2015

Beijing, China

Acceptance Rates

Overall Acceptance Rate 66 of 254 submissions, 26%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

70
Total Citations
View Citations
1,490
Total Downloads

Downloads (Last 12 months)106
Downloads (Last 6 weeks)8

Reflects downloads up to 27 Feb 2025

Other Metrics

View Author Metrics

Citations

Cited By

Mo WDechant MMarquardt NAyobi ASingh AHolloway C(2024)Exploring the Design Space of Input Modalities for Working in Mixed Reality on Long-haul FlightsProceedings of the 2024 ACM Designing Interactive Systems Conference10.1145/3643834.3661560(2267-2285)Online publication date: 1-Jul-2024
https://dl.acm.org/doi/10.1145/3643834.3661560
Shen VHarrison CShultz C(2024)Expressive, Scalable, Mid-air Haptics with Synthetic JetsACM Transactions on Computer-Human Interaction10.1145/363515031:2(1-28)Online publication date: 29-Jan-2024
https://dl.acm.org/doi/10.1145/3635150
Neuhaus RRingfort-Felner RCourtney DKneile MHassenzahl M(2024)Virtual Unreality: Augmentation-Oriented Ideation Through Design CardsProceedings of the 2024 CHI Conference on Human Factors in Computing Systems10.1145/3613904.3642364(1-23)Online publication date: 11-May-2024
https://dl.acm.org/doi/10.1145/3613904.3642364
Krasner AGabbard J(2024)MusiKeys: Exploring Haptic-to-Auditory Sensory Substitution to Improve Mid-Air Text-EntryIEEE Transactions on Visualization and Computer Graphics10.1109/TVCG.2024.337206530:5(2247-2256)Online publication date: 4-Mar-2024
https://dl.acm.org/doi/10.1109/TVCG.2024.3372065
Matsubayashi AMakino YShinoda H(2024)Latency Compensation in Ultrasound Tactile Presentation by Linear Prediction of Hand PostureHaptics: Understanding Touch; Technology and Systems; Applications and Interaction10.1007/978-3-031-70058-3_30(365-377)Online publication date: 3-Nov-2024
https://doi.org/10.1007/978-3-031-70058-3_30
Farrugia PCassar GVella PAbela EWodehouse ATamburrino F(2023)Additive Manufacturing in Bespoke Interactive Devices—A Thematic AnalysisApplied Sciences10.3390/app1311662713:11(6627)Online publication date: 30-May-2023
https://doi.org/10.3390/app13116627
Li BLee JGerling G(2023)Perceptibility of programmable softness displays using magnetorheological elastomersProceedings of the Human Factors and Ergonomics Society Annual Meeting10.1177/2169506723119368567:1(1518-1524)Online publication date: 20-Oct-2023
https://doi.org/10.1177/21695067231193685
Cornelio PHughes SGeorgiou OFrier WMaunsbach MVasudevan MObrist M(2023)Responsible Innovation of Touchless Haptics: A Prospective Design Exploration in Social InteractionProceedings of the 2023 CHI Conference on Human Factors in Computing Systems10.1145/3544548.3581228(1-16)Online publication date: 19-Apr-2023
https://dl.acm.org/doi/10.1145/3544548.3581228
Shen ZVasudevan MKučera JObrist MMartinez Plasencia D(2023)Multi-point STM: Effects of Drawing Speed and Number of Focal Points on Users’ Responses using Ultrasonic Mid-Air HapticsProceedings of the 2023 CHI Conference on Human Factors in Computing Systems10.1145/3544548.3580641(1-11)Online publication date: 19-Apr-2023
https://dl.acm.org/doi/10.1145/3544548.3580641
Matsubayashi AShiku KMakino YShinoda H(2023)Focusing Reflected Ultrasound Using Boundary Element Model for Mid-Air Tactile PresentationIEEE Transactions on Haptics10.1109/TOH.2023.328445216:4(695-701)Online publication date: 1-Oct-2023
https://dl.acm.org/doi/10.1109/TOH.2023.3284452
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Figures

Tables

Media

View Table of Conten