skip to main content
10.1145/2671015.2671123acmconferencesArticle/Chapter ViewAbstractPublication PagesvrstConference Proceedingsconference-collections
research-article

I'm in VR!: using your own hands in a fully immersive MR system

Published: 11 November 2014 Publication History

Abstract

This paper presents a novel fully immersive Mixed Reality system that we have recently developed where the user freely walks in a life-size virtual scenario wearing an HMD and can see and use her/his own body when interacting with objects. This form of natural interaction is made possible in our system because the user's hands are real-time captured by means of a RGBD camera on the HMD. This allow the system to have in real-time a texturized geometric mesh of the hands and body (as seen from her/his own perspective) that can be rendered like any other polygonal model in the scene. Our hypothesis is that by presenting to the users an egocentric view of the virtual environment "populated" by their own bodies, a very strong feeling of presence is developed as well.

Supplementary Material

JPG File (p73-tecchia.jpg)
MP4 File (p73-tecchia.mp4)

References

[1]
Heikkinen, T., Iselius, L., Leijonmarck, C. E., Rutqvist, J., & Arvidsson, D. (2002). Does training in a virtual reality simulator improve surgical performance?. Surgical Endoscopy and Other Interventional Techniques, 16(1), 126--129.
[2]
Seymour, N. E., Gallagher, A. G., Roman, S. A., O'Brien, M. K., Bansal, V. K., Andersen, D. K., & Satava, R. M. (2002). Virtual reality training improves operating room performance: results of a randomized, double-blinded study. Annals of surgery, 236(4), 458.
[3]
Van Wyk, E., & de Villiers, R. (2009, February). Virtual reality training applications for the mining industry. In Proceedings of the 6th international conference on computer graphics, virtual reality, visualisation and interaction in Africa (pp. 53--63). ACM.
[4]
Vora, J., Nair, S., Gramopadhye, A. K., Duchowski, A. T., Melloy, B. J., & Kanki, B. (2002). Using virtual reality technology for aircraft visual inspection training: presence and comparison studies. Applied Ergonomics, 33(6), 559--570.
[5]
Gomes de Sá, A., & Zachmann, G. (1999). Virtual reality as a tool for verification of assembly and maintenance processes. Computers & Graphics, 23(3), 389--403.
[6]
Li, J. R., Khoo, L. P., & Tor, S. B. (2003). Desktop virtual reality for maintenance training: an object oriented prototype system (V-REALISM). Computers in Industry, 52(2), 109--125.
[7]
Magee, D., Zhu, Y., Ratnalingam, R., Gardner, P., & Kessel, D. (2007). An augmented reality simulator for ultrasound guided needle placement training. Medical & biological engineering & computing, 45(10), 957--967.
[8]
Lapeer, R., Chen, M. S., & Villagrana, J. (2004, November). An augmented reality based simulation of obstetric forceps delivery. In Mixed and Augmented Reality, 2004. ISMAR 2004. Third IEEE and ACM International Symposium on (pp. 274--275). IEEE.
[9]
Henderson, S., & Feiner, S. (2011). Exploring the benefits of augmented reality documentation for maintenance and repair. Visualization and Computer Graphics, IEEE Transactions on, 17(10), 1355--1368.
[10]
Livingston, M. A., Rosenblum, L. J., Julier, S. J., Brown, D., Baillot, Y., II, J. E. S.,... & Hix, D. (2002, January). An augmented reality system for military operations in urban terrain. In The Interservice/Industry Training, Simulation & Education Conference (I/ITSEC) (Vol. 2002, No. 1). National Training Systems Association.
[11]
Waller, D., Bachmann, E., Hodgson, E., & Beall, A. C. (2007). The HIVE: A huge immersive virtual environment for research in spatial cognition. Behavior Research Methods, 39(4), 835--843.
[12]
Bruder, G., et al. (2009). "Enhancing presence in head-mounted display environments by visual body feedback using head-mounted cameras." CyberWorlds, 2009. CW'09. International Conference on. IEEE.
[13]
Fiore, L., P., and Interrante V., (2012). Towards Achieving Robust Video Self-avatars under Flexible Environment Conditions, International Journal of Virtual Reality (special issue featuring papers from the IEEE VR Workshop on Off-The-Shelf Virtual Reality), 11(3), pp. 33--41.
[14]
Tecchia F., Carrozzino M., Bacinelli S., Rossi F., Vercelli D., Marino G., Gasparello P., Bergamasco M. (2010). A flexible framework for wide-spectrum VR development. Presence: Teleoperators and Virtual Environments 19:302--312
[15]
Tecchia, F., Alem, L., & Huang, W. (2012, December). 3D helping hands: a gesture based MR system for remote collaboration. In Proceedings of the 11th ACM SIGGRAPH International Conference on Virtual-Reality Continuum and its Applications in Industry (pp. 323--328). ACM.
[16]
Suma E., Krum D. and Bolas M., (2011) Sharing Space in Mixed and Virtual Reality Environments Using a Low-Cost Depth Sensor, Proceedings of IEEE ISVRI 2011, pp. 349--350

Cited By

View all
  • (2024)MagNeura: DNN Factored Magnetic Sensor Based Finger Tracking Wearable Design2024 IEEE Biomedical Circuits and Systems Conference (BioCAS)10.1109/BioCAS61083.2024.10798145(1-5)Online publication date: 24-Oct-2024
  • (2023)Combined EEG and immersive virtual reality unveil dopaminergic modulation of error monitoring in Parkinson’s Diseasenpj Parkinson's Disease10.1038/s41531-022-00441-59:1Online publication date: 13-Jan-2023
  • (2023)Synchronous mixed reality (SMR): A personalized virtual‐real fusion framework with high immersion and effective interactionJournal of the Society for Information Display10.1002/jsid.125931:11(621-637)Online publication date: 11-Oct-2023
  • Show More Cited By

Recommendations

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences
VRST '14: Proceedings of the 20th ACM Symposium on Virtual Reality Software and Technology
November 2014
238 pages
ISBN:9781450332538
DOI:10.1145/2671015
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 11 November 2014

Permissions

Request permissions for this article.

Check for updates

Author Tags

  1. hand gestures
  2. mixed reality
  3. natural interaction

Qualifiers

  • Research-article

Conference

VRST '14
Sponsor:

Acceptance Rates

Overall Acceptance Rate 66 of 254 submissions, 26%

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • Downloads (Last 12 months)21
  • Downloads (Last 6 weeks)0
Reflects downloads up to 05 Mar 2025

Other Metrics

Citations

Cited By

View all
  • (2024)MagNeura: DNN Factored Magnetic Sensor Based Finger Tracking Wearable Design2024 IEEE Biomedical Circuits and Systems Conference (BioCAS)10.1109/BioCAS61083.2024.10798145(1-5)Online publication date: 24-Oct-2024
  • (2023)Combined EEG and immersive virtual reality unveil dopaminergic modulation of error monitoring in Parkinson’s Diseasenpj Parkinson's Disease10.1038/s41531-022-00441-59:1Online publication date: 13-Jan-2023
  • (2023)Synchronous mixed reality (SMR): A personalized virtual‐real fusion framework with high immersion and effective interactionJournal of the Society for Information Display10.1002/jsid.125931:11(621-637)Online publication date: 11-Oct-2023
  • (2022)Food Desires and Hedonic Discrimination in Virtual Reality Varying in Product–Context Appropriateness among Older ConsumersFoods10.3390/foods1120322811:20(3228)Online publication date: 15-Oct-2022
  • (2022)Incorporating real-world object into virtual reality: using mobile device input with augmented virtualityMultimedia Tools and Applications10.1007/s11042-022-13637-x83:15(46625-46652)Online publication date: 30-Aug-2022
  • (2021)Presenting Walking Route for VR ZombieIEICE Transactions on Information and Systems10.1587/transinf.2020EDP7084E104.D:1(162-173)Online publication date: 1-Jan-2021
  • (2020)Haptic Space: The Effect of a Rigid Hand Representation on Presence when Interacting with Passive Haptics Controls in VRProceedings of the 19th International Conference on Mobile and Ubiquitous Multimedia10.1145/3428361.3428388(245-253)Online publication date: 22-Nov-2020
  • (2020)Do Virtual Humans Dream of Digital Sheep?IEEE Computer Graphics and Applications10.1109/MCG.2020.299334540:4(71-83)Online publication date: 1-Jul-2020
  • (2020)WikNectVR: A Gesture-Based Approach for Interacting in Virtual Reality Based on WikNect and Gestural WritingVirtual, Augmented and Mixed Reality. Design and Interaction10.1007/978-3-030-49695-1_20(299-312)Online publication date: 19-Jul-2020
  • (2019)Enhancing Augmented VR Interaction via Egocentric Scene AnalysisProceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies10.1145/33512633:3(1-24)Online publication date: 9-Sep-2019
  • Show More Cited By

View Options

Login options

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Figures

Tables

Media

Share

Share

Share this Publication link

Share on social media