Abstract
Immersive visualisation is increasingly being used for comprehensive and rapid analysis of objects in 3D and object dynamic behaviour in 4D. Challenges are therefore presented to provide natural user interaction to enable effortless virtual object manipulation. Presented in this paper is the development and evaluation of an immersive human–computer interaction system based on stereoscopic viewing and natural hand gestures. For the development, it is based on the integration of a back-projection stereoscopic system for object and hand display, a hybrid inertial and ultrasonic tracking system to provide the absolute positions and orientations of the user’s head and hands, as well as a pair of high degrees-of-freedom data gloves to provide the relative positions and orientations of digit joints and tips on both hands. For the evaluation, it is based on a two-object scene with a virtual cube and a CT (computed tomography) volume created for demonstration of real-time immersive object manipulation. The system is shown to provide a correct user view of objects and hands in 3D with depth, as well as to enable a user to use a number of simple hand gestures to perform basic object manipulation tasks involving selection, release, translation, rotation and scaling. Also included in the evaluation are some quantitative tests of the system performance in terms of speed and latency.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
Refereneces
Adamo-Villani N., Heisler J, Arns L (2007) Two gesture recognition systems for immersive math education for the deaf. In: ACM proceedings of IMMERSCOM 2007, Verona, Italy, pp 10–12
Badler NI, Philips CB, Webber BL (1993) Simulating humans, computer graphics animation and control. Oxford University Press, New York
Bowman D, Wingrave C, Campbell J, Ly V (2001) Using pinch gloves for both natural and abstract interaction techniques in virtual environments. In: Proceedings of HCI international, New Orleans, USA, pp 629–633
Corradini A, Cohen P (2002) On the relationships among speech, gestures, and object manipulation in virtual environments: initial Evidence. In: Proceedings of the international CLASS workshop on natural, intelligent and effective interaction in multimodal dialogue systems, Copenhagen, Denmark
Corvaglia D (2004) Virtual training for manufacturing and maintenance based on Web3d technologies. In: Proceeding of LET-Web3D 2004: 1st international workshop on Web3D technologies in learning, education and training, pp 28–33
Danisch L, Englehart K, Trivett A (1999) Spatially continuous six degree of freedom position, orientation sensor. Sens Rev 19(2):106–112
Demirdjian D, Ko T, Darrell T (2005) Untethered gesture acquisition and recognition for virtual world manipulation. Virutal Real 8:222–230
Dipietro L, Sabatini AM (2008) A survey of glove-based systems and their applications. IEEE Trans Syst Man Cybern Part C: Appl Rev 38:461–482
Erol A, Bebis G, Nicolescu M, Boyle RD, Twombly X (2007) Vision-based hand pose estimation: a review. Comput Vis Image Underst 108:52–73
Garg P, Aggarwal N, Sofat S (2009) Vision based hand gesture recognition. In: Proceedings of world academy of science, engineering and technology, pp 972–977
IS-900 Precision Inertial-ultrasonic Motion Tracking System, InterSense Inc. http://www.isense.com
Johnson WL, Rickel J (1997) Steve: an animated pedagogical agent for procedural training in virtual environments. Sigart Bulletin, ACM Press 8(1–4): 16–21
Kober C, Boerner BI, Mori S, Tellez CB, Klarhöfer M, Scheffler K, Sader R, Zeilhofer HF (2007) Stereoscopic 4D-visualization of craniofacial soft tissue based on dynamic MRI and 256 row 4D-CT. Adv Med Eng Springer Proc Phys Part II, pp 175–180
Lécuyer A, Lotte F, Reilly R, Leeb R, Hirose M, Slater M (2008) Brain-computer interfaces, virtual reality, and videogames. Computer 41:66–72
Liverneaux P, Nectoux E, Taleb C (2009) The future of robotics in hand surgery. Chirurgie de la Main 28:278–285
O’Hagan RG, Zelinsky A, Rougeaux S (2002) Visual gesture interfaces for virtual environments, interacting with computers, vol 14, pp 231–250, April, 2002
Patel D, Muren LP, Mehus A, Kvinnsland Y, Ulvang DM, Villanger KP (2007) A virtual reality solution for evaluation of radiotherapy plans. Radiother Oncol 82(2):218–221
Scharver C, Evenhouse R, Johnson A, Leigh J (2004) Designing cranial implants in a haptic augmented reality environment. Commun ACM 47:32–38
ShapeHand Data Glove, Measurand Inc. http://www.measurand.com
Sturman DJ, Zeltzer D (1994) A survey of glove-based input. IEEE Comput Graph Appl 14(1):30–39
Toyama H (2006) Trials on grasping of a 3D virtual object in CAVE using EEG signals. IEIC Tech Rep 91:53–56
Wang XH, Good WF, Fuhrman CR, Sumkin JH, Britton CA, Golla SK (2005) Stereo CT image compositing methods for lung nodule detection, characterization. Acad Radiol 12(12):1512–1520
Woods AJ (2005) Compatibility of display products with stereoscopic display methods. In: Proceedings of the international display manufacturing conference, Taipei, Taiwan
Wormell D, Foxlin E (2003) Advancements in 3D interactive devices for virtual environments. In: Proceedings of the workshop on virtual environments, Zurich, Switzerland, pp 47–56
Zhang S, Demiralp C, Keefe D, DaSilva M, Laidlaw DH, Greenberg, BD, Basser PJ, Pierpaoli C, Chiocca EA, Deisboeck TS (2001) An immersive virtual environment for DT-MRI volume visualization applications: a case study. In: Proceedings of IEEE visualization conference, San Diego, USA, pp 437–440
Acknowledgments
The authors thank Dr. X. Chen for his technical support.
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Lu, G., Shark, LK., Hall, G. et al. Immersive manipulation of virtual objects through glove-based hand gesture interaction. Virtual Reality 16, 243–252 (2012). https://doi.org/10.1007/s10055-011-0195-9
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10055-011-0195-9