Myung Jin Kim
Neung Ryu
Andrea Bianchi
ABSTRACT
This paper’s goal is to understand the haptic-visual congruency perception of skin-slip on the fingertips given visual cues in Virtual Reality (VR). We developed SpinOcchio (Spin for the spinning mechanism used, Occhio for the Italian word “eye”), a handheld haptic controller capable of rendering the thickness and slipping of a virtual object pinched between two fingers. This is achieved using a mechanism with spinning and pivoting disks that apply a tangential skin-slip movement to the fingertips. With SpinOcchio, we determined the baseline haptic discrimination threshold for skin-slip, and, using these results, we tested how haptic realism of motion and thickness is perceived with varying visual cues in VR. Surprisingly, the results show that in all cases, visual cues dominate over haptic perception. Based on these results, we suggest applications that leverage skin-slip and grip interaction, contributing further to realistic experiences in VR.
Supplemental Material
- Mahdi Azmandian, Mark Hancock, Hrvoje Benko, Eyal Ofek, and Andrew D Wilson. 2016. Haptic retargeting: Dynamic repurposing of passive haptics for enhanced virtual reality experiences. In Proceedings of the 2016 chi conference on human factors in computing systems. 1968–1979.Google Scholar
Digital Library
- Joshua H Bacon and Linda Shaw. 1982. Effect of conflict awareness on visual dominance. Perceptual and motor skills 54, 1 (1982), 263–267.Google Scholar
- Hrvoje Benko, Christian Holz, Mike Sinclair, and Eyal Ofek. 2016. Normaltouch and texturetouch: High-fidelity 3d haptic shape rendering on handheld virtual reality controllers. In Proceedings of the 29th Annual Symposium on User Interface Software and Technology. 717–728.Google ScholarDigital Library
- Katja Biermann, Frank Schmitz, Otto W Witte, Jürgen Konczak, Hans-Joachim Freund, and Alfons Schnitzler. 1998. Interaction of finger representation in the human first somatosensory cortex: a neuromagnetic study. Neuroscience letters 251, 1 (1998), 13–16.Google Scholar
- Frank Biocca, Jin Kim, and Yung Choi. 2001. Visual touch in virtual environments: An exploratory study of presence, multimodal interfaces, and cross-modal sensory illusions. Presence: Teleoperators & Virtual Environments 10, 3(2001), 247–265.Google ScholarDigital Library
- Daniel KY Chen, Jean-Baptiste Chossat, and Peter B Shull. 2019. Haptivec: Presenting haptic feedback vectors in handheld controllers using embedded tactile pin arrays. In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems. 1–11.Google ScholarDigital Library
- Inrak Choi, Heather Culbertson, Mark R Miller, Alex Olwal, and Sean Follmer. 2017. Grabity: A wearable haptic interface for simulating weight and grasping in virtual reality. In Proceedings of the 30th Annual ACM Symposium on User Interface Software and Technology. 119–130.Google ScholarDigital Library
- Inrak Choi, Elliot W Hawkes, David L Christensen, Christopher J Ploch, and Sean Follmer. 2016. Wolverine: A wearable haptic interface for grasping in virtual reality. In 2016 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS). IEEE, 986–993.Google ScholarDigital Library
- Inrak Choi, Eyal Ofek, Hrvoje Benko, Mike Sinclair, and Christian Holz. 2018. Claw: A multifunctional handheld haptic controller for grasping, touching, and triggering in virtual reality. In Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems. 1–13.Google ScholarDigital Library
- Massimiliano Di Luca and Arash Mahnan. 2019. Perceptual limits of visual-haptic simultaneity in virtual reality interactions. In 2019 IEEE World Haptics Conference (WHC). IEEE, 67–72.Google ScholarCross Ref
- Marc O Ernst and Martin S Banks. 2002. Humans integrate visual and haptic information in a statistically optimal fashion. Nature 415, 6870 (2002), 429–433.Google Scholar
- Cathy Fang, Yang Zhang, Matthew Dworman, and Chris Harrison. 2020. Wireality: Enabling complex tangible geometries in virtual reality with worn multi-string haptics. In Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems. 1–10.Google ScholarDigital Library
- Sandra G Hart and Lowell E Staveland. 1988. Development of NASA-TLX (Task Load Index): Results of empirical and theoretical research. In Advances in psychology. Vol. 52. Elsevier, 139–183.Google Scholar
- Seongkook Heo, Jaeyeon Lee, and Daniel Wigdor. 2019. PseudoBend: Producing haptic illusions of stretching, bending, and twisting using grain vibrations. In Proceedings of the 32nd Annual ACM Symposium on User Interface Software and Technology. 803–813.Google ScholarDigital Library
- James M Hillis, Marc O Ernst, Martin S Banks, and Michael S Landy. 2002. Combining sensory information: mandatory fusion within, but not between, senses. Science 298, 5598 (2002), 1627–1630.Google Scholar
- Colin Ho, Jonathan Kim, Sachin Patil, and Ken Goldberg. 2015. The slip-pad: a haptic display using interleaved belts to simulate lateral and rotational slip. In 2015 IEEE World Haptics Conference (WHC). IEEE, 189–195.Google ScholarCross Ref
- Lynette A Jones and Hong Z Tan. 2012. Application of psychophysical techniques to haptic research. IEEE transactions on haptics 6, 3 (2012), 268–284.Google Scholar
- Amanda L Kaas, Hanneke I van Mier, Johan Lataster, Mirella Fingal, and Alexander T Sack. 2007. The effect of visuo-haptic congruency on haptic spatial matching. Experimental brain research 183, 1 (2007), 75–85.Google Scholar
- Mirela Kahrimanovic, Wouter M Bergmann Tiest, and Astrid ML Kappers. 2009. Context effects in haptic perception of roughness. Experimental brain research 194, 2 (2009), 287–297.Google Scholar
- Tanja Kassuba, Corinna Klinge, Cordula Hölig, Brigitte Röder, and Hartwig R Siebner. 2013. Vision holds a greater share in visuo-haptic object recognition than touch. Neuroimage 65(2013), 59–68.Google ScholarCross Ref
- Hwan Kim, HyeonBeom Yi, Hyein Lee, and Woohun Lee. 2018. Hapcube: A wearable tactile device to provide tangential and normal pseudo-force feedback on a fingertip. In Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems. 1–13.Google ScholarDigital Library
- Robert Kovacs, Eyal Ofek, Mar Gonzalez Franco, Alexa Fay Siu, Sebastian Marwecki, Christian Holz, and Mike Sinclair. 2020. Haptic pivot: On-demand handhelds in vr. In Proceedings of the 33rd Annual ACM Symposium on User Interface Software and Technology. 1046–1059.Google ScholarDigital Library
- Susan J Lederman. 1981. The perception of surface roughness by active and passive touch. Bulletin of the Psychonomic Society 18, 5 (1981), 253–255.Google ScholarCross Ref
- Susan J Lederman and Lynette A Jones. 2011. Tactile and haptic illusions. IEEE Transactions on Haptics 4, 4 (2011), 273–294.Google ScholarDigital Library
- Jo-Yu Lo, Da-Yuan Huang, Chen-Kuo Sun, Chu-En Hou, and Bing-Yu Chen. 2018. RollingStone: Using single slip taxel for enhancing active finger exploration with a virtual reality controller. In Proceedings of the 31st Annual ACM Symposium on User Interface Software and Technology. 839–851.Google ScholarDigital Library
- Victor Rodrigo Mercado, Maud Marchal, and Anatole Lécuyer. 2019. Entropia: towards infinite surface haptic displays in virtual reality using encountered-type rotating props. IEEE transactions on visualization and computer graphics (2019).Google Scholar
- Kouta Minamizawa, Souichiro Fukamachi, Hiroyuki Kajimoto, Naoki Kawakami, and Susumu Tachi. 2007. Gravity grabber: wearable haptic display to present virtual mass sensation. In ACM SIGGRAPH 2007 emerging technologies. 8–es.Google ScholarDigital Library
- Roderick P Power. 1980. The dominance of touch by vision: Sometimes incomplete. Perception 9, 4 (1980), 457–466.Google ScholarCross Ref
- Roderick P Power and Anne Graham. 1976. Dominance of touch by vision: generalization of the hypothesis to a tactually experienced population. Perception 5, 2 (1976), 161–166.Google ScholarCross Ref
- William Provancher. 2014. Creating greater VR immersion by emulating force feedback with ungrounded tactile feedback. IQT Quarterly 6, 2 (2014), 18–21.Google Scholar
- William R Provancher and Nicholas D Sylvester. 2009. Fingerpad skin stretch increases the perception of virtual friction. IEEE Transactions on Haptics 2, 4 (2009), 212–223.Google ScholarDigital Library
- Irvin Rock and Jack Victor. 1964. Vision and touch: An experimentally created conflict between the two senses. Science (1964), 594–596.Google Scholar
- Neung Ryu, Hye-Young Jo, Michel Pahud, Mike Sinclair, and Andrea Bianchi. 2021. GamesBond: Bimanual Haptic Illusion of Physically Connected Objects for Immersive VR Using Grip Deformation. In Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems. 1–10.Google ScholarDigital Library
- Ali Sengül, Michiel van Elk, Olaf Blanke, and Hannes Bleuler. 2018. Congruent visuo-tactile feedback facilitates the extension of peripersonal space. In International Conference on Human Haptic Sensing and Touch Enabled Computer Applications. Springer, 673–684.Google ScholarCross Ref
- Mike Sinclair, Eyal Ofek, Mar Gonzalez-Franco, and Christian Holz. 2019. Capstancrunch: A haptic vr controller with user-supplied force feedback. In Proceedings of the 32nd annual ACM symposium on user interface software and technology. 815–829.Google ScholarDigital Library
- Hsin-Ruey Tsai, Ching-Wen Hung, Tzu-Chun Wu, and Bing-Yu Chen. 2020. Elastoscillation: 3d multilevel force feedback for damped oscillation on vr controllers. In Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems. 1–12.Google ScholarDigital Library
- Chi Wang, Da-Yuan Huang, Shuo-wen Hsu, Chu-En Hou, Yeu-Luen Chiu, Ruei-Che Chang, Jo-Yu Lo, and Bing-Yu Chen. 2019. Masque: Exploring lateral skin stretch feedback on the face with head-mounted displays. In Proceedings of the 32nd Annual ACM Symposium on User Interface Software and Technology. 439–451.Google ScholarDigital Library
- Chi Wang, Da-Yuan Huang, Shuo-Wen Hsu, Cheng-Lung Lin, Yeu-Luen Chiu, Chu-En Hou, and Bing-Yu Chen. 2020. Gaiters: exploring skin stretch feedback on legs for enhancing virtual reality experiences. In Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems. 1–14.Google ScholarDigital Library
- Eric Whitmire, Hrvoje Benko, Christian Holz, Eyal Ofek, and Mike Sinclair. 2018. Haptic revolver: Touch, shear, texture, and shape rendering on a reconfigurable virtual reality controller. In Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems. 1–12.Google ScholarDigital Library
- Shunki Yamashita, Ryota Ishida, Arihide Takahashi, Hsueh-Han Wu, Hironori Mitake, and Shoichi Hasegawa. 2018. Gum-gum shooting: Inducing a sense of arm elongation via forearm skin-stretch and the change in the center of gravity. In ACM SIGGRAPH 2018 Emerging Technologies. 1–2.Google ScholarDigital Library
- Vibol Yem, Mai Shibahara, Katsunari Sato, and Hiroyuki Kajimoto. 2016. Expression of 2DOF fingertip traction with 1DOF lateral skin stretch. In International AsiaHaptics conference. Springer, 21–25.Google Scholar
- Shigeo Yoshida, Yuqian Sun, and Hideaki Kuzuoka. 2020. Pocopo: Handheld pin-based shape display for haptic rendering in virtual reality. In Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems. 1–13.Google ScholarDigital Library
Index Terms
- SpinOcchio: Understanding Haptic-Visual Congruency of Skin-Slip in VR with a Dynamic Grip Controller
Recommendations
GamesBond: Bimanual Haptic Illusion of Physically Connected Objects for Immersive VR Using Grip Deformation
CHI '21: Proceedings of the 2021 CHI Conference on Human Factors in Computing SystemsVirtual Reality experiences, such as games and simulations, typically support the usage of bimanual controllers to interact with virtual objects. To recreate the haptic sensation of holding objects of various shapes and behaviors with both hands, ...
Haptic Revolver: Touch, Shear, Texture, and Shape Rendering on a Reconfigurable Virtual Reality Controller
CHI '18: Proceedings of the 2018 CHI Conference on Human Factors in Computing SystemsWe present Haptic Revolver, a handheld virtual reality controller that renders fingertip haptics when interacting with virtual surfaces. Haptic Revolver's core haptic element is an actuated wheel that raises and lowers underneath the finger to render ...
Haptic around: multiple tactile sensations for immersive environment and interaction in virtual reality
VRST '18: Proceedings of the 24th ACM Symposium on Virtual Reality Software and TechnologyIn this paper, we present Haptic Around, a hybrid-haptic feedback system, which utilizes fan, hot air blower, mist creator and heat light to recreate multiple tactile sensations in virtual reality for enhancing the immersive environment and interaction. ...
Comments