skip to main content
10.1145/3411764.3445727acmconferencesArticle/Chapter ViewAbstractPublication PageschiConference Proceedingsconference-collections
research-article

GamesBond: Bimanual Haptic Illusion of Physically Connected Objects for Immersive VR Using Grip Deformation

Published: 07 May 2021 Publication History

Abstract

Virtual Reality experiences, such as games and simulations, typically support the usage of bimanual controllers to interact with virtual objects. To recreate the haptic sensation of holding objects of various shapes and behaviors with both hands, previous researchers have used mechanical linkages between the controllers that render adjustable stiffness. However, the linkage cannot quickly adapt to simulate dynamic objects, nor it can be removed to support free movements. This paper introduces GamesBond, a pair of 4-DoF controllers without physical linkage but capable to create the illusion of being connected as a single device, forming a virtual bond. The two controllers work together by dynamically displaying and physically rendering deformations of hand grips, and so allowing users to perceive a single connected object between the hands, such as a jumping rope. With a user study and various applications we show that GamesBond increases the realism, immersion, and enjoyment of bimanual interaction.

Supplementary Material

MP4 File (3411764.3445727_videofigure.mp4)
Supplemental video
MP4 File (3411764.3445727_videopreview.mp4)
Preview video

References

[1]
Ravin Balakrishnan and Ken Hinckley. 1999. The Role of Kinesthetic Reference Frames in Two-Handed Input Performance. In Proceedings of the 12th Annual ACM Symposium on User Interface Software and Technology(Asheville, North Carolina, USA) (UIST ’99). Association for Computing Machinery, New York, NY, USA, 171–178. https://doi.org/10.1145/320719.322599
[2]
J. J. O. Barros, V. M. F. d. Santos, and F. M. T. P. d. Silva. 2015. Bimanual Haptics for Humanoid Robot Teleoperation Using ROS and V-REP. In 2015 IEEE International Conference on Autonomous Robot Systems and Competitions. 174–179. https://doi.org/10.1109/ICARSC.2015.27
[3]
Sonny Chan, François Conti, Kenneth Salisbury, and Nikolas H Blevins. 2013. Virtual reality simulation in neurosurgery: technologies and evolution. Neurosurgery 72, suppl_1 (2013), A154–A164. https://doi.org/10.1227/NEU.0b013e3182750d26
[4]
Daniel KY Chen, Jean-Baptiste Chossat, and Peter B Shull. 2019. Haptivec: Presenting haptic feedback vectors in handheld controllers using embedded tactile pin arrays. In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems. 1–11. https://doi.org/10.1145/3290605.3300401
[5]
Eric J. Gonzalez and Sean Follmer. 2019. Investigating the Detection of Bimanual Haptic Retargeting in Virtual Reality. In 25th ACM Symposium on Virtual Reality Software and Technology (Parramatta, NSW, Australia) (VRST ’19). Association for Computing Machinery, New York, NY, USA, Article 12, 5 pages. https://doi.org/10.1145/3359996.3364248
[6]
Mar Gonzalez Franco and Christopher C Berger. 2019. Avatar Embodiment Enhances Haptic Confidence on the Out-of-Body Touch Illusion. IEEE Transactions on Haptics 12, 3 (June 2019), 319–326. https://doi.org/10.1109/TOH.2019.2925038
[7]
Yves Guiard. 1987. Asymmetric division of labor in human skilled bimanual action: The kinematic chain as a model. Journal of motor behavior 19, 4 (1987), 486–517. https://doi.org//10.1080/00222895.1987.10735426
[8]
Ashley L Guinan, Markus N Montandon, Andrew J Doxon, and William R Provancher. 2014. Discrimination thresholds for communicating rotational inertia and torque using differential skin stretch feedback in virtual environments. In 2014 IEEE Haptics Symposium (HAPTICS). IEEE, 277–282. https://doi.org/10.1109/HAPTICS.2014.6775467
[9]
Ashley L Guinan, Markus N Montandon, Andrew J Doxon, and William R Provancher. 2014. An ungrounded tactile feedback device to portray force and torque-like interactions in virtual environments. In 2014 IEEE Virtual Reality (VR). IEEE, 171–172. https://doi.org/10.1109/VR.2014.6802106
[10]
Takuya Handa, Makiko Azuma, Toshihiro Shimizu, and Satoru Kondo. 2017. GraspForm: Shape and Hardness Rendering on Handheld Device toward Universal Interface for 3D Haptic TV. In Adjunct Publication of the 30th Annual ACM Symposium on User Interface Software and Technology. 147–149. https://doi.org/10.1145/3131785.3131824
[11]
Tomás Henriques. 2019. SPR1NG Controller: An Interface for Kinesthetic Spatial Manipulation. In 2019 IEEE 7th International Conference on Serious Games and Applications for Health (SeGAH). IEEE, 1–5. https://doi.org/10.1109/SeGAH.2019.8882478
[12]
Seongkook Heo, Jaeyeon Lee, and Daniel Wigdor. 2019. PseudoBend: Producing haptic illusions of stretching, bending, and twisting using grain vibrations. In Proceedings of the 32nd Annual ACM Symposium on User Interface Software and Technology. 803–813. https://doi.org/10.1145/3332165.3347941
[13]
Ken Hinckley, Michel Pahud, Hrvoje Benko, Pourang Irani, François Guimbretière, Marcel Gavriliu, Xiang ’Anthony’ Chen, Fabrice Matulic, William Buxton, and Andrew Wilson. 2014. Sensing Techniques for Tablet+stylus Interaction. In Proceedings of the 27th Annual ACM Symposium on User Interface Software and Technology (Honolulu, Hawaii, USA) (UIST ’14). Association for Computing Machinery, New York, NY, USA, 605–614. https://doi.org/10.1145/2642918.2647379
[14]
Ken Hinckley, Koji Yatani, Michel Pahud, Nicole Coddington, Jenny Rodenhouse, Andy Wilson, Hrvoje Benko, and Bill Buxton. 2010. Pen + Touch = New Tools. In Proceedings of the 23nd Annual ACM Symposium on User Interface Software and Technology (New York, New York, USA) (UIST ’10). Association for Computing Machinery, New York, NY, USA, 27–36. https://doi.org/10.1145/1866029.1866036
[15]
Yuki Imamura, Hironori Arakawa, Sho Kamuro, Kouta Minamizawa, and Susumu Tachi. 2011. HAPMAP: Haptic walking navigation system with support by the sense of handrail. In ACM SIGGRAPH 2011 Emerging Technologies. 1–1. https://doi.org/10.1145/2048259.2048265
[16]
Seungwoo Je, Myung Jin Kim, Woojin Lee, Byungjoo Lee, Xing-Dong Yang, Pedro Lopes, and Andrea Bianchi. 2019. Aero-plane: A Handheld Force-Feedback Device that Renders Weight Motion Illusion on a Virtual 2D Plane. In Proceedings of the 32nd Annual ACM Symposium on User Interface Software and Technology. 763–775. https://doi.org/10.1145/3332165.3347926
[17]
Oussama Khatib, Xiyang Yeh, Gerald Brantner, Brian Soe, Boyeon Kim, Shameek Ganguly, Hannah Stuart, Shiquan Wang, Mark Cutkosky, Aaron Edsinger, 2016. Ocean one: A robotic avatar for oceanic discovery. IEEE Robotics & Automation Magazine 23, 4 (2016), 20–29. https://doi.org/10.1109/MRA.2016.2613281
[18]
Andrey Krekhov, Katharina Emmerich, Philipp Bergmann, Sebastian Cmentowski, and Jens Krüger. 2017. Self-transforming controllers for virtual reality first person shooters. In Proceedings of the Annual Symposium on Computer-Human Interaction in Play. 517–529. https://doi.org/10.1145/3116595.3116615
[19]
Alexander Kron, Günther Schmidt, Bernd Petzold, Michael I Zah, Peter Hinterseer, and Eckehard Steinbach. 2004. Disposal of explosive ordnances by use of a bimanual haptic telepresence system. In IEEE International Conference on Robotics and Automation, 2004. Proceedings. ICRA’04. 2004, Vol. 2. IEEE, 1968–1973. https://doi.org/10.1109/ROBOT.2004.1308112
[20]
Michael A Liss and Elspeth M McDougall. 2013. Robotic surgical simulation. The Cancer Journal 19, 2 (2013), 124–129. https://doi.org/10.1097/PPO.0b013e3182885d79
[21]
Pedro Lopes, Sijing You, Lung-Pan Cheng, Sebastian Marwecki, and Patrick Baudisch. 2017. Providing Haptics to Walls & Heavy Objects in Virtual Reality by Means of Electrical Muscle Stimulation. In Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems (Denver, Colorado, USA) (CHI ’17). Association for Computing Machinery, New York, NY, USA, 1471–1482. https://doi.org/10.1145/3025453.3025600
[22]
Benjamin C Mac Murray, Bryan N Peele, Patricia Xu, Josef Spjut, Omer Shapira, David Luebke, and Robert F Shepherd. 2018. A variable shape and variable stiffness controller for haptic virtual interactions. In 2018 IEEE International Conference on Soft Robotics (RoboSoft). IEEE, 264–269. https://doi.org/10.1109/ROBOSOFT.2018.8404930
[23]
Miles Macklin, Matthias Müller, and Nuttapong Chentanez. 2016. XPBD: Position-Based Simulation of Compliant Constrained Dynamics. In Proceedings of the 9th International Conference on Motion in Games (Burlingame, California) (MIG ’16). Association for Computing Machinery, New York, NY, USA, 49–54. https://doi.org/10.1145/2994258.2994272
[24]
Jun Murayama, Laroussi Bougrila, YanLin Luo, Katsuhito Akahane, Shoichi Hasegawa, Béat Hirsbrunner, and Makoto Sato. 2004. SPIDAR G&G: a two-handed haptic interface for bimanual VR interaction. In Proceedings of EuroHaptics, Vol. 2004. Citeseer, 138–146.
[25]
Ken Nakagaki, Chikara Inamura, Pasquale Totaro, Thariq Shihipar, Chantine Akikyama, Yin Shuang, and Hiroshi Ishii. 2015. Linked-Stick: Conveying a Physical Experience Using a Shape-Shifting Stick. In Proceedings of the 33rd Annual ACM Conference Extended Abstracts on Human Factors in Computing Systems (Seoul, Republic of Korea) (CHI EA ’15). Association for Computing Machinery, New York, NY, USA, 1609–1614. https://doi.org/10.1145/2702613.2732712
[26]
MJ Patel, JHY Ng, WE Hawkins, KF Pitts, and S Chakrabarti-Bell. 2012. Effects of fungal α-amylase on chemically leavened wheat flour doughs. Journal of cereal science 56, 3 (2012), 644–651. https://doi.org/10.1016/j.jcs.2012.08.002
[27]
Angelika Peer and Martin Buss. 2008. A new admittance-type haptic interface for bimanual manipulations. IEEE/ASME Transactions on mechatronics 13, 4 (2008), 416–428. https://doi.org/10.1109/TMECH.2008.2001690
[28]
Ken Pfeuffer, Ken Hinckley, Michel Pahud, and Bill Buxton. 2017. Thumb + Pen Interaction on Tablets. In Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems (Denver, Colorado, USA) (CHI ’17). Association for Computing Machinery, New York, NY, USA, 3254–3266. https://doi.org/10.1145/3025453.3025567
[29]
Tomoya Sasaki, Richard Sahala Hartanto, Kao-Hua Liu, Keitarou Tsuchiya, Atsushi Hiyama, and Masahiko Inami. 2018. Leviopole: mid-air haptic interactions using multirotor. In ACM SIGGRAPH 2018 Emerging Technologies. 1–2. https://doi.org/10.1145/3214907.3214913
[30]
Jotaro Shigeyama, Takeru Hashimoto, Shigeo Yoshida, Takuji Narumi, Tomohiro Tanikawa, and Michitaka Hirose. 2019. Transcalibur: A weight shifting virtual reality controller for 2d shape rendering based on computational perception model. In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems. 1–11. https://doi.org/10.1145/3290605.3300241
[31]
Peng Song, Wooi Boon Goh, William Hutama, Chi-Wing Fu, and Xiaopei Liu. 2012. A Handle Bar Metaphor for Virtual Object Manipulation with Mid-Air Interaction. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (Austin, Texas, USA) (CHI ’12). Association for Computing Machinery, New York, NY, USA, 1297–1306. https://doi.org/10.1145/2207676.2208585
[32]
Adam J Spiers and Aaron M Dollar. 2016. Design and evaluation of shape-changing haptic interfaces for pedestrian navigation assistance. IEEE transactions on haptics 10, 1 (2016), 17–28. https://doi.org/10.1109/TOH.2016.2582481
[33]
Adam J Spiers, Janet van Der Linden, Maria Oshodi, and Aaron M Dollar. 2016. Development and experimental validation of a minimalistic shape-changing haptic navigation device. In 2016 IEEE International Conference on Robotics and Automation (ICRA). IEEE, 2688–2695. https://doi.org/10.1109/ICRA.2016.7487430
[34]
Evan Strasnick, Christian Holz, Eyal Ofek, Mike Sinclair, and Hrvoje Benko. 2018. Haptic links: Bimanual haptics for virtual reality using variable stiffness actuation. In Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems. 1–12. https://doi.org/10.1145/3173574.3174218
[35]
Yuqian Sun, Shigeo Yoshida, Takuji Narumi, and Michitaka Hirose. 2019. Pacapa: A handheld vr device for rendering size, shape, and stiffness of virtual objects in tool-based interactions. In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems. 1–12. https://doi.org/10.1145/3290605.3300682
[36]
Colin Swindells, Alex Unden, and Tao Sang. 2003. TorqueBAR: an ungrounded haptic feedback device. In Proceedings of the 5th international conference on Multimodal interfaces. 52–59. https://doi.org/10.1145/958432.958445
[37]
Shan-Yuan Teng, Tzu-Sheng Kuo, Chi Wang, Chi-huan Chiang, Da-Yuan Huang, Liwei Chan, and Bing-Yu Chen. 2018. PuPoP: Pop-up Prop on Palm for Virtual Reality. In Proceedings of the 31st Annual ACM Symposium on User Interface Software and Technology (Berlin, Germany) (UIST ’18). Association for Computing Machinery, New York, NY, USA, 5–17. https://doi.org/10.1145/3242587.3242628
[38]
Julie M Walker, Nabil Zemiti, Philippe Poignet, and Allison M Okamura. 2019. Holdable haptic device for 4-dof motion guidance. In 2019 IEEE World Haptics Conference (WHC). IEEE, 109–114. https://doi.org/10.1109/WHC.2019.8816171
[39]
Shigeo Yoshida, Yuqian Sun, and Hideaki Kuzuoka. 2020. PoCoPo: Handheld Pin-based Shape Display for Haptic Rendering in Virtual Reality. In Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems. 1–13. https://doi.org/10.1145/3313831.3376358
[40]
Andre Zenner and Antonio Krüger. 2017. Shifty: A weight-shifting dynamic passive haptic proxy to enhance object perception in virtual reality. IEEE transactions on visualization and computer graphics 23, 4(2017), 1285–1294. https://doi.org/10.1109/TVCG.2017.2656978
[41]
André Zenner and Antonio Krüger. 2019. Drag: on: A virtual reality controller providing haptic feedback based on drag and weight shift. In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems. 1–12. https://doi.org/10.1145/3290605.3300441

Cited By

View all
  • (2025)How Does Games User Research Measure and Guide Haptic Experience Design? A Critical ReviewHuman-Computer Interaction. Design and Research10.1007/978-3-031-80832-6_2(23-38)Online publication date: 14-Feb-2025
  • (2024)MorphGrip: Haptic Guidance Using a Shape-Changing GripProceedings of the International Conference on Mobile and Ubiquitous Multimedia10.1145/3701571.3701573(196-209)Online publication date: 1-Dec-2024
  • (2024)MetaDragonBoat: Exploring Paddling Techniques of Virtual Dragon Boating in a Metaverse CampusProceedings of the 32nd ACM International Conference on Multimedia10.1145/3664647.3681078(6335-6344)Online publication date: 28-Oct-2024
  • Show More Cited By

Index Terms

  1. GamesBond: Bimanual Haptic Illusion of Physically Connected Objects for Immersive VR Using Grip Deformation
          Index terms have been assigned to the content through auto-classification.

          Recommendations

          Comments

          Information & Contributors

          Information

          Published In

          cover image ACM Conferences
          CHI '21: Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems
          May 2021
          10862 pages
          ISBN:9781450380966
          DOI:10.1145/3411764
          Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

          Sponsors

          Publisher

          Association for Computing Machinery

          New York, NY, United States

          Publication History

          Published: 07 May 2021

          Permissions

          Request permissions for this article.

          Check for updates

          Badges

          • Honorable Mention

          Author Tags

          1. Haptics
          2. Virtual Reality
          3. bimanual interaction
          4. grip deformation
          5. shape-changing

          Qualifiers

          • Research-article
          • Research
          • Refereed limited

          Funding Sources

          • NRF-MSIT

          Conference

          CHI '21
          Sponsor:

          Acceptance Rates

          Overall Acceptance Rate 6,199 of 26,314 submissions, 24%

          Upcoming Conference

          CHI 2025
          ACM CHI Conference on Human Factors in Computing Systems
          April 26 - May 1, 2025
          Yokohama , Japan

          Contributors

          Other Metrics

          Bibliometrics & Citations

          Bibliometrics

          Article Metrics

          • Downloads (Last 12 months)191
          • Downloads (Last 6 weeks)46
          Reflects downloads up to 03 Mar 2025

          Other Metrics

          Citations

          Cited By

          View all
          • (2025)How Does Games User Research Measure and Guide Haptic Experience Design? A Critical ReviewHuman-Computer Interaction. Design and Research10.1007/978-3-031-80832-6_2(23-38)Online publication date: 14-Feb-2025
          • (2024)MorphGrip: Haptic Guidance Using a Shape-Changing GripProceedings of the International Conference on Mobile and Ubiquitous Multimedia10.1145/3701571.3701573(196-209)Online publication date: 1-Dec-2024
          • (2024)MetaDragonBoat: Exploring Paddling Techniques of Virtual Dragon Boating in a Metaverse CampusProceedings of the 32nd ACM International Conference on Multimedia10.1145/3664647.3681078(6335-6344)Online publication date: 28-Oct-2024
          • (2024)RePresent: Enabling Access to Justice for Pro Se Litigants via Co-Authored Serious GamesProceedings of the 2024 ACM Designing Interactive Systems Conference10.1145/3643834.3661632(3225-3242)Online publication date: 1-Jul-2024
          • (2024)Exploring Bimanual Haptic Feedback for Spatial Search in Virtual RealityIEEE Transactions on Visualization and Computer Graphics10.1109/TVCG.2024.337204530:5(2422-2433)Online publication date: 4-Mar-2024
          • (2024)Bridging the digital–physical divide using haptic and wearable technologiesNature Electronics10.1038/s41928-024-01325-77:12(1098-1110)Online publication date: 20-Dec-2024
          • (2023)WriMouCon: Wrist-Mounted Haptic Controller for Rendering Physical Properties in Virtual Reality2023 IEEE World Haptics Conference (WHC)10.1109/WHC56415.2023.10224423(34-40)Online publication date: 10-Jul-2023
          • (2023)Don't Walk Away! Virtual Safety Boundaries for Collaborative Virtual Reality Learning Environments2023 IEEE Frontiers in Education Conference (FIE)10.1109/FIE58773.2023.10343427(1-5)Online publication date: 18-Oct-2023
          • (2022)HapticLever: Kinematic Force Feedback using a 3D PantographAdjunct Proceedings of the 35th Annual ACM Symposium on User Interface Software and Technology10.1145/3526114.3558736(1-4)Online publication date: 29-Oct-2022
          • (2022)ELAXO : Rendering Versatile Resistive Force Feedback for Fingers Grasping and TwistingProceedings of the 35th Annual ACM Symposium on User Interface Software and Technology10.1145/3526113.3545677(1-14)Online publication date: 29-Oct-2022
          • Show More Cited By

          View Options

          Login options

          View options

          PDF

          View or Download as a PDF file.

          PDF

          eReader

          View online with eReader.

          eReader

          HTML Format

          View this article in HTML Format.

          HTML Format

          Figures

          Tables

          Media

          Share

          Share

          Share this Publication link

          Share on social media