skip to main content
10.1145/3294109.3300988acmconferencesArticle/Chapter ViewAbstractPublication PagesteiConference Proceedingsconference-collections
Work in Progress

SWISH: Shifting Weight-based Interfaces for Simulated Hydrodynamics in Mixed-Reality Fluid Vessels

Published: 17 March 2019 Publication History

Abstract

Mixed-reality haptic devices introduce a gateway to otherwise intangible virtual content, creating a life-like immersive experience. Congruent haptic sensation requires faithful integration of visual stimuli and perceived tactile sensation. Unfortunately, current commercial mixed-reality systems are unable to reproduce the physical sensation of fluid vessels, due to the shifting nature of fluid motion. To this end, we introduce SWISH, a novel type of ungrounded mixed-reality system, capable of affording the users a realistic haptic sensation of fluid behavior. We also present solutions to prominent challenges of rendering haptic fluid behavior, especially in coordinate translation and virtual adaptation to physical limitation. Our virtual-to-physical coupling uses Nvidia Flex's Unreal Engine integration, wirelessly controlling a motorized mechanical actuation system housed in a plastic "vessel''. In this paper we discuss the current state of SWISH and present results from our preliminary user study, followed by a description of our future planned phases.

Supplementary Material

suppl.mov (teiwp1062.mp4)
Supplemental video

References

[1]
J. M. Andujar, A. Mejias, and M. A. Marquez. 2011. Augmented Reality for the Improvement of Remote Laboratories: An Augmented Remote Laboratory. IEEE Transactions on Education 54, 3 (Aug 2011), 492--500.
[2]
Bruno Araujo, Ricardo Jota, Varun Perumal, Jia Xian Yao, Karan Singh, and Daniel Wigdor. 2016. Snake Charmer: Physically Enabling Virtual Objects. In Proceedings of the TEI'16: Tenth International Conference on Tangible, Embedded, and Embodied Interaction. ACM, 218--226.
[3]
Ronald T. Azuma. 1997. A Survey of Augmented Reality. Presence: Teleoper. Virtual Environ. 6, 4 (Aug. 1997), 355--385.
[4]
Hrvoje Benko, Christian Holz, Mike Sinclair, and Eyal Ofek. 2016. Normaltouch and texturetouch: High-fidelity 3d haptic shape rendering on handheld virtual reality controllers. In Proceedings of the 29th Annual Symposium on User Interface Software and Technology. ACM, 717--728.
[5]
Inrak Choi, Elliot W Hawkes, David L Christensen, Christopher J Ploch, and Sean Follmer. 2016. Wolverine: A wearable haptic interface for grasping in virtual reality. In Intelligent Robots and Systems (IROS), 2016 IEEE/RSJ International Conference on. IEEE, 986--993.
[6]
Gabriel Cirio, Maud Marchal, Miguel A Otaduy, and Anatole Lécuyer. 2013. Six-oof haptic interaction with fluids, solids, and their transitions. In World Haptics Conference (WHC), 2013. IEEE, 157--162.
[7]
Heather Culbertson, Julie M Walker, and Allison M Okamura. 2016. Modeling and design of asymmetric vibrations to induce ungrounded pulling sensation through asymmetric skin displacement. In Haptics Symposium (HAPTICS), 2016 IEEE. IEEE, 27--33.
[8]
Nick Foster and Dimitri Metaxas. 1996. Realistic Animation of Liquids. Graphical Models and Image Processing 58, 5 (1996), 471 -- 483.
[9]
Nirit Gavish, Teresa Gutiérrez, Sabine Webel, Jorge Rodríguez, Matteo Peveri, Uli Bockholt, and Franco Tecchia. 2015. Evaluating virtual reality and augmented reality training for industrial maintenance and assembly tasks. Interactive Learning Environments 23, 6 (2015), 778--798.
[10]
Prashant Goswami, Philipp Schlegel, Barbara Solenthaler, and Renato Pajarola. 2010. Interactive SPH simulation and rendering on the GPU. In Proceedings of the 2010 ACM SIGGRAPH/Eurographics Symposium on Computer Animation. Eurographics Association, 55--64.
[11]
Xiaochi Gu, Yifei Zhang, Weize Sun, Yuanzhe Bian, Dao Zhou, and Per Ola Kristensson. 2016. Dexmo: An inexpensive and lightweight mechanical exoskeleton for motion capture and force feedback in VR. In Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems. ACM, 1991--1995.
[12]
Takahiro Harada, Seiichi Koshizuka, and Yoichiro Kawaguchi. 2007. Smoothed particle hydrodynamics on GPUs. In Computer Graphics International, Vol. 40. SBC Petropolis, 63--70.
[13]
Daniel Harley, Alexander Verni, Mackenzie Willis, Ashley Ng, Lucas Bozzo, and Ali Mazalek. 2018. Sensory VR: Smelling, Touching, and Eating Virtual Reality. In Proceedings of the Twelfth International Conference on Tangible, Embedded, and Embodied Interaction. ACM, 386--397.
[14]
Hannes Kaufmann. 2003. Collaborative augmented reality in education. Institute of Software Technology and Interactive Systems, Vienna University of Technology (2003).
[15]
Kangdon Lee. 2012. Augmented Reality in Education and Training. TechTrends 56, 2 (01 Mar 2012), 13--21.
[16]
Thomas H Massie, J Kenneth Salisbury, and others. 1994. The phantom haptic interface: A device for probing virtual objects. In Proceedings of the ASME winter annual meeting, symposium on haptic interfaces for virtual environment and teleoperator systems, Vol. 55. Citeseer, 295--300.
[17]
Kouta Minamizawa, Souichiro Fukamachi, Hiroyuki Kajimoto, Naoki Kawakami, and Susumu Tachi. 2007. Gravity grabber: wearable haptic display to present virtual mass sensation. In ACM SIGGRAPH 2007 emerging technologies. ACM, 8.
[18]
Anne M Murray, Roberta L Klatzky, and Pradeep K Khosla. 2003. Psychophysical characterization and testbed validation of a wearable vibrotactile glove for telemanipulation. Presence: Teleoperators & Virtual Environments 12, 2 (2003), 156--182.
[19]
Ryuma Niiyama, Lining Yao, and Hiroshi Ishii. 2014. Weight and volume changing device with liquid metal transfer. In Proceedings of the 8th International Conference on Tangible, Embedded and Embodied Interaction. ACM, 49--52.
[20]
Eugenio Rustico, Giuseppe Bilotta, Alexis Herault, Ciro Del Negro, and Giovanni Gallo. 2014. Advances in multi-GPU smoothed particle hydrodynamics simulations. IEEE Transactions on Parallel and Distributed Systems 25, 1 (2014), 43--52.
[21]
Adalberto L Simeone, Eduardo Velloso, and Hans Gellersen. 2015. Substitutional reality: Using the physical environment to design virtual reality experiences. In Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems. ACM, 3307--3316.
[22]
Jos Stam. 1999. Stable fluids. In Proceedings of the 26th annual conference on Computer graphics and interactive techniques. ACM Press/Addison-Wesley Publishing Co., 121--128.
[23]
Colin Swindells, Alex Unden, and Tao Sang. 2003. TorqueBAR: an ungrounded haptic feedback device. In Proceedings of the 5th international conference on Multimodal interfaces. ACM, 52--59.
[24]
Richard Q Van der Linde, Piet Lammertse, Erwin Frederiksen, and B Ruiter. 2002. The HapticMaster, a new high-performance haptic interface. In Proc. Eurohaptics. 1--5.
[25]
Mauricio Vines, Javier Mora, and Won-Sook Lee. 2009. Haptic display of 3D liquids for interactive applications. In Games Innovations Conference, 2009. ICE-GIC 2009. International IEEE Consumer Electronics Society's. IEEE, 140--148.
[26]
Julie Walker, Heather Culbertson, Michael Raitor, and Allison Okamura. 2017. Haptic orientation guidance using two parallel double-gimbal control moment gyroscopes. IEEE transactions on haptics (2017).
[27]
Sabine Webel, Uli Bockholt, Timo Engelke, Nirit Gavish, Manuel Olbrich, and Carsten Preusche. 2013. An Augmented Reality Training Platform for Assembly and Maintenance Skills. Robot. Auton. Syst. 61, 4 (April 2013), 398--403.
[28]
Kyle N Winfree, Joseph M Romano, Jamie Gewirtz, and Katherine J Kuchenbecker. 2010. Control of a high fidelity ungrounded torque feedback device: The iTorqU 2.1. In Robotics and Automation (ICRA), 2010 IEEE International Conference on. IEEE, 1347--1352.
[29]
Hsin-Kai Wu, Silvia Wen-Yu Lee, Hsin-Yi Chang, and Jyh-Chong Liang. 2013. Current status, opportunities and challenges of augmented reality in education. Computers & education 62 (2013), 41--49.
[30]
Vibol Yem, Ryuta Okazaki, and Hiroyuki Kajimoto. 2016. FinGAR: combination of electrical and mechanical stimulation for high-fidelity tactile presentation. In ACM SIGGRAPH 2016 Emerging Technologies. ACM, 7.
[31]
Steve Chi-Yin Yuen, Gallayanee Yaoyuneyong, and Erik Johnson. 2011. Augmented reality: An overview and five directions for AR in education. Journal of Educational Technology Development and Exchange (JETDE) 4, 1 (2011), 11.
[32]
André Zenner. 2016. Investigating weight distribution in virtual reality proxy interaction. Universit"at des Saarlandes, Masterarbeit (2016).
[33]
Andre Zenner and Antonio Krüger. 2017. Shifty: A Weight-Shifting Dynamic Passive Haptic Proxy to Enhance Object Perception in Virtual Reality. IEEE Transactions on Visualization and Computer Graphics 23, 4 (2017), 1285--1294.
[34]
Fengquan Zhang, Lei Hu, Jiawen Wu, and Xukun Shen. 2011. A SPH-based method for interactive fluids simulation on the multi-GPU. In Proceedings of the 10th international conference on virtual reality continuum and its applications in industry. ACM, 423--426.
[35]
Jiachao Zhang, Shunpei Yuasa, Shinji Fukuma, and Shin-Ichiro Mori. 2016. A real-time GPU-based coupled fluid-structure simulation with haptic interaction. In 2016 IEEE/ACIS 15th International Conference on Computer and Information Science (ICIS). IEEE, 1--6.
[36]
MA Zhou and Pinhas Ben-Tzvi. 2015. RML glove-An exoskeleton glove mechanism with haptics feedback. IEEE/ASME Transactions on Mechatronics 20, 2 (2015), 641--652.

Cited By

View all
  • (2024)HapticWhirl, a Flywheel-Gimbal Handheld Haptic Controller for Exploring Multimodal Haptic FeedbackSensors10.3390/s2403093524:3(935)Online publication date: 31-Jan-2024
  • (2024)Advancing Turbine Prediction: Harnessing Conformable Artificial Neural Networks for the Fracture AnalysisJournal of Failure Analysis and Prevention10.1007/s11668-024-02087-2Online publication date: 29-Dec-2024
  • (2020)Glissade: Generating Balance Shifting Feedback to Facilitate Auxiliary Digital Pen InputProceedings of the 2020 CHI Conference on Human Factors in Computing Systems10.1145/3313831.3376505(1-13)Online publication date: 21-Apr-2020

Index Terms

  1. SWISH: Shifting Weight-based Interfaces for Simulated Hydrodynamics in Mixed-Reality Fluid Vessels

      Recommendations

      Comments

      Information & Contributors

      Information

      Published In

      cover image ACM Conferences
      TEI '19: Proceedings of the Thirteenth International Conference on Tangible, Embedded, and Embodied Interaction
      March 2019
      785 pages
      ISBN:9781450361965
      DOI:10.1145/3294109
      Permission to make digital or hard copies of part or all of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for third-party components of this work must be honored. For all other uses, contact the Owner/Author.

      Sponsors

      Publisher

      Association for Computing Machinery

      New York, NY, United States

      Publication History

      Published: 17 March 2019

      Check for updates

      Author Tags

      1. input devices
      2. perception
      3. ungrounded haptic feedback system

      Qualifiers

      • Work in progress

      Conference

      TEI '19
      Sponsor:

      Acceptance Rates

      TEI '19 Paper Acceptance Rate 36 of 110 submissions, 33%;
      Overall Acceptance Rate 393 of 1,367 submissions, 29%

      Upcoming Conference

      TEI '25

      Contributors

      Other Metrics

      Bibliometrics & Citations

      Bibliometrics

      Article Metrics

      • Downloads (Last 12 months)11
      • Downloads (Last 6 weeks)1
      Reflects downloads up to 27 Jan 2025

      Other Metrics

      Citations

      Cited By

      View all
      • (2024)HapticWhirl, a Flywheel-Gimbal Handheld Haptic Controller for Exploring Multimodal Haptic FeedbackSensors10.3390/s2403093524:3(935)Online publication date: 31-Jan-2024
      • (2024)Advancing Turbine Prediction: Harnessing Conformable Artificial Neural Networks for the Fracture AnalysisJournal of Failure Analysis and Prevention10.1007/s11668-024-02087-2Online publication date: 29-Dec-2024
      • (2020)Glissade: Generating Balance Shifting Feedback to Facilitate Auxiliary Digital Pen InputProceedings of the 2020 CHI Conference on Human Factors in Computing Systems10.1145/3313831.3376505(1-13)Online publication date: 21-Apr-2020

      View Options

      Login options

      View options

      PDF

      View or Download as a PDF file.

      PDF

      eReader

      View online with eReader.

      eReader

      Figures

      Tables

      Media

      Share

      Share

      Share this Publication link

      Share on social media