Marco Kurzweg
Katrin Wolf
ABSTRACT
Vibrations are the dominant way to create haptic feedback for interactive systems and are most often induced by vibrotactile actuators. However, virtual content created for augmented reality usually does not support that modality, instead relying mainly on visual and auditive output. Aiming to provide haptic feedback for augmented reality in cases where real vibrations cannot be used, we explore how vibrations can be felt using vision and audio only. In a user study, a virtual 10 x 10 cm white square-shaped cuboid was influenced by animation and/or sound to induce a haptic illusion when being touched. We were able to identify a specific range where the perception of vibration was significantly stronger and more realistic compared to all other values. This was the case if the virtual object’s edges were blurred up to a range of 0.4 cm or 0.6 cm, correspondingly accompanied by sounds, where the spectrum was cut off at a frequency of 256 Hz (for 0.4 cm) or 966 Hz (for 0.6 cm). With that, we aim to enrich augmented reality systems.
Supplemental Material
Available for Download
- Florian Anwander. 2000. Synthesizer: so funktioniert elektronische Klangerzeugung-erklärt am Beispiel modularer Synthesizer. PPV, Presse-Project-Verlag-GmbH, Feldgeding, Germany.Google Scholar
- Ferran Argelaguet, David Antonio Gómez Jáuregui, Maud Marchal, and Anatole Lécuyer. 2013. Elastic Images: Perceiving Local Elasticity of Images through a Novel Pseudo-Haptic Deformation Effect. ACM Trans. Appl. Percept. 10, 3, Article 17 (aug 2013), 14 pages. https://doi.org/10.1145/2501599Google ScholarDigital Library
- Yuki Ban, Takuji Narumi, Tomohiro Tanikawa, and Michitaka Hirose. 2014. Displaying Shapes with Various Types of Surfaces Using Visuo-Haptic Interaction. In Proceedings of the 20th ACM Symposium on Virtual Reality Software and Technology (Edinburgh, Scotland) (VRST ’14). Association for Computing Machinery, New York, NY, USA, 191–196. https://doi.org/10.1145/2671015.2671028Google ScholarDigital Library
- Joanna Bergström, Aske Mottelson, and Jarrod Knibbe. 2019. Resized Grasping in VR: Estimating Thresholds for Object Discrimination. In Proceedings of the 32nd Annual ACM Symposium on User Interface Software and Technology (New Orleans, LA, USA) (UIST ’19). Association for Computing Machinery, New York, NY, USA, 1175–1183. https://doi.org/10.1145/3332165.3347939Google ScholarDigital Library
- Carlos Bermejo and Pan Hui. 2021. A Survey on Haptic Technologies for Mobile Augmented Reality. ACM Comput. Surv. 54, 9, Article 184 (oct 2021), 35 pages. https://doi.org/10.1145/3465396Google ScholarDigital Library
- Raoul Bickmann, Celine Tran, Ninja Ruesch, and Katrin Wolf. 2019. Haptic Illusion Glove: A Glove for Illusionary Touch Feedback When Grasping Virtual Objects. In Proceedings of Mensch Und Computer 2019 (Hamburg, Germany) (MuC ’19). Association for Computing Machinery, New York, NY, USA, 565–569. https://doi.org/10.1145/3340764.3344459Google ScholarDigital Library
- James V Bradley. 1958. Complete counterbalancing of immediate sequential effects in a Latin square design. J. Amer. Statist. Assoc. 53, 282 (1958), 525–528.Google ScholarCross Ref
- Antoine Chaigne, Cyril Touzé, and Olivier Thomas. 2005. Nonlinear vibrations and chaos in gongs and cymbals. Acoustical science and technology 26, 5 (2005), 403–409.Google Scholar
- Youngjun Cho, Andrea Bianchi, Nicolai Marquardt, and Nadia Bianchi-Berthouze. 2016. RealPen: Providing Realism in Handwriting Tasks on Touch Surfaces Using Auditory-Tactile Feedback. In Proceedings of the 29th Annual Symposium on User Interface Software and Technology (Tokyo, Japan) (UIST ’16). Association for Computing Machinery, New York, NY, USA, 195–205. https://doi.org/10.1145/2984511.2984550Google ScholarDigital Library
- Fabien Danieau, Anatole Lecuyer, Philippe Guillotel, Julien Fleureau, Nicolas Mollet, and Marc Christie. 2013. Enhancing Audiovisual Experience with Haptic Feedback: A Survey on HAV. EEE Trans. Haptics 6, 2 (apr 2013), 193–205. https://doi.org/10.1109/TOH.2012.70Google ScholarDigital Library
- Marc O. Ernst. 2005. The "Puzzle" of Sensory Perception: Putting Together Multisensory Information. In Proceedings of the 7th International Conference on Multimodal Interfaces (Torento, Italy) (ICMI ’05). Association for Computing Machinery, New York, NY, USA, 1. https://doi.org/10.1145/1088463.1088464Google ScholarDigital Library
- Marc O Ernst and Martin S Banks. 2002. Humans integrate visual and haptic information in a statistically optimal fashion. Nature 415, 6870 (2002), 429–433.Google Scholar
- Roberta Etzi, Francesco Ferrise, Monica Bordegoni, Massimiliano Zampini, and Alberto Gallace. 2018. The Effect of Visual and Auditory Information on the Perception of Pleasantness and Roughness of Virtual Surfaces. Multisensory Research 31, 6 (2018), 501 – 522.Google ScholarCross Ref
- Roberta Etzi, Francesco Ferrise, Monica Bordegoni, Massimiliano Zampini, and Alberto Gallace. 2018. The Effect of Visual and Auditory Information on the Perception of Pleasantness and Roughness of Virtual Surfaces. Multisensory Research 31, 6 (2018), 501 – 522.Google ScholarCross Ref
- Emma Fallows, David White, and Neil Brownsword. 2022. Design and Development Approach for an Interactive Virtual Museum with Haptic Glove Technology. In Proceedings of the 25th International Academic Mindtrek Conference (Tampere, Finland) (Academic Mindtrek ’22). Association for Computing Machinery, New York, NY, USA, 242–255. https://doi.org/10.1145/3569219.3569382Google ScholarDigital Library
- Sean Follmer, Daniel Leithinger, Alex Olwal, Akimitsu Hogge, and Hiroshi Ishii. 2013. InFORM: Dynamic Physical Affordances and Constraints through Shape and Object Actuation. In Proceedings of the 26th Annual ACM Symposium on User Interface Software and Technology (St. Andrews, Scotland, United Kingdom) (UIST ’13). Association for Computing Machinery, New York, NY, USA, 417–426. https://doi.org/10.1145/2501988.2502032Google ScholarDigital Library
- Brian Gleeson and David E. Johnson. 2010. Expressive haptic rendering with cartoon-inspired effects. In 2010 IEEE Haptics Symposium. IEEE, Waltham, MA, USA, 191–194. https://doi.org/10.1109/HAPTIC.2010.5444656Google ScholarDigital Library
- Mar Gonzalez-Franco and Christopher C. Berger. 2019. Avatar Embodiment Enhances Haptic Confidence on the Out-of-Body Touch Illusion. IEEE Transactions on Haptics 12, 3 (2019), 319–326. https://doi.org/10.1109/TOH.2019.2925038Google ScholarDigital Library
- Monica Gori, Sara Price, Fiona N Newell, Nadia Berthouze, and Gualtiero Volpe. 2022. Multisensory Perception and Learning: Linking Pedagogy, Psychophysics, and Human–Computer Interaction. Multisensory Research 35, 4 (2022), 335–366.Google ScholarCross Ref
- Taku Hachisu, Gabriel Cirio, Maud Marchal, Anatole Lécuyer, and Hiroyuki Kajimoto. 2011. Pseudo-haptic feedback augmented with visual and tactile vibrations. In 2011 IEEE International Symposium on VR Innovation. IEEE, IEEE, Singapore, 327–328.Google ScholarCross Ref
- Taku Hachisu and Kenji Suzuki. 2019. Representing Interpersonal Touch Directions by Tactile Apparent Motion Using Smart Bracelets. IEEE Transactions on Haptics 12, 3 (2019), 327–338. https://doi.org/10.1109/TOH.2019.2929810Google ScholarDigital Library
- Satoshi Hashiguchi, Shohei Mori, Miho Tanaka, Fumihisa Shibata, and Asako Kimura. 2018. Perceived Weight of a Rod under Augmented and Diminished Reality Visual Effects. In Proceedings of the 24th ACM Symposium on Virtual Reality Software and Technology (Tokyo, Japan) (VRST ’18). Association for Computing Machinery, New York, NY, USA, Article 12, 6 pages. https://doi.org/10.1145/3281505.3281545Google ScholarDigital Library
- Seongkook Heo and Geehyuk Lee. 2017. Vibrotactile Compliance Feedback for Tangential Force Interaction. IEEE Transactions on Haptics 10, 3 (2017), 444–455. https://doi.org/10.1109/TOH.2016.2604305Google ScholarDigital Library
- Brett R. Jones, Hrvoje Benko, Eyal Ofek, and Andrew D. Wilson. 2013. IllumiRoom: Peripheral Projected Illusions for Interactive Experiences. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (Paris, France) (CHI ’13). Association for Computing Machinery, New York, NY, USA, 869–878. https://doi.org/10.1145/2470654.2466112Google ScholarDigital Library
- Namkyoo Kang, Young June Sah, and Sangwon Lee. 2021. Effects of visual and auditory cues on haptic illusions for active and passive touches in mixed reality. International Journal of Human-Computer Studies 150 (2021), 102613.Google ScholarCross Ref
- Wilfried Kausel. 2018. Vibrations and Waves. Springer, Berlin,Heidelberg, 29–47. https://doi.org/10.1007/978-3-662-55004-5_2Google ScholarCross Ref
- Patty Kero and Daniel Lee. 2015. Slider Scales and Web-Based Surveys: A Cautionary Note. Journal of Research Practice 11 (09 2015).Google Scholar
- Jinsoo Kim, Seungjae Oh, Chaeyong Park, and Seungmoon Choi. 2020. Body-Penetrating Tactile Phantom Sensations. Association for Computing Machinery, New York, NY, USA, 1–13. https://doi.org/10.1145/3313831.3376619Google ScholarDigital Library
- Arata Kokubun, Yuki Ban, Takuji Narumi, Tomohiro Tanikawa, and Michitaka Hirose. 2013. ARAtouch: Visuo-Haptic Interaction with Mobile Rear Touch Interface. In SIGGRAPH Asia 2013 Emerging Technologies (Hong Kong, Hong Kong) (SA ’13). Association for Computing Machinery, New York, NY, USA, Article 2, 3 pages. https://doi.org/10.1145/2542284.2542286Google ScholarDigital Library
- Marco Kurzweg, Maximilian Letter, and Katrin Wolf. 2023. Increasing Realism of Displayed Vibrating AR Objects through Edge Blurring. In Proceedings of Mensch Und Computer 2023 (Rapperswil, Switzerland) (MuC ’23). Association for Computing Machinery, New York, NY, USA, 16–26. https://doi.org/10.1145/3603555.3603570Google ScholarDigital Library
- Chi-Hsia Lai, Matti Niinimäki, Koray Tahiroglu, Johan Kildal, and Teemu Ahmaniemi. 2011. Perceived Physicality in Audio-Enhanced Force Input. In Proceedings of the 13th International Conference on Multimodal Interfaces (Alicante, Spain) (ICMI ’11). Association for Computing Machinery, New York, NY, USA, 287–294. https://doi.org/10.1145/2070481.2070533Google ScholarDigital Library
- Anatole Lécuyer. 2009. Simulating Haptic Feedback Using Vision: A Survey of Research and Applications of Pseudo-Haptic Feedback. Presence: Teleoper. Virtual Environ. 18, 1 (jan 2009), 39–53. https://doi.org/10.1162/pres.18.1.39Google ScholarDigital Library
- Anatole Lécuyer, Jean-Marie Burkhardt, and Laurent Etienne. 2004. Feeling Bumps and Holes without a Haptic Interface: The Perception of Pseudo-Haptic Textures. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (Vienna, Austria) (CHI ’04). Association for Computing Machinery, New York, NY, USA, 239–246. https://doi.org/10.1145/985692.985723Google ScholarDigital Library
- Shang Ping Lee, Tran Cong Thien Qui, Shing Chuan Loy, and William Russell Pensyl. 2009. Haptic Interaction in Augmented Reality. In Proceedings of the 17th ACM International Conference on Multimedia (Beijing, China) (MM ’09). Association for Computing Machinery, New York, NY, USA, 975–976. https://doi.org/10.1145/1631272.1631475Google ScholarDigital Library
- Pedro Lopes, Sijing You, Alexandra Ion, and Patrick Baudisch. 2018. Adding Force Feedback to Mixed Reality Experiences and Games Using Electrical Muscle Stimulation. In Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems (Montreal QC, Canada) (CHI ’18). Association for Computing Machinery, New York, NY, USA, 1–13. https://doi.org/10.1145/3173574.3174020Google ScholarDigital Library
- Koert Van Mensvoort, Peter Vos, Dik J. Hermes, and Robert Van Liere. 2010. Perception of Mechanically and Optically Simulated Bumps and Holes. ACM Trans. Appl. Percept. 7, 2, Article 10 (feb 2010), 24 pages. https://doi.org/10.1145/1670671.1670674Google ScholarDigital Library
- Forrest W Nutter Jr. 2010. Weber-Fechner Law. Encyclopedia of research design 3 (2010), 1612–1615.Google Scholar
- Florian Perteneder, Kathrin Probst, Joanne Leong, Sebastian Gassler, Christian Rendl, Patrick Parzer, Katharina Fluch, Sophie Gahleitner, Sean Follmer, Hideki Koike, and Michael Haller. 2020. Foxels: Build Your Own Smart Furniture. In Proceedings of the Fourteenth International Conference on Tangible, Embedded, and Embodied Interaction (Sydney NSW, Australia) (TEI ’20). Association for Computing Machinery, New York, NY, USA, 111–122. https://doi.org/10.1145/3374920.3374935Google ScholarDigital Library
- Andreas Pusch and Anatole Lécuyer. 2011. Pseudo-Haptics: From the Theoretical Foundations to Practical System Design Guidelines. In Proceedings of the 13th International Conference on Multimodal Interfaces (Alicante, Spain) (ICMI ’11). Association for Computing Machinery, New York, NY, USA, 8 pages. https://doi.org/10.1145/2070481.2070494Google ScholarDigital Library
- Michael Rietzler, Florian Geiselhart, Julian Frommel, and Enrico Rukzio. 2018. Conveying the Perception of Kinesthetic Feedback in Virtual Reality Using State-of-the-Art Hardware. In Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems (Montreal QC, Canada) (CHI ’18). Association for Computing Machinery, New York, NY, USA, 1–13. https://doi.org/10.1145/3173574.3174034Google ScholarDigital Library
- Michael Rietzler, Gabriel Haas, Thomas Dreja, Florian Geiselhart, and Enrico Rukzio. 2019. Virtual Muscle Force: Communicating Kinesthetic Forces Through Pseudo-Haptic Feedback and Muscle Input. In Proceedings of the 32nd Annual ACM Symposium on User Interface Software and Technology (New Orleans, LA, USA) (UIST ’19). Association for Computing Machinery, New York, NY, USA, 913–922. https://doi.org/10.1145/3332165.3347871Google ScholarDigital Library
- Majed Samad, Elia Gatti, Anne Hermes, Hrvoje Benko, and Cesare Parise. 2019. Pseudo-Haptic Weight: Changing the Perceived Weight of Virtual Objects By Manipulating Control-Display Ratio. In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems (Glasgow, Scotland Uk) (CHI ’19). Association for Computing Machinery, New York, NY, USA, 1–13. https://doi.org/10.1145/3290605.3300550Google ScholarDigital Library
- Carl Schissler, Aaron Nicholls, and Ravish Mehra. 2016. Efficient HRTF-based Spatial Audio for Area and Volumetric Sources. IEEE Transactions on Visualization and Computer Graphics 22, 4 (2016), 1356–1366. https://doi.org/10.1109/TVCG.2016.2518134Google ScholarDigital Library
- Jonas Schmidtler and Klaus Bengler. 2018. Influence of Size-Weight Illusion on Usability in Haptic Human-Robot Collaboration. IEEE Transactions on Haptics 11, 1 (2018), 85–96. https://doi.org/10.1109/TOH.2017.2757925Google ScholarCross Ref
- Scott Sinnett, Charles Spence, and Salvador Soto-Faraco. 2007. Visual dominance and attention: The Colavita effect revisited. Perception & Psychophysics 69, 5 (01 Jul 2007), 673–686. https://doi.org/10.3758/BF03193770Google ScholarCross Ref
- Marco Speicher, Jan Ehrlich, Vito Gentile, Donald Degraen, Salvatore Sorce, and Antonio Krüger. 2019. Pseudo-Haptic Controls for Mid-Air Finger-Based Menu Interaction. In Extended Abstracts of the 2019 CHI Conference on Human Factors in Computing Systems (Glasgow, Scotland Uk) (CHI EA ’19). Association for Computing Machinery, New York, NY, USA, 1–6. https://doi.org/10.1145/3290607.3312927Google ScholarDigital Library
- Stanley S Stevens. 1960. The psychophysics of sensory function. American scientist 48, 2 (1960), 226–253.Google Scholar
- Patrick L. Strandholt, Oana A. Dogaru, Niels C. Nilsson, Rolf Nordahl, and Stefania Serafin. 2020. Knock on Wood: Combining Redirected Touching and Physical Props for Tool-Based Interaction in Virtual Reality. In Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems (Honolulu, HI, USA) (CHI ’20). Association for Computing Machinery, New York, NY, USA, 1–13. https://doi.org/10.1145/3313831.3376303Google ScholarDigital Library
- Anselm Strauss and Juliet M Corbin. 1997. Grounded theory in practice. Sage, Thousand Oaks, CA, USA.Google Scholar
- Takeshi Tanabe, Hiroaki Yano, and Hiroo Iwata. 2018. Evaluation of the Perceptual Characteristics of a Force Induced by Asymmetric Vibrations. IEEE Transactions on Haptics 11, 2 (2018), 220–231. https://doi.org/10.1109/TOH.2017.2743717Google ScholarCross Ref
- Luca Turchet, Maud Marchal, Anatole Lécuyer, Rolf Nordahl, and Stefania Serafin. 2010. Influence of Auditory and Visual Feedback for Perceiving Walking over Bumps and Holes in Desktop VR. In Proceedings of the 17th ACM Symposium on Virtual Reality Software and Technology (Hong Kong) (VRST ’10). Association for Computing Machinery, New York, NY, USA, 139–142. https://doi.org/10.1145/1889863.1889893Google ScholarDigital Library
- Luca Turchet, Stefania Serafin, and Paola Cesari. 2013. Walking Pace Affected by Interactive Sounds Simulating Stepping on Different Terrains. ACM Trans. Appl. Percept. 10, 4, Article 23 (oct 2013), 14 pages. https://doi.org/10.1145/2536764.2536770Google ScholarDigital Library
- Yusuke Ujitoko and Yuki Ban. 2021. Survey of Pseudo-Haptics: Haptic Feedback Design and Application Proposals. IEEE Transactions on Haptics 14, 4 (2021), 699–711. https://doi.org/10.1109/TOH.2021.3077619Google ScholarDigital Library
- Yusuke Ujitoko, Yuki Ban, Takuji Narumi, Tomohiro Tanikawa, Koichi Hirota, and Michitaka Hirose. 2015. Yubi-Toko: Finger Walking in Snowy Scene Using Pseudo-Haptic Technique on Touchpad. In SIGGRAPH Asia 2015 Emerging Technologies (Kobe, Japan) (SA ’15). Association for Computing Machinery, New York, NY, USA, Article 29, 3 pages. https://doi.org/10.1145/2818466.2818491Google ScholarDigital Library
- Lia Villanueva and Massimiliano Zampini. 2018. Reciprocal Interference Between Audition and Touch in the Perception of Duration. Multisensory Research 31, 5 (2018), 351 – 371.Google ScholarCross Ref
- Chris Waltham and Shigeru Yoshikawa. 2018. Construction of Wooden Musical Instruments. Springer Berlin Heidelberg, Berlin, Heidelberg, 63–79. https://doi.org/10.1007/978-3-662-55004-5_4Google ScholarCross Ref
- Mark Weiser. 1991. The Computer for the 21 st Century. Scientific american 265, 3 (1991), 94–105.Google Scholar
- Mark Weiser and John Seely Brown. 1996. Designing calm technology. PowerGrid Journal 1, 1 (1996), 75–85.Google Scholar
- Katrin Wolf and Timm Bäder. 2015. Illusion of Surface Changes Induced by Tactile and Visual Touch Feedback. In Proceedings of the 33rd Annual ACM Conference Extended Abstracts on Human Factors in Computing Systems (Seoul, Republic of Korea) (CHI EA ’15). Association for Computing Machinery, New York, NY, USA, 1355–1360. https://doi.org/10.1145/2702613.2732703Google ScholarDigital Library
- Hong-In Won and M. Ercan Altinsoy. 2020. Effect of Auditory Feedback on Tactile Intensity Perception in a Touchscreen Application. IEEE Transactions on Haptics 13, 2 (2020), 343–353. https://doi.org/10.1109/TOH.2019.2947553Google ScholarDigital Library
- Jackie (Junrui) Yang, Hiroshi Horii, Alexander Thayer, and Rafael Ballagas. 2018. VR Grabbers: Ungrounded Haptic Retargeting for Precision Grabbing Tools. In Proceedings of the 31st Annual ACM Symposium on User Interface Software and Technology (Berlin, Germany) (UIST ’18). Association for Computing Machinery, New York, NY, USA, 889–899. https://doi.org/10.1145/3242587.3242643Google ScholarDigital Library
- Y. Yokokohji, R.L. Hollis, and T. Kanade. 1996. What you can see is what you can feel-development of a visual/haptic interface to virtual environment. In Proceedings of the IEEE 1996 Virtual Reality Annual International Symposium. IEEE, Santa Clara, CA, USA, 46–53. https://doi.org/10.1109/VRAIS.1996.490509Google ScholarCross Ref
- Maki Yokoyama, Taku Hachisu, Michi Sato, Shogo Fukushima, and Hiroyuki Kajimoto. 2012. Control of Ridge by Using Visuotactile Cross-Modal Phenomenon. In Proceedings of the 2012 ACM International Conference on Interactive Tabletops and Surfaces (Cambridge, Massachusetts, USA) (ITS ’12). Association for Computing Machinery, New York, NY, USA, 335–338. https://doi.org/10.1145/2396636.2396694Google ScholarDigital Library
- HP ZENNER. 2013. Die Kommunikation des Menschen. Springer-Verlag, Berlin,Heidelberg, 259–277.Google Scholar
Index Terms
- Vibrollusion: Creating a Vibrotactile Illusion Induced by Audiovisual Touch Feedback
Recommendations
Augmenting Physical Buttons with Vibrotactile Feedback for Programmable Feels
UIST '20: Proceedings of the 33rd Annual ACM Symposium on User Interface Software and TechnologyPhysical buttons provide clear haptic feedback when pressed and released, but their responses are unvarying. Physical buttons can be powered by force actuators to produce unlimited click sensations, but the cost is substantial. An alternative can be ...
Tactile Information Transmission by 2D Stationary Phantom Sensations
CHI '18: Proceedings of the 2018 CHI Conference on Human Factors in Computing SystemsA phantom sensation refers to an illusory tactile sensation perceived midway between multiple distant stimulations on the skin. Phantom sensations have been used intensively in tactile interfaces owing to their simplicity and effectiveness. Despite that,...
Wearable head-mounted 3D tactile display application scenarios
MobileHCI '16: Proceedings of the 18th International Conference on Human-Computer Interaction with Mobile Devices and Services AdjunctCurrent generation virtual reality (VR) and augmented reality (AR) head-mounted displays (HMDs) usually include no or only a single vibration motor for haptic feedback and do not use it for guidance. In a previous work, we presented HapticHead, a ...
Comments