Skip to main content
Springer Nature Link
Log in

Creation of Realistic Haptic Experiences for Materialized Graphics

  • Conference paper
  • First Online:
Haptic Interaction (AsiaHaptics 2022)

Part of the book series: Lecture Notes in Computer Science ((LNCS,volume 14063))

Included in the following conference series:

  • 244 Accesses

Abstract

Aerial images that can interact with the hands and fingers inducing realistic tactile sensations and behave as though they are composed of physical substances are referred to as materialized graphics. Materialized graphics provides a natural interface that humans can handle and manipulate using skills that are inherent in any human. The technology also enables us to confirm and enjoy the tactile feeling and mediates human–human communication. This paper reports the ten-year progress of materialized graphics based on airborne ultrasound tactile displays. Early symbolic demonstrations of materialized graphics are presented, and the recent technological advances in haptic rendering that improve the realism are summarized. It explains the current non-contact display covers the sensations of static pressure and thermal interaction in addition to vibratory sensations.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Similar content being viewed by others

References

  1. Iwamoto, T., Tatezono, M., Shinoda, H.: Non-contact method for producing tactile sensation using airborne ultrasound. In: Proceedings of the Eurohaptics, pp. 504–513 (2008)

    Google Scholar 

  2. Hoshi, T., Takahashi, M., Iwamoto, T., Shinoda, H.: Noncontact tactile display based on radiation pressure of airborne ultrasound. IEEE Trans. Haptics 3(3), 155–165 (2010)

    Article  Google Scholar 

  3. Carter, T., Seah, S.A., Long, B., Drinkwater, B., Subramanian, S.: UltraHaptics: multi-point mid-air haptic feedback for touch surfaces. In: Proceeding UIST 2013, pp. 505–514 (2013)

    Google Scholar 

  4. Shinoda, H.: Tactile interaction with 3D images. In: The 17th International Display Workshops (IDW 2010), INP4: 3D Interactive Systems, pp. 1743–1746 (2010)

    Google Scholar 

  5. https://materialized-graphics.hapislab.org/. JST CREST, Materialized Graphics Project (since 2018)

  6. Hoshi, T., Takahashi, M., Nakatsuma, K., Shinoda, H.: Touchable holography. In: Proceeding of the ACM SIGGRAPH 2009 Emerging Technologies, New York, NY, USA, Article No. 23. ACM (2009)

    Google Scholar 

  7. Monnai, Y., Hasegawa, K., Fujiwara, M., Yoshino, K., Inoue, S., Shinoda, H.: HaptoMime: mid-air haptic interaction with a floating virtual screen. In: Proceeding of the 27th Annual ACM Symposium on User Interface Software and Technology (UIST 2014), pp. 663–667 (2014)

    Google Scholar 

  8. https://www.youtube.com/watch?v=Bb0hNMxxewg. “Visuo-Tactile Projector” video produced by Keisuke Hasegawa in Shinoda laboratory

  9. Hasegawa, K., Shinoda, H.: Aerial display of vibrotactile sensation with high spatial-temporal resolution using large-aperture airborne ultrasound phased array. In: Proceeding of the IEEE World Haptics Conference 2013, pp. 31–36 (2013)

    Google Scholar 

  10. Makino, Y., Furuyama, Y., Inoue, S., Shinoda, H.: HaptoClone (Haptic-Optical Clone) for mutual tele-environment by real-time 3D image transfer with midair force feedback. In: Proceeding of the 2016 CHI Conference on Human Factors in Computing Systems, pp. 1980–1990 (2016)

    Google Scholar 

  11. Rakkolainen, I., Freeman, E., Sand, A., Raisamo, R., Brewster, S.: A survey of mid-air ultrasound haptics and its applications 14(1), 2–19 (2021)

    Google Scholar 

  12. Hasegawa, K., Qiu, L., Noda, A., Inoue, S., Shinoda, H.: Electronically steerable ultrasound-driven long narrow air stream. Appl. Phys. Lett. 111(064104) (2017)

    Google Scholar 

  13. Ito, M., Kokumai, Y., Shinoda, H.: Midair click of dual-layer haptic button. In: Proceedings of the 2019 IEEE World Haptics Conference, Tokyo, Japan, 9–12 July 2019, pp. 349–352 (2019)

    Google Scholar 

  14. Suzuki, S., Fujiwara, M., Makino, Y., Shinoda, H.: Reducing amplitude fluctuation by gradual phase shift in midair ultrasound haptics. IEEE Trans. Haptics 13(1), 87–93 (2020)

    Google Scholar 

  15. Inoue, S., Makino, Y., Shinoda, H.: Mid-air ultrasonic pressure control on skin by adaptive focusing. In: Proceedings of the Eurohaptics, pp. 68–77, 4–8 July 2016, London, UK (2016)

    Google Scholar 

  16. Matsubayashi, A., Makino, Y., Shinoda, H.: Rendering ultrasound pressure distribution on hand surface in real-time. In: International Conference on Human Haptic Sensing and Touch Enabled Computer Applications (Euro Haptics), 6–9 September 2020, pp. 407–415 (2020)

    Google Scholar 

  17. Wyrowski, F.: Iterative quantization of digital amplitude holograms. Appl. Opt. 28(18), 3864–3870 (1989)

    Article  Google Scholar 

  18. Long, B., Seah, S.A., Carter, T., Subramanian, S.: Rendering volumetric haptic shapes in mid-air using ultrasound. ACM Trans. Graph. 33(6), 1–10 (2014)

    Article  Google Scholar 

  19. Marzo, A., Drinkwater, B.W.: Holographic acoustic tweezers. Proc. Nat. Acad. Sci. 116(1), 84–89 (2019)

    Article  Google Scholar 

  20. Plasencia, D.M., Hirayama, R., Montano-Murillo, R., Subramanian, S.: Gs-pat: high-speed multi-point sound-fields for phased arrays of transducers. ACM Trans. Graph 39(4) (2020)

    Google Scholar 

  21. Kakeya, H., Okada, K., Takahashi, H.: Time-division quadruplexing parallax barrier with subpixel-based slit control. ITE Trans. Media Technol. Appl. 6(3), 237–246 (2018)

    Google Scholar 

  22. Matsubayashi, A., Makino, Y., Shinoda, H.: Direct finger manipulation of 3D object image with ultrasound haptic feedback. In: Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems, Paper No. 87, pp. 1–11 (2019)

    Google Scholar 

  23. Matsubayashi, A., Oikawa, H., Mizutani, S., Makino, Y., Shinoda, H.: Display of haptic shape using ultrasound pressure distribution forming cross-sectional shape. In: Proceeding of the 2019 IEEE World Haptics Conference, Tokyo, Japan, 9–12 July 2019, pp. 419–424 (2019)

    Google Scholar 

  24. Matsubayashi, A., Yamaguchi, T., Makino, Y., Shinoda, H.: Rendering softness using airborne ultrasound. In: Proceedings of the 2021 IEEE World Haptics Conference, pp. 355–360 (2021)

    Google Scholar 

  25. Frier, W., et al.: Using spatiotemporal modulation to draw tactile patterns in mid-air. In: Proceedings of the EuroHaptics 2018, Part I, pp. 270–281 (2018)

    Google Scholar 

  26. Takahashi, R., Hasegawa, K., Shinoda, H.: Lateral modulation of midair ultrasound focus for intensified vibrotactile stimuli. In: Proceedings of the EuroHaptics 2018, Part II, pp. 276–288 (2018)

    Google Scholar 

  27. Takahashi, R., Hasegawa, K., Shinoda, H.: Tactile stimulation by repetitive lateral movement of midair ultrasound focus. IEEE Trans. Haptics 13(2), 334–342 (2020)

    Article  Google Scholar 

  28. Morisaki, T., Fujiwara, M., Makino, Y., Shinoda, H.: Non-vibratory pressure sensation produced by ultrasound focus moving laterally and repetitively with fine spatial step width. IEEE Trans. Haptics 15(2), 441–450 (2022)

    Google Scholar 

  29. Konyo, M., Tadokoro, S., Yoshida, A., Saiwaki, N.: A tactile synthesis method using multiple frequency vibrations for representing virtual touch. In: 2005 IEEE/RSJ International Conference on Intelligent Robots and Systems. IEEE, pp. 3965–3971 (2005)

    Google Scholar 

  30. Suzuki, S., Inoue, S., Fujiwara, M., Makino, Y., Shinoda, H.: AUTD3: scalable airborne ultrasound tactile display. IEEE Trans. Haptics 14(4), 740–749 (2021)

    Article  Google Scholar 

  31. Saga, S.: Thermal-radiation-based haptic display using laser-emission-based radiation control. In: Proceedings of the IEEE World Haptics 2019, WP2P.10 (Work-in-Progress Papers) (2019)

    Google Scholar 

  32. Nakajima, M., Hasegawa, K., Makino, Y., Shinoda, H.: Spatiotemporal pinpoint cooling sensation produced by ultrasound-driven mist vaporization on skin. IEEE Trans. Haptics 14(4), 874–884 (2021)

    Article  Google Scholar 

  33. Hasegawa, K., Qiu, L., Shinoda, H.: Midair ultrasound fragrance rendering. IEEE Trans. Vis. Comput. Graph. 24(4), 1477–1485 (2018)

    Article  Google Scholar 

  34. Rim, S., Suzuki, S., Toide, Y., Fujiwara, M., Makino, Y., Shinoda, H.: Sound-image icon with aerial haptic feedback. In: Proceedings of Euro Haptics 2020, pp. 497–505 (2020)

    Google Scholar 

  35. Ochiai, Y., Hoshi, T., Suzuki, I.: Holographic whisper: rendering audible sound spots in three-dimensional space by focusing ultrasonic waves. In: Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems, pp. 4314–4325 (2017)

    Google Scholar 

  36. Suzuki, S., Fujiwara, M., Makino, Y., Shinoda, H.: Midair hand guidance by an ultrasound virtual handrail. In: Proceedings of the 2019 IEEE World Haptics Conference, pp. 271–276 (2019)

    Google Scholar 

  37. Obrist, M., Subramanian, S., Gatti, E., Long, B., Carter, T.: Emotions mediated through mid-air haptics. In: Proceedings 33rd Annual ACM Conference on Human Factors in Computing Systems (CHI 2015), pp. 2053–2062 (2015)

    Google Scholar 

  38. Eid, M.A., Osman, H.A.: Affective haptics: current research and future directions. IEEE Access 26–40 (2016). https://doi.org/10.1109/ACCESS.2015.2497316

  39. Vi, C.T., Ablart, D., Gatti, E., Velasco, C., Obrist, M.: Not just seeing, but also feeling art: mid-air haptic experiences integrated in a multisensory art exhibition. Int. J. Hum.-Comput. Stud. 108, 1–14 (2017)

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. The University of Tokyo, 5-1-5 Kashiwano-Ha, Kashiwa, 277-8561, Chiba, Japan

    Hiroyuki Shinoda

Authors
  1. Hiroyuki Shinoda
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Hiroyuki Shinoda .

Editor information

Editors and Affiliations

  1. Beihang University, Beijing, China

    Dangxiao Wang

  2. Southeast University, Nanjing, China

    Aiguo Song

  3. Dalian University of Technology, Dalian, China

    Qian Liu

  4. Korea Advanced Institute of Science and Technology (KAIST), Daejeon, Korea (Republic of)

    Ki-Uk Kyung

  5. Tohoku University, Sendai, Japan

    Masashi Konyo

  6. The University of Electro-Communications, Tokyo, Tokyo, Japan

    Hiroyuki Kajimoto

  7. Peking University, Beijing, China

    Lihan Chen

  8. Korea Advanced Institute of Science and Technology (KAIST), Daejeon, Korea (Republic of)

    Jee-Hwan Ryu

Rights and permissions

Reprints and permissions

Copyright information

© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Shinoda, H. (2023). Creation of Realistic Haptic Experiences for Materialized Graphics. In: Wang, D., et al. Haptic Interaction. AsiaHaptics 2022. Lecture Notes in Computer Science, vol 14063. Springer, Cham. https://doi.org/10.1007/978-3-031-46839-1_4

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-46839-1_4

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-46838-4

  • Online ISBN: 978-3-031-46839-1

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics