Skip to main content
SpringerLink
Log in

Multimodal perception study on virtual 3D curved textures with vision and touch for interactive multimedia systems

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

Understanding the multimodal rendering of 3D shapes is becoming an important research topic as multimedia and virtual reality technologies are rapidly advancing. This study is aimed to investigate human perceptibility on the curvature and texture changes of 3D virtual surfaces across modalities, vision and touch. Our interest is to obtain perception data that can be used for 3D watermarking or data compression under a virtual reality environment providing multimodal interactions. For this study, we designed two psychophysical experiments to estimate curvature discrimination and texture detection thresholds on curvature surfaces over three conditions: vision only, touch only, and both vision and touch. The results show that touch is dominant at both discriminating curvature surfaces and detecting surface texture changes on a curved surface. In addition, the sensitivity of the both senses to detect texture changes linearly increases as a curvature value increases. Finally, the vision and touch senses compensate each other when both modalities are available at the same time. The thresholds from the present study can potentially be used as the upper limit for selecting watermark strengths or compression in order to ensure imperceptibility in a 3D visuohaptic multimedia systems.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8

Similar content being viewed by others

References

  1. Dostmohamed H, Hayward V (2005) Trajectory of contact region on the fingerpad gives the illusion of haptic shape. Exp Brain Res 164(3):387

    Article  Google Scholar 

  2. Ernst MO, Banks MS (2002) Humans integrate visual and haptic information in a statistically optimal fashion. Nature 415(6870):429

    Article  Google Scholar 

  3. Ernst MO, Banks MS, Grafton ST, Sathian K (2002) Humans integrate visual and haptic information in a statistically optimal fashion. Nature 415(6870):429

    Article  Google Scholar 

  4. Frisoli A, Solazzi M, Salsedo F, Bergamasco M (2008) A fingertip haptic display for improving curvature discrimination. Presence: Teleoperators and Virtual Environments 17(6):550

    Article  Google Scholar 

  5. Frizoli A, Barbagli R, Wu SL, Ruffaldi E, Bergamasco M, Salisbury K (2004) Evaluation of multipoint contact interfaces in haptic perception of shapes, Symposium of multipoint interaction, IEEE ICRA 2004, proc, 177–188

  6. Gibson JJ (1950) The perception of visual surfaces. Am J Psychol 63(3):367

    Article  Google Scholar 

  7. Gibson JJ (2014) The ecological approach to visual perception: classic edition, Psychology Press

  8. Gordon IE, Morison V (1982) The haptic perception of curvature. Percept Psychophys 31(5):446

    Article  Google Scholar 

  9. Helbig HB, Ernst MO (2007) Optimal integration of shape information from vision and touch. Exp Brain Res 179(4):595

    Article  Google Scholar 

  10. Hoffman DD, Richards WA (1984) Parts of recognition. Cognition 18(1):65

    Article  Google Scholar 

  11. Huffman DA (1977) Realizable configurations of lines in pictures of polyhedra. Machine Intelligence 8(493-509):2

    Google Scholar 

  12. Jansson G (2002) Pure-form deliverable d6: haptic exploration for perception of the shape of virtual objects, Tech. rep., Tech. Rep.. Upssala University

  13. Kang Z, Landry SJ (2015) An eye movement analysis algorithm for a multielement target tracking task: maximum transition-based agglomerative hierarchical clustering. IEEE Transactions on Human-Machine Systems 45(1):13

    Article  Google Scholar 

  14. Kim K (2015) Roughness based perceptual analysis towards digital skin imaging system with haptic feedback. Skin Res Technol 22(3):334–340

    Article  Google Scholar 

  15. Kim K, Barni M, Prattichizzo D, Tan HZ (2012) Visuohaptic discrimination of 3D gross shape. Seeing Perceiving 25(3-4):351

    Article  Google Scholar 

  16. Klatzky RL, Lederman SJ, Metzger VA (1985) Identifying objects by touch: an expert system. Percept Psychophys 37(4):299

    Article  Google Scholar 

  17. Koenderink JJ (1984) What does the occluding contour tell us about solid shape? Perception 13(3):321

    Article  Google Scholar 

  18. Lederman SJ, Klatzky RL (2004) Haptic identification of common objects: Effects of constraining the manual exploration process. Percept Psychophys 66(4):618

    Article  Google Scholar 

  19. Levitt H (1971) Transformed Up-Down methods in psychoacoustics. J Acoust Soc Am 49(2B):467

    Article  Google Scholar 

  20. Malik J (1987) Interpreting line drawings of curved objects. Int J Comput Vis 1 (1):73

    Article  Google Scholar 

  21. Malik J, Rosenholtz R (1997) Interpreting line drawings of curved objects. Int J Comput Vis 23(2):149

    Article  Google Scholar 

  22. Phong BT (1975) Illumination for computer generated pictures. Commun. ACM 18(6):311. doi:10.1145/360825.360839

    Article  Google Scholar 

  23. Pont SC, Kappers AM, Koenderink JJ (1999) Similar mechanisms underlie curvature comparison by static and dynamic touch. Percept Psychophys 61(5):874

    Article  Google Scholar 

  24. Provancher WR, Cutkosky MR, Kuchenbecker KJ, Niemeyer G (2005) Contact location display for haptic perception of curvature and object motion. Int J Robot Res 24(9):691

    Article  Google Scholar 

  25. Reed CL, Shoham S, Halgren E (2004) Neural substrates of tactile object recognition: an fMRI study. Hum Brain Mapp 21(4):236

    Article  Google Scholar 

  26. Rock I, Victor J (1964) Vision and touch: an experimentally created conflict between the two senses. Science 143 (3606):594. doi:10.1126/science.143.3606.594. http://www.sciencemag.org/cgi/content/abstract/143/3606/594

    Article  Google Scholar 

  27. Ruspini DC, Kolarov K, Khatib O (1997) The haptic display of complex graphical environments, SIGGRAPH ’97: proceedings of the 24th annual conference on computer graphics and interactive techniques, ACM Press/Addison-Wesley Publishing Co., New York, NY, USA, 345–352. doi:10.1145/258734.258878

  28. Singh M, Hoffman DD (1999) Completing visual contours: the relationship between relatability and minimizing inflections. Percept Psychophys 61(5):943

    Article  Google Scholar 

  29. Todd JT, Norman JF (2003) The visual perception of 3-D shape from multiple cues: are observers capable of perceiving metric structure? Percept Psychophys 65(1):31

    Article  Google Scholar 

  30. Tse PU (2002) A contour propagation approach to surface filling-in and volume formation. Psychol Rev 109(1):91

    Article  Google Scholar 

  31. Van Damme WJ, Oosterhoff FH, van de Grind WA (1994) Discrimination of 3-D shape and 3-D curvature from motion in active vision. Percept Psychophys 55 (3):340

  32. Vogels IM, Kappers AM, Koenderink JJ (1999) Influence of shape on haptic curvature perception. Acta Psychol 100(3):267

    Article  Google Scholar 

  33. Wijntjes MW, Sato A, Hayward V, Kappers AM (2009) Local surface orientation dominates haptic curvature discrimination. IEEE Trans Haptic 2(2):94

    Article  Google Scholar 

  34. Wijntjes MW, Sato A, Kappers AM, Hayward V (2008) Haptic perception of real and virtual curvature, Haptics: perception, devices and scenarios, Springer, 361–366

  35. Zangaladze A, Epstein CM, Grafton ST, Sathian K (1999) Involvement of visual cortex in tactile discrimination of orientation. Nature 401(6753):587

    Article  Google Scholar 

Download references

Acknowledgments

This work was supported by Incheon National University (International Cooperative) Research Grant in 2015 (grant no. 2015)

Author information

Authors and Affiliations

  1. School of Industrial and Systems Engineering, University of Oklahoma, Norman, OK, USA

    Ziho Kang

  2. Department of Information and Telecommunication Engineering, Incheon National University, Songdo, Incheon, Republic of Korea

    Kwangtaek Kim

Authors
  1. Ziho Kang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Kwangtaek Kim
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Kwangtaek Kim.

Additional information

Competing interests

The authors declare that they have no competing interests.

This paper or a similar version is not currently under review by a journal or conference, nor will it be submitted to such within the next three months. This paper is free of plagiarism or self-plagiarism as defined in Springer’s Policy on Publishing Integrity

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Kang, Z., Kim, K. Multimodal perception study on virtual 3D curved textures with vision and touch for interactive multimedia systems. Multimed Tools Appl 77, 2209–2223 (2018). https://doi.org/10.1007/s11042-017-4392-8

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-017-4392-8

Keywords

Search

Navigation