Abstract
The processing of shape is one of the most fundamental abilities of the human brain enabling us to efficiently recognize and interact with objects in the environment. To date, the vast majority of research into shape processing has been conducted in the visual modality. Given the importance of touch for interacting and manipulating objects—both in the early stage of human development as well as in general—here we argue that shape and object representations should be regarded as multisensory entities. Spurred by new developments in multisensory rendering technologies and rapid prototyping, recent research has advanced from the investigation of lower-level properties of multisensory processing to that of higher-level object processing. In this chapter, we present a framework for studying multisensory shape processing and discuss results from experiments based on this framework. Taken together, these research areas provide a more complete picture of how we see and grasp the world.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Borg I, Groenen P (2005) Modern multidimensional scaling, 2nd edn. Springer, Berlin
Casey SJ, Newell FN (2007) Are representations of unfamiliar faces independent of encoding modality? Neuropsychologia 45:506–513
Cooke T, Jäkel F, Wallraven C, Bülthoff HH (2007) Multimodal similarity and categorization of novel, three-dimensional objects. Neuropsychologia 45(3):484–495
Cutzu F, Edelman S (1998) Representation of object similarity in human vision: psychophysics and a computational model. Vis Res 38:2229–2257
Dahl CD, Wallraven C, Bülthoff HH, Logothetis NK (2009) Humans and macaques employ similar face-processing strategies. Curr Biol 19(6):509–513
de Heering A, Rossion B, Maurer D (2012) Developmental changes in face recognition during childhood: evidence from upright and inverted faces. Cogn Dev 27(1):17–27
DiCarlo JJ, Cox DD (2007) Untangling invariant object recognition. Trends Cogn Sci 11:333–341
Dopjans L, Bülthoff HH, Wallraven C (2012) Serial exploration of faces: comparing vision and touch. J Vis 12(1):6. (14 pp)
Dopjans L, Wallraven C, Bülthoff HH (2009) Cross-modal transfer in visual and haptic face recognition. IEEE Trans Haptics 200(4):236–240
Gaissert N, Bülthoff HH, Wallraven C (2011) Similarity and categorization: from vision to touch. Acta Psychol 138:219–230
Gaissert N, Wallraven C (2012) Categorizing natural objects: a comparison of the visual and the haptic modalities. Exp Brain Res 216:123–134
Gaissert N, Wallraven C, Bülthoff HH (2010) Visual and haptic perceptual spaces show high similarity in humans. J Vis 10(11):2. (20 pp)
Gallace A, Spence C (2009) The cognitive and neural correlates of tactile memory. Psychol Bull 135:380–406
Jäkel F, Schölkopf B, Wichmann FA (2009) Does cognitive science need kernels? Trends Cogn Sci 13:381–388
Kilgour AR, Lederman SJ (2002) Face recognition by hand. Percept Psychophys 64:339–352
Kilgour AR, Lederman SJ (2006) A haptic face-inversion effect. Perception 35:921–931
Kitada R, Johnsrude IS, Kochiyama T, Lederman SJ (2009) Functional specialization and convergence in the occipito-temporal cortex supporting haptic and visual identification of human faces and body parts: an fMRI study. J Cogn Neurosci 21:2027–2045
Lacey S, Campbell C, Sathian K (2007) Vision and touch: multiple or multisensory representations of objects? Perception 36:1513–1521
Lacey S, Tal N, Amedi A, Sathian K (2009) A putative model of multisensory object representation. Brain Topogr 21:269–274
Lederman S, Klatzky R (2009) Haptic perception: a tutorial. Atten Percept Psychophys 71(7):1439–1459
Lynott D, Connell L (2012) Modality exclusivity norms for 400 nouns: The relationship between perceptual experience and surface word form. Behav Res Methods, 1–11
Schwaninger A, Wallraven C, Cunningham DW, Chiller-Glaus S (2006) Processing of identity and emotion in faces: a psychophysical, physiological and computational perspective. Prog Brain Res 156:321–343
Schwarzer G (2000) Development of face processing: the effect of face inversion. Child Dev 71(2):391–401
Shepard R (1987) Toward a universal law of generalization for psychological science. Science 237(4820):1317–1323
Shepard R (2001) Perceptual-cognitive universals as reflections of the world. Behav Brain Sci 24(04):581–601
Sinha P, Balas B, Ostrovsky Y, Russell R (2006) Face recognition by humans: nineteen results all computer vision researchers should know about. Proc IEEE 94(11):1948–1962
Troje N, Bülthoff H (1996) Face recognition under varying poses: the role of texture and shape. Vis Res 36(12):1761–1771
Tversky A (1977) Features of similarity. Psychol Rev 84(4):327
Wallraven C, Dopjans L, Bülthoff HH (2012) Learning to recognize faces through serial exploration. Exp Brain Res 226(4):513–523
Acknowledgements
This work was done in collaboration with Theresa Cooke, Nina Gaißert, Lisa Dopjans, and Heinrich Bülthoff. It was supported by PhD stipends from the Max Planck Society, and by the WCU (World Class University) program through the National Research Foundation of Korea funded by the Ministry of Education, Science and Technology (R31-1008-000-10008-0).
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2013 Springer-Verlag London
About this chapter
Cite this chapter
Wallraven, C. (2013). Multisensory Shape Processing. In: Dickinson, S., Pizlo, Z. (eds) Shape Perception in Human and Computer Vision. Advances in Computer Vision and Pattern Recognition. Springer, London. https://doi.org/10.1007/978-1-4471-5195-1_32
Download citation
DOI: https://doi.org/10.1007/978-1-4471-5195-1_32
Publisher Name: Springer, London
Print ISBN: 978-1-4471-5194-4
Online ISBN: 978-1-4471-5195-1
eBook Packages: Computer ScienceComputer Science (R0)