Abstract
Researchers have reported that audiovisual object presentation improves memory encoding of object identity in comparison to either auditory or visual object presentation. However, multisensory memory effects on retrieval, on object location, and of other multisensory combinations are yet unknown. We investigated the effects of visuotactile presentation on the encoding and retrieval of object identity memory and object location memory. Participants played an electronic memory card game consisting of an encoding and retrieval phase. In the encoding phase (c) they explored four game cards, which were presented on a computer screen in a two by two arrangement. Participants could touch each card to experience a Morse code presented on the screen (V) and/or via a tactile vibrator attached to the participant’s index finger (T). In the retrieval phase (r), they had to indicate for eight cards if (recognition) and where (relocation) these had been presented earlier. Compared with the visual base line (cV-rV), we found that both ‘multisensory encoding’ (cVT-rV) and ‘multisensory retrieval’ (cV-rVT) significantly improved both recognition and relocation performance. Compared with the tactile base line (cT-rT), we found no multisensory encoding or retrieval effects. This means that vision can benefit from adding touch but not vice versa. We conclude that visuotactile presentation improves memory encoding and retrieval of object identity and location. However, it is not yet clear whether these benefits are due to multisensory integration or simply due to the processing of the same information in multiple sensory modalities.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Ernst, M.O., Bülthoff, H.H.: Merging the senses into a robust percept. Trends Cogn. Sci. 8, 162–169 (2004)
Philippi, T.G., Van Erp, J.B.F., Werkhoven, P.J.: Multisensory temporal numerosity judgment. Brain Res. 1242, 116–125 (2008)
Spence, C., Driver, J.: Crossmodal Space and Crossmodal Attention. Oxford University Press, Oxford (2004)
Werkhoven, P.J., Van Erp, J.B.F., Philippi, T.G.: Counting of visual and tactile events: the effect of attention on multisensory integration. Atten. Percept. Psychophys. 71(8), 1854–1861 (2009)
Botta, F., Santangelo, V., Raffone, A., Sanabria, D., Lupiáñez, J., Belardinelli, M.O.: Multisensory integration affects visuo-spatial working memory. J. Exp. Psychol. Hum. Percept. Perform. 37(4), 1099–1109 (2011)
Delogu, F., Raffone, A., Olivetti Belardinelli, M.: Semantic encoding in working memory: is there a (multi)modality effect? Memory 17(7), 655–663 (2009)
Thompson, V.A., Paivio, A.: Memory for pictures and sounds: independence of auditory and visual codes. Can. J. Exp. Psychol. 48(3), 380–398 (1994)
Haas, E.C., Van Erp, J.B.F.: Multimodal warnings to enhance risk communication and safety. Saf. Sci. 61, 29–35 (2014)
Lehmann, S., Murray, M.M.: The role of multisensory memories in unisensory object discrimination. Cogn. Brain. Res. 24(2), 326–334 (2005)
Van Erp, J.B.F., Eriksson, L., Levin, B., Carlander, O., Veltman, J.A., Vos, W.K.: Tactile cueing effects on performance in simulated aerial combat with high acceleration. Aviat. Space Environ. Med. 78(12), 1128–1134 (2007)
Van Erp, J.B.F., Werkhoven, P.J.: Vibro-tactile and visual asynchronies: sensitivity and consistency. Perception 33, 103–111 (2004)
Murray, M.M., Michel, C.M., Grave de Peralta, R., Ortigue, S., Brunet, S., Gonzales Andino, S., Schnider, A.: Rapid discrimination of visual and multisensory memories revealed by electrical neuroimaging. Neuroimage 21(1), 125–135 (2004)
Von Kriegstein, K., Giraud, A.L.: Implicit multisensory associations influence voice recognition. PLoS Biol. 4, e326 (2006)
Loftus, G.R.: Comparison of recognition and recall in a continuous memory task. J. Exp. Psychol. 91, 220–226 (1971)
Tversky, B.: Encoding processes in recognition and recall. Cogn. Psychol. 5, 275–287 (1973)
Ernst, M.O., Lange, C., Newell, F.N.: Multisensory recognition of actively explored objects. Can. J. Exp. Psychol. 61(3), 242–253 (2007)
Kent, C., Lamberts, K.: The encoding–retrieval relationship: retrieval as mental simulation. Trends Cogn. Sci. 12, 92–98 (2008)
Tulving, E., Thompson, D.M.: Encoding specificity and retrieval processes in episodic memory. Psychol. Rev. 80, 352–373 (1973)
Shams, L., Seitz, A.: Benefits of multisensory learning. Trends Cogn. Sci. 12(11), 411–417 (2008)
Paivio, A.: Mental Representations: A Dual Coding Approach. Oxford University Press, Oxford. England (1986)
Van Erp, J.B.F.: Absolute localization of vibrotactile stimuli on the torso. Percept. Psychophys. 70(6), 1016–1023 (2008)
Hasher, L., Zacks, R.T.: Automatic and effortful processes in memory. J. Exp. Psychol. Gen. 108, 356–388 (1979)
Postma, A., Kessels, R.P.C., Van Asselen, M.: How the brain remembers and forgets where things are: the neurocognition of object-location memory. Neurosci. Biobehav. Rev. 32, 1339–1345 (2008)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2014 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
van Erp, J.B.F., Philippi, T.G., Werkhoven, P.J. (2014). Multisensory Memory for Object Identity and Location. In: Auvray, M., Duriez, C. (eds) Haptics: Neuroscience, Devices, Modeling, and Applications. EuroHaptics 2014. Lecture Notes in Computer Science(), vol 8618. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-662-44193-0_22
Download citation
DOI: https://doi.org/10.1007/978-3-662-44193-0_22
Published:
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-662-44192-3
Online ISBN: 978-3-662-44193-0
eBook Packages: Computer ScienceComputer Science (R0)