Skip to main content
SpringerLink
Log in

The Bayesian Causal Inference in Multisensory Information Processing: A Narrative Review

  • Conference paper
  • First Online:
Recent Advances in Intelligent Information Hiding and Multimedia Signal Processing (IIH-MSP 2018)

Part of the book series: Smart Innovation, Systems and Technologies ((SIST,volume 109))

Abstract

When processing the simultaneous multisensory information, the brain must first infer whether the information comes from the same object, which is a prerequisite for multisensory information processing. The Bayesian causal inference can effectively simulate the inference process in the brain and predict the results. This paper reviews the research of multisensory information processing based on Bayesian causal inference, introduces the Bayesian causal inference theory in multisensory information processing, explains the multisensory information processing based on this theory in detail, analyzed the factors influencing the causal inference and the future research direction, in order to enhance the new understanding of the brain-like model for multisensory information processing, and to provide reference for the research of multisensory information processing in future.

This work was supported by the National Natural Science Foundation of China (grant numbers 61773076, 61806025) and the Jilin Scientific and Technological Development Program of China (grant number 20180519012JH).

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 169.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 219.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD 219.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Kayser, C., Shams, L.: Multisensory causal inference in the brain. PLoS Biol. 13(2), e1002075 (2015)

    Article  Google Scholar 

  2. Tong, J., Parisi, G.I., Wermter, S.: Closing the loop on multisensory interactions: a neuralarchitecture for multisensory causal inference and recalibration (2018)

    Google Scholar 

  3. Körding, K.P., Beierholm, U., Ma, W.J., et al.: Causal inference in multisensory perception. PLoS One 2(9), e943 (2007)

    Article  Google Scholar 

  4. Shams, L., Beierholm, U.R.: Causal inference in perception. Trends Cogn. Sci. 14(9), 425–432 (2010)

    Article  Google Scholar 

  5. Spence, C., Squire, S.: Multisensory integration: maintaining the perception of synchrony. Curr. Biol. 13(13), 519–521 (2003)

    Article  Google Scholar 

  6. Ernst, M.O., Bülthoff, A.H.H.: Merging the senses into a robust percept. Trends Cogn. Sci. 8(4), 162–169 (2004)

    Article  Google Scholar 

  7. De, G.B., Bertelson, P.: Multisensory integration, perception and ecological validity. Trends Cogn. Sci. 7(10), 460–467 (2003)

    Article  Google Scholar 

  8. Winkel, K.N.D., Katliar, M., Bülthoff, H.H.: Forced fusion in multisensory heading estimation. PLoS One 10(5), e0127104 (2015)

    Article  Google Scholar 

  9. Bresciani, J.P., Dammeier, F., Ernst, M.O.: Vision and touch are automatically integrated for the perception of sequences of events. J. Vis. 6(5), 554–564 (2006)

    Article  Google Scholar 

  10. Stevenson, I., Koerding, K.: Structural inference affects depth perception in the context of potential occlusion. Adv. Neural Inf. Process. Syst. 1777–1784 (2009)

    Google Scholar 

  11. Shams, L., Ma, W.J., Beierholm, U.: Sound-induced flash illusion as an optimal percept. NeuroReport 16(17), 1923–1927 (2005)

    Article  Google Scholar 

  12. Samad, M., Chung, A.J., Shams, L.: Perception of body ownership is driven by Bayesian sensory inference. PLoS One 10(2), e0117178 (2015)

    Article  Google Scholar 

  13. Mendonça, C., Mandelli, P., Pulkki, V.: Modeling the perception of audiovisual distance: Bayesian causal inference and other models. PLoS One 11(12) (2016)

    Article  Google Scholar 

  14. Ursino, M., Cuppini, C., Magosso, E.: Neurocomputational approaches to modelling multisensory integration in the brain: a review. Neural Netw. 60, 141–165 (2014)

    Article  Google Scholar 

  15. Battaglia, P.W., Jacobs, R.A., Aslin, R.N.: Bayesian integration of visual and auditory signals for spatial localization. J. Opt. Soc. Am. Opt. Image Sci. Vis. 20(7), 1391–1397 (2003)

    Article  Google Scholar 

  16. Rowland, B., Stanford, T., Stein, B.: A bayesian model unifies multisensory spatial localization with the physiological properties of the superior colliculus. Exp. Brain Res. 180(1), 153–161 (2007)

    Article  Google Scholar 

  17. Ernst, M.O.: A bayesian view on multimodal cue integration. Behav. Brain Sci. (2006)

    Google Scholar 

  18. Samad, M., Chung, A.J., Shams, L.: Perception of body ownership is driven by bayesian sensory inference. PLoS One 10(2), e0117178 (2015)

    Article  Google Scholar 

  19. Brian, O., Wozny, D.R., Ladan, S.: Biases in visual, auditory, and audiovisual perception of space. PLoS Comput. Biol. 11(12), e1004649 (2015)

    Article  Google Scholar 

  20. Rohe, T., Noppeney, U.: Cortical hierarchies perform Bayesian causal inference in multisensory perception. PLoS Biol. 13(2), e1002073 (2015)

    Article  Google Scholar 

  21. Rohe, T., Noppeney, U.: Sensory reliability shapes perceptual inference via two mechanisms. J. Vis. 15(5), 22 (2015)

    Article  Google Scholar 

  22. Mahani, M.N., Sheybani, S., Bausenhart, K.M.: multisensory perception of contradictory information in an environment of varying reliability: evidence for conscious perception and optimal causal inference. Sci. Rep. 7(1), 3167 (2017)

    Google Scholar 

  23. Roach, N.W., James, H., Mcgraw, P.V.: Resolving multisensory conflict: a strategy for balancing the costs and benefits of audio-visual integration. Proc. Biol. Sci. 273(1598), 2159–2168 (2006)

    Article  Google Scholar 

  24. Wozny, D.R., Beierholm, U.R., Shams, L.: Probability matching as a computational strategy used in perception. PLoS Comput. Biol. 6(8), 861–864 (2010)

    Article  MathSciNet  Google Scholar 

  25. Drugowitsch, J., Deangelis, G.C., Angelaki, D.E.: Tuning the speed-accuracy trade-off to maximize reward rate in multisensory decision-making. Elife Sci. 4(2015)

    Google Scholar 

  26. Ernst, M.O., Banks, M.S.: Humans integrate visual and haptic information in a statistically optimal fashion. Nature 415(6870), 429–433 (2002)

    Article  Google Scholar 

  27. Girshick, A.R., Banks, M.S.: Probabilistic combination of slant information: weighted averaging and robustness as optimal percepts. J. Vis. 9(9), 1–20 (2009)

    Article  Google Scholar 

  28. Gepshtein, S., Burge, J., Ernst, M.O.: The combination of vision and touch depends on spatial proximity. J. Vis. 5(11), 1013 (2005)

    Article  Google Scholar 

  29. Alais, D., Burr, D.: The ventriloquist effect results from near-optimal bimodal integration. Curr. Biol. Cb 14(3), 257–262 (2004)

    Article  Google Scholar 

  30. Magnotti, J.F., Beauchamp, M.S: A causal inference model explains perception of the McGurk effect and other incongruent audiovisual speech. Plos Comput. Biol. 13(2), e1005229 (2017)

    Article  Google Scholar 

  31. Daemi, M., Harris, L.R., Crawford, J.D.: Causal inference for cross-modal action selection: a computational study in a decision making framework. Front. Comput. Neurosci. 10(11) (2016)

    Google Scholar 

  32. Locke, S.M., Landy, M.S.: Temporal causal inference with stochastic audiovisual sequences. PLoS One 12(9), e0183776 (2017)

    Article  Google Scholar 

  33. Gurler, D., Doyle, N., Walker, E.: A link between individual differences in multisensory speech perception and eye movements. Atten. Percept. Psychophys. 77(4), 1333–1341 (2015)

    Article  Google Scholar 

  34. Seilheimer, R.L., Rosenberg, A., Angelaki, D.E.: Models and processes of multisensory cue combination. Curr. Opin. Neurobiol. 25(2), 38 (2014)

    Article  Google Scholar 

  35. Odegaard, B., Wozny, D.R., Shams, L.: A simple and efficient method to enhance audiovisual binding tendencies. Peerj 5(5), e3143 (2017)

    Article  Google Scholar 

  36. Horst, A.C.T, Koppen, M., Selen, L.P.J.: Reliability-based weighting of visual and vestibular cues in displacement estimation. Plos One 10(12), e0145015 (2015)

    Google Scholar 

  37. Beauchamp, M.S., Pasalar, S.: Neural substrates of reliability-weighted visual-tactile multisensory integration. Front. Syst. Neurosci. 4, 25 (2010)

    Google Scholar 

  38. Helbig, H.B., Ernst, M.O., Ricciardi, E.: The neural mechanisms of reliability weighted integration of shape information from vision and touch. Neuroimage 60(2), 1063–1072 (2012)

    Article  Google Scholar 

  39. Morgan, M.L., Deangelis, G.C., Angelaki, D.E.: Multisensory integration in macaque visual cortex depends on cue reliability. Neuron 59(4), 662–673 (2008)

    Article  Google Scholar 

  40. Hillock-Dunn, A., Grantham, D.W., Wallace, M.T.: The temporal binding window for audiovisual speech: children are like little adults. Neuropsychologia 88, 74–82 (2016)

    Article  Google Scholar 

  41. Helbig, H.B., Ernst, M.O.: Knowledge about a common source can promote visual-haptic integration. Perception 36(10), 1523–1533 (2007)

    Article  Google Scholar 

  42. Beierholm, U.R., Quartz, S.R., Shams, L.: Bayesian priors are encoded independently from likelihoods in human multisensory perception. J. Vis. 9(5), 23.1 (2009)

    Article  Google Scholar 

  43. Cuppini, C., Shams, L., Magosso, E., Mauro, U.: A biologically inspired neurocomputational model for audio-visual integration and causal inference. Eur. J. Neurosci. 46(9), 2481–2498 (2017)

    Article  Google Scholar 

  44. Mcgovern, D.P., Roudaia, E., Newell, F.N.: Perceptual learning shapes multisensory causal inference via two distinct mechanisms. Sci. Rep. 6, 24673 (2016)

    Google Scholar 

  45. 曾毅, 刘成林, 谭铁牛: 类脑智能研究的回顾与展望. 计算机学报 39 (1), 212–222 (2016)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. School of Computer Science and Technology, Changchun University of Science and Technology, Changchun, 130022, People’s Republic of China

    Yang Xi, Ning Gao & Qi Li

  2. College of Information Engineering, Northeast Electric Power University, Jilin, 132012, People’s Republic of China

    Yang Xi

  3. Department of Radiology, China-Japan Union Hospital of Jilin University, Changchun, 130022, People’s Republic of China

    Mengchao Zhang & Lin Liu

Authors
  1. Yang Xi
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Ning Gao
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Mengchao Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Lin Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Qi Li
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Qi Li .

Editor information

Editors and Affiliations

  1. College of Information Science and Engineering, Fujian University of Technology, Fuzhou, Fujian, China

    Jeng-Shyang Pan

  2. Graduate School of Engineering, Tohoku University, Sendai, Miyagi, Japan

    Akinori Ito

  3. Swinburne University of Technology, Hawthorn, VIC, Australia

    Pei-Wei Tsai

  4. Centre for Artificial Intelligence, Faculty of Engineering and Information Technology, University of Technology Sydney, Canberra, ACT, Australia

    Lakhmi C. Jain

Rights and permissions

Reprints and permissions

Copyright information

© 2019 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Xi, Y., Gao, N., Zhang, M., Liu, L., Li, Q. (2019). The Bayesian Causal Inference in Multisensory Information Processing: A Narrative Review. In: Pan, JS., Ito, A., Tsai, PW., Jain, L. (eds) Recent Advances in Intelligent Information Hiding and Multimedia Signal Processing. IIH-MSP 2018. Smart Innovation, Systems and Technologies, vol 109. Springer, Cham. https://doi.org/10.1007/978-3-030-03745-1_19

Download citation

Publish with us

Policies and ethics

Search

Navigation