Skip to main content
SpringerLink
Log in

Human–simian correspondence in the early cortical processing of multisensory cues

  • Review
  • Published:
Cognitive Processing Aims and scope Submit manuscript

Abstract

Recent findings in both monkeys and humans indicate that multisensory convergence occurs in low-level cortical structures generally believed to be unisensory in function. There is also evidence that multisensory convergence in higher-order regions occurs at very short post-stimulus latencies. Both types of convergence are of interest as they represent substrates for multisensory interactions in early cortical processing. This paper reviews the correspondence between specific findings in humans and monkeys, focusing on two areas, posterior auditory association cortex and posterior parietal visual association cortex. In each case we examine evidence for “low-level” and/or “early” multisensory convergence in humans, and then examine the results of direct physiological investigation of homologous effects in macaque monkeys. The latter allow a precise physiological and anatomical description of the effects noted in human subjects to be found. We then consider the functional implications of multisensory integration in early and low-level sensory processing, both in relation to our basic hierarchical model of cortical processing and in relation to our understanding of multisensory processes in perception. We end by considering the importance of human–simian homology in the study of the multisensory components of primate (human and monkey) communication.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1A–C
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6A, B
Fig. 7
Fig. 8

Similar content being viewed by others

Notes

  1. The actual electrophysiological recording procedure is to record sensory-evoked field potentials and action potentials from each of the contacts of the multielectrode, and CSD is then calculated using a second-derivative approximation applied to the field potential profile (Nicholson 1973).

References

  • Belin P, Zatorre R et al (2000) Voice-selective areas of human auditory cortex. Nature 403:309–312

    Article  CAS  PubMed  Google Scholar 

  • Belin P, Zatorre R et al (2002) Human temporal lobe response to vocal sounds. Cogn Brain Res 13:17–26

    Article  Google Scholar 

  • Benevento LA, Fallon J et al (1977) Auditory–visual interaction in single cells in the cortex of the superior temporal sulcus and the orbital frontal cortex of the macaque monkey. Exp Neurol 57(3):849–872

    Article  CAS  PubMed  Google Scholar 

  • Bruce C, Desimone R et al (1981) Visual properties of neurons in a polysensory area in superior temporal sulcus of the macaque. J Neurophysiol 46(2):369–384

    CAS  PubMed  Google Scholar 

  • Bullier J (2001) Feedback connections and conscious vision. Trends Cogn Sci 5(9):369–370

    Article  PubMed  Google Scholar 

  • Burton H, Sinclair R (1996) Somatosensory cortex and tactile perceptions. In: Kruger L (ed) Pain and touch. Academic Press, San Diego, pp 105–177

    Google Scholar 

  • Calvert GA, Bullmore ET et al (1997) Activation of auditory cortex during silent lipreading. Science 276(5312):593–596

    Article  CAS  PubMed  Google Scholar 

  • Calvert G, Brammer M et al (1999) Response amplification in sensory-specific cortices during crossmodal binding. Neuroreport 10:2619–2623

    CAS  PubMed  Google Scholar 

  • Colby (1998) Action-oriented spatial frames of reference in cortex. Neuron 20:15–24

    Article  CAS  PubMed  Google Scholar 

  • Cruz A, Green B (2000) Thermal stimulation of taste. Nature 403(6722):889–892

    Article  CAS  PubMed  Google Scholar 

  • Dalton P, Doolittle NB et al (2000) The merging of senses: integration of subthreshold taste and smell. Nat Neurosci 3(5):431–432

    Article  CAS  PubMed  Google Scholar 

  • Duhamel JR, Colby CL et al (1998) Ventral intraparietal area of the macaque: convergent visual and somatic response properties. J Neurophysiol 79:126–136

    CAS  PubMed  Google Scholar 

  • Falchier A, Clavagnier S et al (2002) Anatomical evidence of multimodal integration in primate striate cortex. J Neurosci 22(13):5749–5759

    CAS  PubMed  Google Scholar 

  • Felleman DJ, Van Essen DC (1991) Distributed hierarchical processing in the primate cerebral cortex. Cereb Cortex 1:1–47

    CAS  PubMed  Google Scholar 

  • Foxe JJ, Morocz IA et al (2000) Multisensory auditory-somatosensory interactions in early cortical processing revealed by high-density electrical mapping. Brain Res Cogn Brain Res 10(1–2):77–83

    Article  CAS  PubMed  Google Scholar 

  • Foxe JJ, Wylie GR et al (2002) Auditory-somatosensory multisensory processing in auditory association cortex: an fMRI study. J Neurophysiol 88(1):540–543

    PubMed  Google Scholar 

  • Fu K, Johnston T et al (2001) Characterization of somatosensory input to auditory association cortex in macaques. Soc Neurosci (Abstr) 27:390

    Google Scholar 

  • Fu KG, Shah AS et al (2002) Integration of auditory and eye position signals in auditory complex. Soc Neurosci (Abstr) 28:220

    Google Scholar 

  • Fu K, Johnston T et al (2003) Auditory cortical neurons respond to somatosensory input. J Neuorsci 23(20):7510–7515

    CAS  PubMed  Google Scholar 

  • Fu K et al (2004) Timing and laminar profile of eye position effects on auditory responses in primate auditory cortex. J Neurophysiol (in press)

  • Ghazanfar A, Logothetis N (2003) Facial expressions linked to monkey calls. Nature 423:937–938

    Article  CAS  PubMed  Google Scholar 

  • Giard M, Peronet F (1999) Auditory-visual integration during multimodal object recognition in humans: a behavioral and electrophysiological study. J Cogn Neurosci 11:473–490

    Article  CAS  PubMed  Google Scholar 

  • Graziano MS, Yap GS et al (1994) Coding of visual space by premotor neurons. Science 266(5187):1054–1057

    CAS  PubMed  Google Scholar 

  • Graziano MS, Hu XT et al (1997) Visuospatial properties of ventral premotor cortex. J Neurophysiol 77(5):2268–2292

    CAS  PubMed  Google Scholar 

  • Grunewald A, Linden JF et al (1999) Responses to auditory stimuli in macaque lateral intraparietal area. I. Effects of training. J Neurophysiol 82(1):330–342

    CAS  PubMed  Google Scholar 

  • Hackett TA, Stepniewska I et al (1998) Subdivisions of auditory cortex and ipsilateral cortical connections of the parabelt auditory cortex in macaque monkeys. J Comp Neurol 394(4):475–495

    Article  CAS  PubMed  Google Scholar 

  • Hackett TA, Preuss TM et al (2001) Architectonic identification of the core region in auditory cortex of macaques, chimpanzees, and humans. J Comp Neurol 441(3):197–222

    Article  CAS  PubMed  Google Scholar 

  • Hikosaka K, Iwai E et al (1988) Polysensory properties of neurons in the anterior bank of the caudal superior temporal sulcus of the macaque monkey. J Neurophysiol 60(5):1615–1637

    CAS  PubMed  Google Scholar 

  • Hyvarinen J, Shelepin Y (1979) Distribution of visual and somatic functions in the parietal associative area 7 of the monkey. Brain Res 169(3):561–564

    Article  CAS  PubMed  Google Scholar 

  • Jiang W, Wallace M et al (2001) Two cortical areas mediate multisensory integration in superior colliculus neurons. J Neurophysiol 85:506–522

    CAS  PubMed  Google Scholar 

  • Jones E (1998) Viewpoint: the core and matrix of thalamic organization. Neuroscience 85(2):331–345

    Article  CAS  PubMed  Google Scholar 

  • Jones EG, Powell TP (1970) An anatomical study of converging sensory pathways within the cerebral cortex of the monkey. Brain 93(4):793–820

    CAS  PubMed  Google Scholar 

  • Kosaki H, Hashikawa T et al (1997) Tonotopic organization of auditory cortical fields delineated by parvalbumin immunoreactivity in macaque monkeys. J Comp Neurol 386:304–316

    Article  CAS  PubMed  Google Scholar 

  • Lamme V (1995) The neurophysiology of figure-ground segregation in primary visual cortex. J Neurosci 15:1605–1615

    CAS  PubMed  Google Scholar 

  • Leinonen L (1980) Functional properties of neurones in the parietal retroinsular cortex in awake monkey. Acta Physiol Scand 108(4):381–384

    CAS  PubMed  Google Scholar 

  • Leinonen L, Hyvarinen J et al (1980) Functional properties of neurons in the temporo-parietal association cortex of awake monkey. Exp Brain Res 39(2):203–215

    CAS  PubMed  Google Scholar 

  • Levanen S, Jousmaki V et al (1998) Vibration-induced auditory cortex activation in a congenitally deaf adult. Curr Biol 8:869–872

    Article  CAS  PubMed  Google Scholar 

  • Lewis JW, Van Essen DC (2000) Corticocortical connections of visual, sensorimotor, and multimodal processing areas in the parietal lobe of the macaque monkey. J Comp Neurol 428(1):112–137

    Article  CAS  PubMed  Google Scholar 

  • Lewis JW, Beauchamp MS et al (2000) A comparison of visual and auditory motion processing in human cerebral cortex. Cereb Cortex 10(9):873–888

    Article  CAS  PubMed  Google Scholar 

  • Linden JF, Grunewald A et al (1999) Responses to auditory stimuli in macaque lateral intraparietal area. II. Behavioral modulation. J Neurophysiol 82(1):343–358

    CAS  PubMed  Google Scholar 

  • Lindsley RW, Jayachandra M, Javitt DC, Schroeder CE (1998) Somatosensory input to macaque auditory cortex. Neurosci Abstr 24(1):434

    Google Scholar 

  • Lindsley RW, Feldman R, Marcovici A, Specht C, Dias E, Javitt DC, Schroeder CE (1999) Somatosensory-auditory convergence in lateral sulcal regions in macaques. Neurosci Abstr 24(2):1417

    Google Scholar 

  • Mazzoni P, Bracewell RM et al (1996) Spatially tuned auditory responses in area LIP of macaques performing delayed memory saccades to acoustic targets. J Neurophysiol 75(3):1233–1241

    CAS  PubMed  Google Scholar 

  • Mehrabian A, Ferris S (1967) Inference of attitudes from nonverbal communication in two channels. J Consult Psychol 31:248–252

    CAS  PubMed  Google Scholar 

  • Mehta AD, Schroeder CE (2000) Intermodal selective attention in monkeys. II. Physiologic mechanisms of modulation. Cereb Cortex 10:359–370

    Article  CAS  PubMed  Google Scholar 

  • Mehta AD, Ulbert I et al (2000) Intermodal selective attention in monkeys. I. Distribution and timing of effects across visual areas. Cereb Cortex 10:343–358

    Article  CAS  PubMed  Google Scholar 

  • Molhom S, Ritter W et al (2002) Multisensory auditory-visual interactions during early sensory processing in humans: a high-density electrical mapping study. Brain Res Cogn Brain Res 14(1):115–128

    Article  PubMed  Google Scholar 

  • Nicholson C (1973) Theoretical analysis of field potentials in anisotropic ensembles of neuronal elements. IEEE Trans Biomed Eng 20:278–288

    Google Scholar 

  • Pallas SL, Roe AW et al (1990) Visual projections induced into the auditory pathway of ferrets. I. Novel inputs to primary auditory cortex (AI) from the LP/pulvinar complex and the topography of the MGN-AI projection. J Comp Neurol 298(1):50–68

    CAS  PubMed  Google Scholar 

  • Pandya DN, Hallett M et al (1969) Intra- and interhemispheric connections of the neocortical auditory system in the rhesus monkey. Brain Res 14(1):49–65

    Article  CAS  PubMed  Google Scholar 

  • Rauschecker JP, Tian B et al (1997) Serial and parallel processing in rhesus monkey auditory cortex. J Comp Neurol 382:89–103

    Article  CAS  PubMed  Google Scholar 

  • Rizzolatti G, Scandolara C et al (1981a) Response properties and behavioral modulation of “mouth” neurons of the postarcuate cortex (area 6) in macaque monkeys. Brain Res 225(2):421–424

    Article  CAS  PubMed  Google Scholar 

  • Rizzolatti G, Scandolara C et al (1981b) Afferent properties of periarcuate neurons in macaque monkeys. I. Somatosensory responses. Behav Brain Res 2(2):125–146

    Article  CAS  PubMed  Google Scholar 

  • Rizzolatti G, Scandolara C et al (1981c) Afferent properties of periarcuate neurons in macaque monkeys. II. Visual responses. Behav Brain Res 2(2):147–163

    Article  CAS  PubMed  Google Scholar 

  • Rockland K, Ojima H (2003) Multisensory convergence in calcarine visual areas in macaque monkey. Int J Psychophysiol Spec Issue Multisens Process 50:19–26

    Article  Google Scholar 

  • Roe AW, Pallas SL et al (1990) A map of visual space induced in primary auditory cortex. Science 250(4982):818–820

    CAS  PubMed  Google Scholar 

  • Romanski LM, Bates JF et al (1999a) Auditory belt and parabelt projections to the prefrontal cortex in the rhesus monkeys. J Comp Neurol 403:141–157

    Article  CAS  PubMed  Google Scholar 

  • Romanski LM, Tian B et al (1999b) Dual streams of auditory afferents target multiple domains in the primate prefrontal cortex. Nat Neurosci 2(12):1131–1136

    Article  CAS  PubMed  Google Scholar 

  • Sams M, Aulanko R et al (1991) Seeing speech: visual information from lip movements modifies activity in the human auditory cortex. Neurosci Lett 127:141–145

    Article  CAS  PubMed  Google Scholar 

  • Schroeder CE, Foxe JJ (2002) The timing and laminar profile of converging inputs to multisensory areas of the macaque neocortex. Brain Res Cogn Brain Res 14(1):187–198

    Article  PubMed  Google Scholar 

  • Schroeder CE, Seto S et al (1995a) Electrophysiological evidence for overlapping dominant and latent inputs to somatosensory cortex in squirrel monkeys. J Neurophysiol 74(2):722–732

    CAS  PubMed  Google Scholar 

  • Schroeder CE, Steinschneider M et al (1995b) Localization of ERP generators and identification of underlying neural processes. Electroencephalogr Clin Neurophysiol 44(Suppl):55–75

    CAS  Google Scholar 

  • Schroeder CE, Mehta AD et al (1998) A spatiotemporal profile of visual system activation revealed by current source density analysis in the awake macaque. Cereb Cortex 8:575–592

    Article  CAS  PubMed  Google Scholar 

  • Schroeder CE, Lindsley RW et al (2001a) Somatosensory input to auditory association cortex in the macaque monkey. J Neurophysiol 85(3):1322–1327

    CAS  PubMed  Google Scholar 

  • Schroeder CE, Mehta AD et al (2001b) Determinants and mechanisms of attentional modulation of neural processing. Front Biosci 6:D672–D684

    CAS  PubMed  Google Scholar 

  • Schroeder CE, Smiley JF et al (2003) Anatomical mechanisms and functional implications of multisensory convergence in early cortical processing. Int J Psychophysiol Spec Issue Multisens Integr 50:5–17

    Article  Google Scholar 

  • Seltzer B, Pandya DN (1980) Converging visual and somatic sensory cortical input to the intraparietal sulcus of the rhesus monkey. Brain Res 192(2):339–351

    Article  CAS  PubMed  Google Scholar 

  • Shams L, Kamitani Y et al (2000) What you see is what you hear. Nature 408(14):788

    Article  CAS  PubMed  Google Scholar 

  • Small D, Gregory M et al (2003) Dissociation of the neural representation of intensity and affective valuation in humans. Neuron 39(4):581–583

    Article  PubMed  Google Scholar 

  • Stein BE, Meredith MA (1993) The merging of the senses. MIT Press, Boston

    Google Scholar 

  • Sur M, Pallas SL et al (1990) Cross-modal plasticity in cortical development: differentiation and specification of sensory neocortex. Trends Neurosci 13(6):227–233

    Article  CAS  PubMed  Google Scholar 

  • Werner-Reiss U, Kelly KA, Trause AS, Underhill AM, Groh JM (2003) Eye position affects activity in primary auditory cortex of primates. Curr Biol 13:554-562

    Article  CAS  PubMed  Google Scholar 

  • Wright T, Pelphrey K et al (2003) Polysensory interactions along lateral temporal regions evoked by audiovisual speech. Cereb Cortex 13:1034–1043

    Article  PubMed  Google Scholar 

  • Zipzer K, Lamme P et al (1996) Contextual modulation in primary visual cortex. J Neurosci 16:7376–7389

    PubMed  Google Scholar 

Download references

Acknowledgements

Thanks to Micah Murray and Ankoor Shah for helpful discussions, and to T. McGinnis and Noelle O’Connell for excellent technical and conceptual assistance. This work was supported in part by grants from the National Institute of Mental Health MH61989 and TW05674.

Author information

Authors and Affiliations

  1. Cognitive Neuroscience and Schizophrenia Program, Nathan Kline Institute for Psychiatric Research, 140 Old Orangeburg Rd., Bldg. 37, Orangeburg, NY, 10962, USA

    Charles E. Schroeder, Sophie Molhom, Peter Lakatos, Walter Ritter & John J. Foxe

  2. Department of Neuroscience, Albert Einstein College of Medicine, Bronx, NY, USA

    Charles E. Schroeder & John J. Foxe

  3. Department of Neurology, Albert Einstein College of Medicine, Bronx, NY, USA

    Charles E. Schroeder

  4. Department of Psychiatry, Albert Einstein College of Medicine, Bronx, NY, USA

    John J. Foxe

  5. Institute for Psychology, Hungarian Academy of Sciences, Budapest, Hungary

    Peter Lakatos

Authors
  1. Charles E. Schroeder
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Sophie Molhom
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Peter Lakatos
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Walter Ritter
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. John J. Foxe
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Charles E. Schroeder.

Additional information

Edited by: Marie-Hélène Giard and Mark Wallace

Rights and permissions

Reprints and permissions

About this article

Cite this article

Schroeder, C.E., Molhom, S., Lakatos, P. et al. Human–simian correspondence in the early cortical processing of multisensory cues. Cogn Process 5, 140–151 (2004). https://doi.org/10.1007/s10339-004-0020-4

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10339-004-0020-4

Keywords

Search

Navigation