Skip to main content
SpringerLink
Log in

Multimodal action representation in human left ventral premotor cortex

  • Research Report
  • Published:
Cognitive Processing Aims and scope Submit manuscript

Abstract

We used functional magnetic resonance imaging (fMRI) to investigate the neural systems responding to the sight and to the sound of an action. Subjects saw a video of paper tearing in silence (V), heard the sound of paper tearing (A), and saw and heard the action simultaneously (A + V). Compared to a non-action control stimulus, we found that hearing action sounds (A) activated the anterior inferior frontal gyrus and middle frontal gyrus in addition to primary auditory cortex. The anterior inferior frontal gyrus, which is known to be activated by environmental sounds, also seems to be involved in recognizing actions by sound. Consistent with previous research, seeing an action video (V) compared with seeing a non-action video activated the premotor cortex, intraparietal cortex, and the pars opercularis of the inferior frontal gyrus. An A + V facilitation effect was found in the ventral premotor cortex on the border of areas 44, 6, 3a, and 3b for the action stimuli but not for the control stimuli. This region may be involved in integrating multimodal information about actions. These data provide evidence that the ventral premotor cortex may provide an action representation that abstracts across both agency (self and other) and sensory modality (hearing and seeing). This function may be an important precursor of language functions.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5

References

  • Adams RB, Janata P (2002) A comparison of neural circuits underlying auditory and visual object categorization. Neuroimage 16(2):361–377

    Article  PubMed  Google Scholar 

  • Amunts K, Schleicher A, Burgel U, Mohlberg H, Uylings HB, Zilles K (1999) Broca’s region revisited: Cytoarchitecture and intersubject variability. J Comp Neurol 412(2):319–341

    Article  PubMed  CAS  Google Scholar 

  • Aziz-Zadeh L, Iacoboni M, Zaidel E, Wilson S, Mazziotta J (2004) Left hemisphere motor facilitation in response to manual action sounds. Eur J Neurosci 19(9):2609–2612

    Article  PubMed  Google Scholar 

  • Aziz-Zadeh L, Wilson SM, Rizzolatti G, Iacoboni M (2006) Congruent embodied representations for visually presented actions and linguistic phrases describing actions. Curr Biol 16(18):1818–1823

    Article  PubMed  CAS  Google Scholar 

  • Barraclough NE, Xiao D, Baker CI, Oram MW, Perrett DI (2005) Integration of visual and auditory information by superior temporal sulcus neurons responsive to the sight of actions. J Cogn Neurosci 17(3):377–391

    Article  PubMed  Google Scholar 

  • Beauchamp MS, Argall BD, Bodurka J, Duyn JH, Martin A (2004) Unraveling multisensory integration: Patchy organization within human sts multisensory cortex. Nat Neurosci 7(11):1190–1192

    Article  PubMed  CAS  Google Scholar 

  • Behrens T, Woolrich MW, Smith S (2003) Multi-subject null hypothesis testing using a fully bayesian framework: theory. In: Hum brain mapping meeting, New York

  • Bestmann S, Baudewig J, Siebner HR, Rothwell JC, Frahm J (2004) Functional mri of the immediate impact of transcranial magnetic stimulation on cortical and subcortical motor circuits. Eur J Neurosci 19(7):1950–1962

    Article  PubMed  Google Scholar 

  • Blakemore SJ, Frith C (2005) The role of motor contagion in the prediction of action. Neuropsychologia 43(2):260–267

    Article  PubMed  Google Scholar 

  • Blakemore SJ, Boyer P, Pachot-Clouard M, Meltzoff A, Segebarth C, Decety J (2003) The detection of contingency and animacy from simple animations in the human brain. Cereb Cortex 13(8):837–844

    Article  PubMed  Google Scholar 

  • Bookheimer S (2002) Functional mri of language: new approaches to understanding the cortical organization of semantic processing. Annu Rev Neurosci 25:151–188

    Article  PubMed  Google Scholar 

  • Buccino G, Binkofski F, Fink GR, Fadiga L, Fogassi L, Gallese V et al (2001) Action observation activates premotor and parietal areas in a somatotopic manner: an fmri study. Eur J Neurosci 13(2):400–404

    Article  PubMed  CAS  Google Scholar 

  • Calvert GA, Campbell R, Brammer MJ (2000) Evidence from functional magnetic resonance imaging of crossmodal binding in the human heteromodal cortex. Curr Biol 10(11):649–657

    Article  PubMed  CAS  Google Scholar 

  • Calvert GA, Hansen PC, Iversen SD, Brammer MJ (2001) Detection of audio-visual integration sites in humans by application of electrophysiological criteria to the bold effect. Neuroimage 14(2):427–438

    Article  PubMed  CAS  Google Scholar 

  • Castelli F, Happe F, Frith U, Frith C (2000) Movement and mind: a functional imaging study of perception and interpretation of complex intentional movement patterns. Neuroimage 12(3):314–325

    Article  PubMed  CAS  Google Scholar 

  • Engelien A, Silbersweig D, Stern E, Huber W, Doring W, Frith C et al (1995) The functional anatomy of recovery from auditory agnosia. A pet study of sound categorization in a neurological patient and normal controls. Brain 118(Pt 6):1395–1409

    Article  PubMed  Google Scholar 

  • Forman SD, Cohen JD, Fitzgerald M, Eddy WF, Mintun MA, Noll DC (1995) Improved assessment of significant activation in functional magnetic resonance imaging (fmri): use of cluster-size threshold. Magn Reson Med 33:636–647

    Article  PubMed  CAS  Google Scholar 

  • Friston KJ, Worsley KJ, Frakowiak RSJ, Mazziotta JC, Evans AC (1994) Assessing the significance of focal activations using their spatial extent. Hum Brain Mapp 1:214–220

    Article  Google Scholar 

  • Gallese V, Lakoff G (2005) Brain's concepts: the role of the sensory-motor system in conceptual knowledge. Cogn Neuropsychol 22:455–479

    Google Scholar 

  • Gallese V, Fadiga L, Fogassi L, Rizzolatti G (1996) Action recognition in the premotor cortex. Brain 119(Pt 2):593–609

    Article  PubMed  Google Scholar 

  • Gazzola V, Aziz-Zadeh L, Keysers C (2006) Empathy and the somatotopic auditory mirror system in humans. Curr Biol 16(18):1824–1829

    Article  PubMed  CAS  Google Scholar 

  • Geyer S (2004) The microstructural border between the motor and cognitive domain in the human cerebral cortex. Adv Anat Embryol Cell Biol 174:1–89

    Google Scholar 

  • Geyer S, Schleicher A, Zilles K (1999) Areas 3a, 3b, and 1 of human primary somatosensory cortex. Neuroimage 10(1):63–83

    Article  PubMed  CAS  Google Scholar 

  • Geyer S, Matelli M, Luppino G, Zilles K (2000a) Functional neuroanatomy of the primate isocortical motor system. Anat Embryol (Berl) 202(6):443–474

    Article  CAS  Google Scholar 

  • Geyer S, Schormann T, Mohlberg H, Zilles K (2000b) Areas 3a, 3b, and 1 of human primary somatosensory cortex. Part 2. Spatial normalization to standard anatomical space. Neuroimage 11(6 Pt 1):684–696

    Article  PubMed  CAS  Google Scholar 

  • Grezes J, Decety J (2001) Functional anatomy of execution, mental simulation, observation, and verb generation of actions: a meta-analysis. Hum Brain Mapp 12(1):1–19

    Article  PubMed  CAS  Google Scholar 

  • Grezes J, Armony JL, Rowe J, Passingham RE (2003) Activations related to “mirror” and “canonical” neurones in the human brain: an fmri study. Neuroimage 18(4):928–937

    Article  PubMed  CAS  Google Scholar 

  • Grossman E, Donnelly M, Price R, Pickens D, Morgan V, Neighbor G et al (2000) Brain areas involved in perception of biological motion. J Cogn Neurosci 12(5):711–720

    Article  PubMed  CAS  Google Scholar 

  • Hackett TA, Stepniewska I, Kaas JH (1999) Prefrontal connections of the parabelt auditory cortex in macaque monkeys. Brain Res 817(1–2):45–58

    Article  PubMed  CAS  Google Scholar 

  • Hikosaka K, Iwai E, Saito H, Tanaka K (1988) Polysensory properties of neurons in the anterior bank of the caudal superior temporal sulcus of the macaque monkey. J Neurophysiol 60(5):1615–1637

    PubMed  CAS  Google Scholar 

  • Iacoboni M, Woods RP, Brass M, Bekkering H, Mazziotta JC, Rizzolatti G (1999) Cortical mechanisms of human imitation. Science 286(5449):2526–2528

    Article  PubMed  CAS  Google Scholar 

  • Iacoboni M, Koski LM, Brass M, Bekkering H, Woods RP, Dubeau MC et al (2001) Reafferent copies of imitated actions in the right superior temporal cortex. Proc Natl Acad Sci USA 98(24):13995–13999

    Article  PubMed  CAS  Google Scholar 

  • Iacoboni M, Molnar-Szakacs I, Gallese V, Buccino G, Mazziotta JC, Rizzolatti G (2005) Grasping the intentions of others with one’s own mirror neuron system. PLoS Biol 3(3):e79

    Article  PubMed  Google Scholar 

  • Iacoboni M, Kaplan JT, Wilson S (2007) A neural architecture for imitation. In: Nehaniv CL, Dautenhahn K (eds) Models and mechanisms of imitation and social learning in robots, humans and animals: behavioural, social, and communicative dimension. Cambridge University Press, London, pp 71–87

  • Jellema T, Perrett DI (2003) Cells in monkey sts responsive to articulated body motions and consequent static posture: a case of implied motion? Neuropsychologia 41(13):1728–1737

    Article  PubMed  Google Scholar 

  • Kaplan JT, Iacoboni M (2005) Listen to my actions!Behav Brain Sci 28(2):135

    Article  Google Scholar 

  • Kaplan JT, Iacoboni M (2006) Getting a grip on other minds: mirror neurons, intention understanding, and cognitive empathy. Soc Neurosci 1(3–4):175–183

    Google Scholar 

  • Keysers C, Kohler E, Umilta MA, Nanetti L, Fogassi L, Gallese V (2003) Audiovisual mirror neurons and action recognition. Exp Brain Res 153(4):628–636

    Article  PubMed  CAS  Google Scholar 

  • Kohler E, Keysers C, Umilta MA, Fogassi L, Gallese V, Rizzolatti G (2002) Hearing sounds, understanding actions: action representation in mirror neurons. Science 297(5582):846–848

    Article  PubMed  CAS  Google Scholar 

  • Koski L, Wohlschlager A, Bekkering H, Woods RP, Dubeau MC, Mazziotta JC et al (2002) Modulation of motor and premotor activity during imitation of target-directed actions. Cereb Cortex 12(8):847–855

    Article  PubMed  Google Scholar 

  • Lahav A, Saltzman E, Schlaug G (2007) Action representation of sound: audiomotor recognition network while listening to newly acquired actions. J Neurosci 27(2):308–314

    Article  PubMed  CAS  Google Scholar 

  • Macaluso E, George N, Dolan R, Spence C, Driver J (2004) Spatial and temporal factors during processing of audiovisual speech: a pet study. Neuroimage 21(2):725–732

    Article  PubMed  CAS  Google Scholar 

  • Maeder PP, Meuli RA, Adriani M, Bellmann A, Fornari E, Thiran JP et al (2001) Distinct pathways involved in sound recognition and localization: a human fmri study. Neuroimage 14(4):802–816

    Article  PubMed  CAS  Google Scholar 

  • Meredith MA, Stein BE (1986) Visual, auditory, and somatosensory convergence on cells in superior colliculus results in multisensory integration. J Neurophysiol 56(3):640–662

    PubMed  CAS  Google Scholar 

  • Molnar-Szakacs I, Iacoboni M, Koski L, Mazziotta JC (2005) Functional segregation within pars opercularis of the inferior frontal gyrus: evidence from fmri studies of imitation and action observation. Cereb Cortex 15(7):986–994

    Article  PubMed  Google Scholar 

  • Pizzamiglio L, Aprile T, Spitoni G, Pitzalis S, Bates E, D’Amico S et al (2005) Separate neural systems for processing action- or non-action-related sounds. Neuroimage 24(3):852–861

    Article  PubMed  CAS  Google Scholar 

  • Puce A, Perrett D (2003) Electrophysiology and brain imaging of biological motion. Philos Trans R Soc Lond B Biol Sci 358(1431):435–445

    Article  PubMed  Google Scholar 

  • Rademacher J, Burgel U, Geyer S, Schormann T, Schleicher A, Freund HJ et al (2001) Variability and asymmetry in the human precentral motor system. A cytoarchitectonic and myeloarchitectonic brain mapping study. Brain 124(Pt 11):2232–2258

    Article  PubMed  CAS  Google Scholar 

  • Rademacher J, Burgel U, Zilles K (2002) Stereotaxic localization, intersubject variability, and interhemispheric differences of the human auditory thalamocortical system. Neuroimage 17(1):142–160

    Article  PubMed  CAS  Google Scholar 

  • Rizzolatti G, Arbib MA (1998) Language within our grasp. Trends Neurosci 21(5):188–194

    Article  PubMed  CAS  Google Scholar 

  • Rizzolatti G, Craighero L (2004) The mirror-neuron system. Annu Rev Neurosci 27:169–192

    Article  PubMed  CAS  Google Scholar 

  • Rizzolatti G, Fadiga L, Gallese V, Fogassi L (1996) Premotor cortex and the recognition of motor actions. Brain Res Cogn Brain Res 3(2):131–141

    Article  PubMed  CAS  Google Scholar 

  • Romanski LM, Goldman-Rakic PS (2002) An auditory domain in primate prefrontal cortex. Nat Neurosci 5(1):15–16

    Article  PubMed  CAS  Google Scholar 

  • Romanski LM, Bates JF, Goldman-Rakic PS (1999a) Auditory belt and parabelt projections to the prefrontal cortex in the rhesus monkey. J Comp Neurol 403(2):141–157

    Article  PubMed  CAS  Google Scholar 

  • Romanski LM, Tian B, Fritz J, Mishkin M, Goldman-Rakic PS, Rauschecker JP (1999b) Dual streams of auditory afferents target multiple domains in the primate prefrontal cortex. Nat Neurosci 2(12):1131–1136

    Article  PubMed  CAS  Google Scholar 

  • Saygin AP, Wilson SM, Hagler DJ Jr, Bates E, Sereno MI (2004) Point-light biological motion perception activates human premotor cortex. J Neurosci 24(27):6181–6188

    Article  PubMed  CAS  Google Scholar 

  • Schubotz RI, von Cramon DY (2003) Functional-anatomical concepts of human premotor cortex: evidence from fmri and pet studies. Neuroimage 20(Suppl 1):S120–S131

    Article  PubMed  Google Scholar 

  • Schubotz RI, von Cramon DY, Lohmann G (2003) Auditory what, where, and when: a sensory somatotopy in lateral premotor cortex. Neuroimage 20(1):173–185

    Article  PubMed  Google Scholar 

  • Schultz RT, Grelotti DJ, Klin A, Kleinman J, Van der Gaag C, Marois R et al (2003) The role of the fusiform face area in social cognition: implications for the pathobiology of autism. Philos Trans R Soc Lond B Biol Sci 358(1430):415–427

    Article  PubMed  Google Scholar 

  • Seltzer B, Pandya DN (1989) Frontal lobe connections of the superior temporal sulcus in the rhesus monkey. J Comp Neurol 281(1):97–113

    Article  PubMed  CAS  Google Scholar 

  • Seltzer B, Pandya DN (1994) Parietal, temporal, and occipital projections to cortex of the superior temporal sulcus in the rhesus monkey: a retrograde tracer study. J Comp Neurol 343(3):445–463

    Article  PubMed  CAS  Google Scholar 

  • Vaina LM, Solomon J, Chowdhury S, Sinha P, Belliveau JW (2001) Functional neuroanatomy of biological motion perception in humans. Proc Natl Acad Sci USA 98(20):11656–11661

    Article  PubMed  CAS  Google Scholar 

  • Wilson SM, Saygin AP, Sereno MI, Iacoboni M (2004) Listening to speech activates motor areas involved in speech production. Nat Neurosci 7(7):701–702

    Article  PubMed  CAS  Google Scholar 

  • Wolpert DM, Doya K, Kawato M (2003) A unifying computational framework for motor control and social interaction. Philos Trans R Soc Lond B Biol Sci 358(1431):593–602

    Article  PubMed  Google Scholar 

  • Worsley KJ, Evans AC, Marrett S, Neelin P (1992) A three-dimensional statistical analysis for cbf activation studies in human brain. J Cereb Blood Flow Metab 12(6):900–918

    PubMed  CAS  Google Scholar 

  • Wright TM, Pelphrey KA, Allison T, McKeown MJ, McCarthy G (2003) Polysensory interactions along lateral temporal regions evoked by audiovisual speech. Cereb Cortex 13(10):1034–1043

    Article  PubMed  Google Scholar 

  • Zatorre RJ, Bouffard M, Belin P (2004) Sensitivity to auditory object features in human temporal neocortex. J Neurosci 24(14):3637–3642

    Article  PubMed  CAS  Google Scholar 

Download references

Acknowledgments

For generous support the authors also wish to thank the Brain Mapping Medical Research Organization, Brain Mapping Support Foundation, Pierson-Lovelace Foundation, The Ahmanson Foundation, Tamkin Foundation, Jennifer Jones-Simon Foundation, Capital Group Companies Charitable Foundation, Robson Family, William M. and Linda R. Dietel Philanthropic Fund at the Northern Piedmont Community Foundation, Northstar Fund, the National Center for Research Resources grants RR12169, RR13642 and RR08655, and NIH grant MH63680. The authors would also like to thank Lisa Aziz-Zadeh for her contributions to this work.

Author information

Authors and Affiliations

  1. Ahmanson-Lovelace Brain Mapping Center, Department of Psychiatry and Biobehavioral Sciences, Neuropsychiatric Institute, David Geffen School of Medicine, University of California, Los Angeles, CA, USA

    Jonas T. Kaplan & Marco Iacoboni

  2. UCLA Ahmanson-Lovelace Brain Mapping Center, 660 Charles Young Dr. South, Los Angeles, CA, 90095, USA

    Jonas T. Kaplan

Authors
  1. Jonas T. Kaplan
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Marco Iacoboni
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Jonas T. Kaplan.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Kaplan, J.T., Iacoboni, M. Multimodal action representation in human left ventral premotor cortex. Cogn Process 8, 103–113 (2007). https://doi.org/10.1007/s10339-007-0165-z

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10339-007-0165-z

Keywords

Search

Navigation