Elsevier

NeuroImage

Volume 59, Issue 4, 15 February 2012, Pages 3677-3689
NeuroImage

Large-scale brain networks emerge from dynamic processing of musical timbre, key and rhythm

https://doi.org/10.1016/j.neuroimage.2011.11.019Get rights and content

Abstract

We investigated the neural underpinnings of timbral, tonal, and rhythmic features of a naturalistic musical stimulus. Participants were scanned with functional Magnetic Resonance Imaging (fMRI) while listening to a stimulus with a rich musical structure, a modern tango. We correlated temporal evolutions of timbral, tonal, and rhythmic features of the stimulus, extracted using acoustic feature extraction procedures, with the fMRI time series. Results corroborate those obtained with controlled stimuli in previous studies and highlight additional areas recruited during musical feature processing. While timbral feature processing was associated with activations in cognitive areas of the cerebellum, and sensory and default mode network cerebrocortical areas, musical pulse and tonality processing recruited cortical and subcortical cognitive, motor and emotion-related circuits. In sum, by combining neuroimaging, acoustic feature extraction and behavioral methods, we revealed the large-scale cognitive, motor and limbic brain circuitry dedicated to acoustic feature processing during listening to a naturalistic stimulus. In addition to these novel findings, our study has practical relevance as it provides a powerful means to localize neural processing of individual acoustical features, be it those of music, speech, or soundscapes, in ecological settings.

Highlights

► Novel paradigm combines fMRI, acoustic feature extraction and behavioral psychology. ► Timbre recruits cerebellar cognitive areas, sensory and DMN-related cortical areas. ► Rhythm and tonality recruit limbic regions, cognitive and somatomotor areas.

Introduction

Music is fundamental to humans across all cultures and is capable of eliciting intense emotions (Salimpoor et al., 2011). Uncovering the neural underpinnings of music processing has become a central theme in cognitive neuroscience in the past decade, as evidenced by the constantly increasing corpus of studies on this topic. The intrinsically multidimensional nature of music renders this task challenging. More specifically, music comprises several perceivable features of varying levels of abstraction, such as loudness, pitch (the organization of sounds along a scale from low to high), rhythm (the perceptual organization of sound events in time) and timbre (property that allows to distinguish between different instrument sounds having the same pitch and loudness). Perceiving polyphonic music involves automatic segregation of the musical information in the brain. For instance, when listening to a piece of music played by an orchestra we are able to distinguish one instrument's timbre from that of another, perceive the leading melody by extracting pitch height, and feel the beat (Bregman, 1990, Janata et al., 2002a, Janata et al., 2002b). In this process, domain-specific neural mechanisms for acoustic feature analysis and integration as well as domain-general neural circuits of attention and memory are required. In particular, hierarchical processing within the auditory cortex going from more simple to more complex features (Chevillet et al., 2011, Patterson et al., 2002), and hemispheric specialization (Samson et al., 2011, Zatorre et al., 2002) for spectral vs. temporal acoustic variations, have been identified as putative principles of functional organization of acoustic feature-related processing. Previous neuroimaging studies of music have attempted to identify brain structures involved in the perception of music-related perceptual features, such as pitch (Patterson et al., 2002), sensory dissonance (Blood et al., 1999, Koelsch et al., 2006), rhythm (Chen et al., 2008, Grahn and Rowe, 2009), timbre (Caclin et al., 2006, Halpern et al., 2004), and key (Janata et al., 2002a, Janata et al., 2002b). However, while these studies have successfully identified brain regions participating in processing of individual musical features they have relied on controlled auditory paradigms in which these features have been presented in isolation and manipulated artificially. Although a few studies have investigated brain responses during continuous listening to relatively simple musical stimuli (Janata et al., 2002a, Janata et al., 2002b, Schaefer et al., 2009), it has not been previously studied how the human brain processes, in parallel, the multitude of musical features when participants are listening to a record of real orchestra music during neuroimaging.

In the visual modality, recent evidence suggests that the brain processes visual stimuli presented in a more ecological setting differently than when presented in conventional controlled settings (Hasson et al., 2004). Assuming that this finding is generalizable across sensory modalities, one could expect that the majority of studies in the auditory modality, in which acoustic features were artificially manipulated, may have revealed an incomplete picture of brain function related to musical feature processing. Therefore, studying music listening as a continuous process using naturalistic stimuli could provide more accurate accounts of the processing of musical features in the brain.

We employed a stimulus-wise and task-wise more ecological setting in which participants freely listened to real music without performing any other task, in order to determine the neural mechanisms and structures responsible for musical feature processing under realistic conditions. To tackle the complexity of the problem, we introduced a novel interdisciplinary approach combining neuroimaging with computational acoustic feature extraction and behavioral psychology. As music stimulus we chose the modern tango Adios Nonino by Astor Piazzolla. The participants were scanned with fMRI while listening to this piece. Temporal evolutions of acoustic components representing timbral, tonal and rhythmic features of the stimulus were computationally extracted and validated via a perceptual experiment. Following this we performed correlation analyses of the time series of the individual acoustic components and the time series of the BOLD signal.

In light of previous studies (Samson et al., 2011), we hypothesized that timbral components would activate mainly sensory areas, such as the superior temporal gyrus (STG) and the Heschl's gyrus (HG). Moreover, we expected that the spectrally varying timbral components would activate particularly the caudolateral and anterior superior temporal regions, respectively (see Samson et al., 2011 for an overview). In addition, we predicted hemispheric lateralization, in particular, that right hemispheric regions would show larger areas involved in the processing of these features (Zatorre et al., 2002). Processing of tonality-related features was expected to recruit areas in the brain formerly known to be neural substrates of tonality processing, such as the rostromedial prefrontal cortex (Janata et al., 2002a, Janata et al., 2002b). In addition, as tonality processing draws on long-term knowledge of hierarchical tonality structures (Krumhansl, 1990), we expected the brain areas related to memory processing, such as the hippocampus (see Burianova et al., 2010 for an overview) to be activated. We hypothesized rhythm-related features to recruit, in addition to areas in the auditory cortex, cortical and subcortical areas related to motor processing, such as the premotor and supplementary motor areas, and subcortical structures involved in the processing of time intervals such as the basal ganglia (Harrington et al., 1998, Janata and Grafton, 2003, Rao et al., 2001, Schwartze et al., 2011). Furthermore, as tonal and rhythmic features are known to elicit expectations in listeners (Janata, 2005, Zanto et al., 2006), we hypothesized them to shape activations in the higher-order areas in the brain, such as the supplementary motor areas, which are known to be involved in perceptual tasks having an anticipatory component (Schubotz and von Cramon, 2002).

Section snippets

Participants

Eleven healthy participants (with no neurological, hearing or psychological problems) with formal musical training participated in the study (mean age: 23.2 ± 3.7 SD; 5 females). We chose participants with formal musical training as it has been shown that musicians display stronger neural responses to various musical features in comparison to non-musicians (Pantev et al., 2001, Wong et al., 2007). Five participants were educated in and performed mainly classical music, two musicians were trained

Results

Fig. 2 displays the results of the correlation analysis performed to test the consistency between the participants' fMRI responses. As can be seen, relatively large areas of the brain were found to display significant mean inter-subject correlations with the maximum found in the auditory cortices (r = .64, p < .0001). Following this, we performed correlation analysis between the fMRI data and acoustic components. As described in Perceptual experiment, perceptual validation of the acoustic

Discussion

In the present study, we investigated the neural correlates of timbral, tonal, and rhythmic feature processing of a naturalistic music stimulus. To this end we employed a novel paradigm combining neuroimaging, computational acoustic feature extraction and behavioral psychology. Participants were scanned using fMRI while they freely listened to the musical piece Adios Nonino by Astor Piazzolla. First, inter-subject consistency on a voxel-by-voxel basis was evaluated using mean inter-subject

Conclusions

To sum up, the current study introduced a new paradigm to investigate and predict the neural mechanisms related to the processing of timbral, tonal, and rhythmic features while listening to a naturalistic stimulus. A notable result of this novel naturalistic approach employed is that, in addition to corroborating findings from previous controlled settings, it revealed additional brain areas involved in music feature processing. First, cognitive areas of the cerebellum as well as sensory and

Acknowledgments

The authors wish to thank Geoff Luck for his help. This research was supported by the Academy of Finland (project numbers 7118616, 130412 and 138145) and by the aivoAALTO project of the Aalto University.

References (66)

  • M. Schwartze et al.

    The impact of basal ganglia lesions on sensorimotor synchronization, spontaneous motor tempo, and the detection of tempo changes

    Behav. Brain Res.

    (2011)
  • A.M. Smith et al.

    Investigation of low frequency drift in fMRI signal

    Neuroimage

    (1999)
  • C.J. Stoodley et al.

    Functional topography in the human cerebellum: a meta-analysis of neuroimaging studies

    Neuroimage

    (2009)
  • R.J. Zatorre et al.

    Structure and function of auditory cortex: music and speech

    Trends Cogn. Sci.

    (2002)
  • V. Alluri et al.

    Exploring perceptual and acoustic correlates of polyphonic timbre

    Music Percept.

    (2010)
  • V. Alluri et al.

    Effect of enculturation on the semantic and acoustic correlates of polyphonic timbre

    Music Percept.

    (2012)
  • A.J. Blood et al.

    Intensely pleasurable responses to music correlates with activity in brain regions implicated in reward and emotion

    Proc. Natl. Acad. Sci.

    (2001)
  • A.J. Blood et al.

    Emotional responses to pleasant and unpleasant music correlate with activity in paralimbic brain regions

    Nat. Neurosci.

    (1999)
  • E. Brattico

    Pitch perception: a physiological and individual act

  • E. Brattico et al.

    Subjective appraisal of music

    Ann. N. Y. Acad. Sci.

    (2009)
  • A.S. Bregman

    Auditory Scene Analysis: The Perceptual Organization of Sound

    (1990)
  • S. Brown et al.

    Stuttered and fluent speech production: an ale meta-analysis of functional neuroimaging studies

    Hum. Brain Mapp.

    (2005)
  • A. Caclin et al.

    Separate neural processing of timbre dimensions in auditory sensory memory

    J. Cogn. Neurosci.

    (2006)
  • J.L. Chen et al.

    Moving on time: brain network for auditory–motor synchronization is modulated by rhythm complexity and musical training

    J. Cogn. Neurosci.

    (2008)
  • M. Chevillet et al.

    Functional correlates of the anterolateral processing hierarchy in human auditory cortex

    J. Neurosci.

    (2011)
  • A. Engel et al.

    The perception of musical spontaneity in improvised and imitated jazz performances

    Front. Psychol.

    (2011)
  • A. Etkin et al.

    Emotional processing in anterior cingulate and medial prefrontal cortex

    Trends Cogn. Sci.

    (2010)
  • R.A. Fisher

    Statistical Methods for Research Workers

    (1950)
  • M.D. Fox et al.

    The Global Signal and Observed Anticorrelated Resting State Brain Networks

    J. Neurophysiol.

    (2009)
  • P. Fraisse

    Rhythm and tempo

  • Gómez, E., 2006. Tonal description of music audio signal. PhD Thesis, Universitat Pompeu Fabra,...
  • J.A. Grahn et al.

    Feeling the beat: premotor and striatal interactions in musicians and nonmusicians during beat perception

    J. Neurosci.

    (2009)
  • D.L. Harrington et al.

    Temporal processing in the basal ganglia

    Neuropsychology

    (1998)
  • Cited by (0)

    1

    Department of Music, University of Jyväskylä, PL 35(M), 40014 Jyväskylä, Finland.

    2

    BECS, Dept. of Biomedical Engineering and Computational Science, P.O.Box 12200, FI-00076 Aalto, Finland.

    3

    Institute of Behavioral Sciences, P.O.B. 9, 00014, University of Helsinki, Finland.

    View full text