Hemispheric lateralization of linguistic prosody recognition in comparison to speech and speaker recognition

Highlights

• Linguistic prosody recognition involves temporo-frontal areas in both hemispheres.

• Lateralization of linguistic prosody recognition depends on control task.

• Right-lateralization in the contrast of prosody vs. speech recognition

• Left-lateralization in the contrast of prosody vs. speaker recognition

• Conjoined activity in left parietal and bilateral IFG for both contrasts

Abstract

Hemispheric specialization for linguistic prosody is a controversial issue. While it is commonly assumed that linguistic prosody and emotional prosody are preferentially processed in the right hemisphere, neuropsychological work directly comparing processes of linguistic prosody and emotional prosody suggests a predominant role of the left hemisphere for linguistic prosody processing. Here, we used two functional magnetic resonance imaging (fMRI) experiments to clarify the role of left and right hemispheres in the neural processing of linguistic prosody. In the first experiment, we sought to confirm previous findings showing that linguistic prosody processing compared to other speech-related processes predominantly involves the right hemisphere. Unlike previous studies, we controlled for stimulus influences by employing a prosody and speech task using the same speech material. The second experiment was designed to investigate whether a left-hemispheric involvement in linguistic prosody processing is specific to contrasts between linguistic prosody and emotional prosody or whether it also occurs when linguistic prosody is contrasted against other non-linguistic processes (i.e., speaker recognition). Prosody and speaker tasks were performed on the same stimulus material. In both experiments, linguistic prosody processing was associated with activity in temporal, frontal, parietal and cerebellar regions. Activation in temporo-frontal regions showed differential lateralization depending on whether the control task required recognition of speech or speaker: recognition of linguistic prosody predominantly involved right temporo-frontal areas when it was contrasted against speech recognition; when contrasted against speaker recognition, recognition of linguistic prosody predominantly involved left temporo-frontal areas. The results show that linguistic prosody processing involves functions of both hemispheres and suggest that recognition of linguistic prosody is based on an inter-hemispheric mechanism which exploits both a right-hemispheric sensitivity to pitch information and a left-hemispheric dominance in speech processing.

Introduction

When we listen to speech, we do not only infer meaning from what our communication partners say but also from how they say it. Such prosodic cues are an integral part of human communication providing rich information about the speaker's emotional state (emotional prosody) and the speech message (linguistic prosody) (e.g., Wildgruber et al., 2006). Linguistic prosody signals, for example, the boundaries between phrases, relevant words within a sentence, and whether a sentence is a question or a statement (for review see e.g., Friederici, 2011). How the brain processes linguistic prosody has been investigated by a number of neuroimaging and lesion studies (for review see e.g., Friederici and Alter, 2004, Wildgruber et al., 2006, Witteman et al., 2011, Wong, 2002). One central question of this research is the functional lateralization of linguistic prosody processing. Although linguistic prosody serves various functions in speech comprehension, its neural representation does not consistently conform to the left-hemispheric dominance found for other speech processes including lexical and syntactic processes (e.g., Binder et al., 2009, Friederici, 2011, Leff et al., 2008, McGettigan and Scott, 2012, Scott, 2005, Vigneau et al., 2006). It is still a matter of debate whether there is a hemispheric specialization for linguistic prosody processing or whether it involves both hemispheres to the same extent (e.g., Witteman et al., 2011).

Most of what we currently know about the functional lateralization of linguistic prosody comes from neuropsychological and neuroimaging work investigating linguistic prosody as compared to two other processes: first, speech-related processes, such as sentence processing or word recognition in sentences (reviewed in Friederici, 2011), and second, extra-linguistic prosody functions; that is, emotional prosody in particular when segmental (lexical) information is degraded by filtering of the stimulus (reviewed in Wildgruber et al., 2006). While studies investigating linguistic prosody in comparison to other speech processes suggest a predominant role of right-hemispheric areas for linguistic prosody (Meyer et al., 2002, Plante et al., 2002, Strelnikov et al., 2006), neuroimaging studies comparing linguistic prosody and emotional prosody have failed to find a clear right-lateralization in linguistic prosody processing (Wildgruber et al., 2004; see also Gandour et al., 2003, for the influence of listeners' language background). Instead, it has been shown that left-hemispheric areas are more strongly activated in linguistic as compared to emotional prosody processing. A recent meta-analysis of lesion studies found that damage to both right and left hemispheres can result in impaired recognition of linguistic prosody and emotional prosody (Witteman et al., 2011). For emotional prosody, however, patients with lesions to the right hemisphere were more severely affected than patients with lesions to the left hemisphere. For linguistic prosody, no such differential impairment from lateralized brain lesions was found. In contrast, electroencephalography (EEG) studies showed that event-related potentials (ERPs) specific to linguistic prosody processing are lateralized to the right hemisphere (for review see, Friederici and Alter, 2004). A recent EEG study on lexical tone perception showed that bottom-up processes related to auditory processing of pitch contours are right-lateralized, whereas top-down processing of lexical tone showed left-lateralization (Shuai and Gong, 2014). The behavioral tasks used in lesion studies put a particular emphasis on top-down processes, such as attention, memory and decision making, which might have occluded hemispheric lateralization specific to the processing of linguistic prosody.

Here, we used two fMRI experiments (Fig. 1A) to further clarify the functional role of the right and left hemispheres for linguistic prosody processing. In both experiments, we took a novel approach to the topic. The first experiment was designed to test the right-hemispheric dominance for linguistic prosody as compared to other speech-related processes using a task rather than a stimulus contrast. Previous studies have either used contrasts across different stimulus manipulations (e.g., Humphries et al., 2005, Ischebeck et al., 2008, Meyer et al., 2002, Meyer et al., 2004, Plante et al., 2002) or contrasts across tasks which involved different stimuli (e.g., Strelnikov et al., 2006). Here, we controlled for stimulus influences by employing two tasks (i.e., prosody and speech tasks) that were performed on the same stimulus material (Fig. 1B). This means that the two conditions (prosody task and speech task) only differed in terms of task instructions. In the prosody task, participants were asked to recognize whether the present stimulus is the same or different to the previously presented stimulus in terms of intonation (i.e., question vs. statement intonation; one-back prosody task). Since intonation (i.e., falling and rising pitch contours) occurred over multiple speech segments, participants had to focus on suprasegmental linguistic information in the prosody task. In the speech task, participants were asked to recognize whether the present stimulus is the same or different to the previously presented stimulus in terms of lexical content (one-back speech task). Differences in lexical content between adjacent stimuli occurred on the phonemic level (see Methods section for details). Participants, therefore, had to focus on segmental linguistic information in the speech task. Neural substrates of linguistic prosody processing were assessed by contrasting blood-oxygen-level-dependent (BOLD) responses to the prosody vs. speech task. The second experiment was designed to test whether a left-hemispheric involvement in linguistic prosody processing is specific to contrasts between linguistic prosody and emotional prosody or whether it also occurs when recognition of linguistic prosody is contrasted against other non-linguistic processes, such as speaker recognition. Processing of speaker information has been associated with predominantly right-hemispheric temporal regions (e.g., Belin and Zatorre, 2003, Belin et al., 2000, von Kriegstein et al., 2003, von Kriegstein et al., 2010). To date, the neural substrates of linguistic prosody in contrast to speaker recognition are unknown. One possibility is that prosody and speaker recognition are supported by different areas in the right hemisphere. The other possibility is that the brain areas involved in prosody and speaker recognition overlap to a great extent due to similar acoustic properties that need to be analyzed (e.g., Abberton and Fourcin, 1978, Magnuson and Nusbaum, 2007). To test this, we contrasted BOLD responses to a one-back prosody task with a one-back speaker task, in which participants reported whether the present stimulus is the same or different to the previously presented stimulus in terms of speaker identity (Fig. 1C). As in the first experiment, both tasks were performed on the same stimulus material; prosody and speaker tasks only differed in terms of task instructions.

Based on previous literature, we expected that activation in language-relevant areas in temporal and frontal lobes, including superior and middle temporal gyri as well as inferior frontal gyrus, is associated with linguistic prosody processing in both experiments (for review see, Friederici, 2011, Friederici and Alter, 2004, Wildgruber et al., 2006, Witteman et al., 2011). We expected that this network of regions is lateralized predominantly to the right hemisphere for the contrast of prosody vs. speech task in Experiment 1. For the contrast of prosody vs. speaker task in Experiment 2, we had two alternative hypotheses: either prosody and speaker recognition involve different areas in the right hemisphere in which case prosody processing would be right-lateralized or the neural substrates of prosody and speaker recognition overlap to a great extent in the right hemisphere which could result in a predominant left-hemispheric processing of linguistic prosody. We also present the results of the reverse contrasts (i.e., speech task > prosody task in Experiment 1 and speaker task > prosody task in Experiment 2).