Learning with the wave of the hand: Kinematic and TMS evidence of primary motor cortex role in category-specific encoding of word meaning

Abstract

Language processing recruits a core fronto-temporal cortical network, which is complemented by a distributed network of modality-specific areas (such as the motor cortex) to encode referential aspects of meaning. Since most studies typically focus on already fully-formed adult vocabulary, it remains unclear how and when exactly modality-specific areas become involved in language processing. Here, we addressed this question using a 3D virtual environment game to teach adult participants new action verbs and object nouns. To test the role of primary motor cortex (M1) in selectively encoding aspects of action verb meaning early on in the process of word learning, we delivered theta-burst stimulation to three groups of participants prior to learning: M1 TMS, active control TMS, and sham TMS. Our results show that TMS of M1 (but not active or sham controls) interferes with the learning process, as indexed by measures of movement kinematics and a higher number of errors during training. Thus, TMS disruption of M1 degrades learning outcomes when motor information is an integral part of lexico-semantic encoding. This effect was corroborated in a subsequent lexical decision task, which showed significant group- and word-category RT differences, suggesting category-specific effects of TMS on word learning. Overall, our study demonstrates the M1’s causal involvement in the earliest phases of word learning and rapid encoding of semantic motor information.

Introduction

Human society and daily life are predicated on our unique ability to use language for communication. As a representational symbolic system, language is capable of encoding and transmitting countless aspects of our internal and external world. Key units in this information flow are words, which we use as shared mental pointers to meaning. To enable this unique and fundamental human ability, we learn words effortlessly and rapidly, building up lexicons of tens of thousands of entries throughout our lifespans. The cognitive and neural mechanisms by which we first establish these word-meaning mappings (that is, word semantics) are nevertheless poorly understood.

One way to link words and their meanings is based on using the internal architecture of the linguistic system itself: words are very commonly defined by reference to other words we know (Larson and Segal, 1995; Levin, 1993). Moreover, even without such explicit definitions, word use is subject to statistical and structural regularities such that lexical items tend to co-occur with each other, entering into predictable and shared sentence contexts thereby forming implicit links (Ellis, 2002; Kintsch, 1998; Landauer et al., 1998; Larson and Segal, 1995; Marslen-Wilson and Tyler, 2007). In addition to this purely linguistic or symbolic route, a different sensorimotor mechanism of semantic grounding has been proposed (Anderson, 2010; Barsalou, 2008; Binder and Desai, 2011). According to this account, we establish word-meaning mappings through direct connections to real-world motor and perceptual experience. The same brain systems which support action, for example, would take part in encoding aspects of verb meaning which refer to specific body movements. Thus, word meaning would be coded not just as a symbolic linguistic representation in “core” perisylvian language areas of the brain but would be tied to sensorimotor experiences (and thus the modality-specific neural systems) associated with word acquisition itself.

If we assume that sensory and motor cortices encode aspects of word meaning during learning, then these same areas should mediate subsequent word retrieval. Indeed, many studies now show that processing native (and even second) language involves activity in motor, visual, auditory, and other sensory modalities, referentially linked to the semantics of particular words (Barrós-Loscertales et al., 2012; Gianelli et al., 2013; Hauk et al., 2008; Meteyard et al., 2010; Vukovic et al., 2016, 2018; Vukovic and Shtyrov, 2014, 2017; Zwaan, 2015). Behavioural research also shows that action verb comprehension involves motor processing. Many studies have observed slower response times when participants had to evaluate hand-action verbs and provide a hand response at the same time – suggesting that the two tasks place demands on the same motor resources (Dam et al., 2010; Gianelli et al., 2013; Glenberg and Gallese, 2011; Glenberg and Kaschak, 2002; Secora and Emmorey, 2015; Taylor and Zwaan, 2009). For example, Buccino et al. (2005) had participants judge the concreteness of hand-action-sentences and leg-action-sentences by pressing a button with either their hand or foot. They observed significantly slower reaction times when participants responded to hand-action-sentences using their hands and, conversely, slower foot responses when judging leg-related sentences. de Vega et al. (2013) similarly found that hand responses to action-related sentences involving movement to/away from body (“I threw the ball to him/He threw the ball to me”) were slower when participants were asked to quickly respond using a congruent hand movement. These findings are explained in the framework of associative learning, according to which co-active neuronal structures become connected – so, hearing a word in conjunction with experiencing the named object or action will strengthen the connections between modality-specific and language systems, creating a distributed neuronal memory trace for the word in question (see, for example, Garagnani and Pulvermüller, 2011). This implies that sensorimotor brain areas are used to learn word meanings, thus becoming parts of words’ memory circuits. When participants process language in everyday contexts, the activity of these circuits can be measured in neuroimaging experiments. However, the causal role of these areas in the acquisition of lexical semantics so far only remains an assumption, since previous studies measured language processing as retrieval of already known words. Thus, they cannot speak to the nature of word encoding, and some scholars have indeed questioned the causal significance of observed perception-action mappings in comprehension (Lotto et al., 2009; Mahon and Caramazza, 2008; Postle et al., 2008). Moreover, memory processes in the brain are usually studied under two distinct categories - those related to encoding and retrieval (Friedman and Johnson, 2000; Radvansky, 2017; Rugg et al., 2002). While there is ample evidence with respect to modality-specific brain systems’ involvement in the latter, the former remains understudied.

To fill this key gap in existing knowledge and answer the question of modality-specific cortical involvement in encoding new word meanings, we scrutinised the process of word learning using action verbs as a test case. In previous research using various neuroimaging techniques (EEG, MEG, fMRI, TMS) as well as behavioural interference experiments, processing of native verbs was shown to involve motor cortex activity during retrieval - an effect often explained by appeal to the specific way in which these are putatively encoded at the point of learning (Barsalou, 1999; Beilock and Goldin-Meadow, 2010; Cross et al., 2006; Kolb, 1984). Here, to fill the missing half of the equation, we explicitly investigate novel verb encoding and the causal role of primary motor cortex (M1) at the earliest stages of such learning. Of relevance, past TMS studies have already shown that M1 is vital in procedural motor learning (Iezzi et al., 2010; Steel et al., 2016), sequence learning (Rosenthal et al., 2009), early consolidation (Wilkinson et al., 2015), and visuo-motor learning (Nudo et al., 1996; Rioult-Pedotti et al., 1998; Muellbacher et al., 2002). To test if a similar causal relationship exists with respect to linguistic learning, we suggested to interfere with M1 function during a word learning task and investigate the effects of such interference on the acquisition of new words of different types: action verbs and object nouns. Past research suggests that TMS disruption of above mechanisms would impair learning performance by hindering the establishment of motor semantic traces for newly learned words.

To disrupt M1 function, we used transcranial magnetic stimulation (TMS) which can alter neural processing in the healthy human brain, thereby establishing the causal role of the stimulated region in a given mental task (Jahanshahi and Rothwell, 2000; Klomjai et al., 2015; Pascual-Leone et al., 2000; Song et al., 2011). Previous research has already shown that cTBS decreases cortical excitability, and results in an effect resembling long-term synaptic depression (LTD) (Chen et al., 1997; Huang et al., 2005; Vukovic et al., 2016; Wilkinson et al., 2015). We assessed word learning performance using several online and offline behavioural measures, and an interactive virtual environment (VE) task which allowed both experimental control and approximation of naturalistic verb learning scenarios. If a relationship between motor cortices and word action semantics exists, we expected that M1 TMS should specifically influence the online learning of new action verbs but not object nouns, since the former are assumed to be (at least in part) motorically coded.

To ensure that any observed effects are the result of our targeted neural manipulation instead of being placebo or a non-specific consequence of TMS, we included a sham stimulation condition. Here, another group of participants underwent the identical testing protocol and experienced similar auditory and somatosensory effects of M1 TMS, without actual stimulation of the underlying motor brain tissue. Finally, a further control group was tested as an active TMS control, using as the stimulation site the right superior-parietal lobule (rSPL, subarea 5l). The rSPL was chosen because, based on a combination of structural, resting state, and task-dependent functional connectivity patterns, is not considered to be crucially involved in linguistic processing or be a part of the canonical language learning network (Wang et al., 2015). Moreover, TMS of the right SPL was less likely to interact with linguistic processing in the left motor hemisphere – unlike that of the right M1 area which, while often used as a TMS control site, results in undesired network effects on its contralateral homologue after cTBS stimulation (Huang and Mouraux, 2015; Y. Z. Huang et al., 2005). Thus, the double-controlled TMS protocol used here allowed us to disentangle specific vs. location-independent neural effects from placebo or somatosensory effects of TMS.

In sum, previous research has demonstrated that processing the meaning of hand action words and providing a concurrent manual response in a behavioural task leads to interference and slower reaction times, since both action words and hand action execution load onto the motor cortex. Based on this, we had two specific and complementary hypotheses: First, we expected that participants in our two control groups (both SPL-TMS and M1-Sham) would encode the meaning of novel action verbs motorically, since their M1 would not be impaired with TMS. Thus, they should make fewer errors during learning, and during later recall they would show the standard pattern found in prior studies: slower responses when providing manual responses to action words. Second, we expected worse learning accuracy in the M1 target group – since they would be asked to learn action-related verbs while their M1 was disrupted. This learning deficit should be evident in the form of more errors in the M1 group relative to the two controls, and more complex hand motion trajectories (indicating participants’ uncertainty in performing the correct action sequences denoted by new verbs). Of note, previous research has already established hand kinematic parameters - such as movement complexity - as a way to investigate a broad range of psychological processes related to learning, information integration, and cognitive uncertainty (Freeman et al., 2011). In contrast to the two control groups, who we expected to show the classic effect of slower reaction times when processing action verbs, the M1-stimulated participants should not present the same RT slow-down. Since in these participants we would disrupt the initial motor aspects of encoding of novel action verbs, it would rely less on the motor system, and thus their later processing should not interfere with manual responses. In other words, preventing the integration of motor information into a novel lexico-semantic representation should minimise or abolish motor interference during offline word processing as well.