Added value of money on motor performance feedback: Increased left central beta-band power for rewards and fronto-central theta-band power for punishments
Introduction
Human motor performance and learning critically depends upon the processing of feedback. Beyond motor performance feedback, which informs of the accuracy of a movement (i.e., seeing oneself hitting or missing a target), external sources of feedback such as monetary rewards or punishments can provide additional guidance as to the behaviors to repeat or avoid. Support for this notion comes from converging lines of evidence showing that monetary feedback enhances short-term performance and retention of motor behaviors (Abe et al., 2011; Dayan et al., 2014; Gajda et al., 2016; Galea et al., 2015; Hasson et al., 2015; Manley et al., 2014; Palminteri et al., 2011; Quattrocchi et al., 2017; Song and Smiley-Oyen, 2017; Steel et al., 2016; Wächter et al., 2009; Widmer et al., 2016). For instance, Galea et al. (2015) provided monetary rewards or punishments depending on task performance while participants acquired a novel upper limb reaching movement pattern. Compared to a control group receiving no monetary feedback, participants receiving monetary rewards following accurate performance showed improved retention of the new movement pattern. Furthermore, participants receiving monetary punishments following inaccurate performance presented more rapid performance adjustments. These results suggest that monetary feedback provides added value to motor performance feedback and acts as a catalyst to promote motor learning and performance. Yet, the neural bases of monetary feedback processing in the context of motor control tasks remain unclear.
Several electroencephalography (EEG) and magnetoencephalography (MEG) studies investigating non-motor tasks such as gambling have provided evidence for frequency-specific responses to monetary rewards and punishments in the high beta-band from 20 to 30 Hz (Andreou et al., 2017; Cohen et al., 2007; HajiHosseini and Holroyd, 2015a, 2015b; HajiHosseini et al., 2012; Marco-Pallares et al., 2008, 2009; Mas-Herrero et al., 2015) and theta-band from 3 to 8 Hz (Andreou et al., 2017; Cohen et al., 2007; De Pascalis et al., 2012; Doñamayor et al., 2011, 2012; Hajihosseini and Holroyd, 2013; Marco-Pallarés et al., 2008), respectively. These power modulations have been shown to occur mainly over fronto-central regions in a time window ranging from about 200 to 600 ms post-feedback and to be enhanced when outcomes are lowly probable (Cohen et al., 2007; Doñamayor et al., 2012; HajiHosseini et al., 2012; Mas-Herrero and Marco-Pallarés, 2014). The role of fronto-central brain regions in monetary feedback processing is further supported by electrophysiological and functional magnetic resonance imaging (fMRI) studies which have reported activity in both the fronto-central cortex (Andreou et al., 2017; Balodis et al., 2012; FitzGerald et al., 2012; Hester et al., 2010; Jarbo and Verstynen, 2015; Mas-Herrero and Marco-Pallarés, 2014; Mas-Herrero et al., 2015; Noonan et al., 2012; Rogers et al., 2004; Wrase et al., 2007) and orbitofrontal cortex (Abler et al., 2009; Camara Mancha et al., 2009; Kim et al., 2015; Klein-Flügge et al., 2013; Noonan et al., 2012; O'Doherty et al., 2001; Roesch and Olson, 2004; Rogers et al., 2004; Xue et al., 2013) following monetary feedback delivery.
Although the above-cited work argues for a frequency-specific signature for the processing of monetary rewards and punishments, it is unknown whether these oscillatory modulations also take place in the context of motor control tasks. In particular, unlike gambling paradigms, the delivery of monetary feedback in motor control tasks is contingent upon the accuracy of the movement and directly influences subsequent behavioral adjustments. Furthermore, to have an impact on motor learning and performance, monetary feedback would be expected to influence activity in brain regions in which movements are planned and executed, namely in functionally lateralized motor regions such as dorsal premotor cortex (PMd) and primary motor cortex (M1) (Fu et al., 1993, 1995; Mandelblat-Cerf et al., 2009, 2011; Overduin et al., 2009; Paz et al., 2003, 2005; Pearce and Moran, 2012; Richardson et al., 2012; Sosnik et al., 2014; Stark et al., 2007; Wise et al., 1998; Xiao, 2005; Xiao et al., 2006). Interestingly, recent studies have provided support for the notion that motor cortical regions are involved in reward processing (Marsh et al., 2015; Ramakrishnan et al., 2017; Ramkumar et al., 2016; Saiki et al., 2014; Suzuki et al., 2014). Indeed, neurons in monkey PMd, M1, and primary somatosensory cortex (S1) have been shown to respond differently when an upper limb reaching movement successfully achieves a target and is rewarded with juice as compared to when a target is missed (Ramakrishnan et al., 2017; Ramkumar et al., 2016). These findings thus open up the possibility that oscillatory modulations associated with monetary feedback processing in the context of motor control tasks would be lateralized over motor cortical regions.
In light of the preceding evidence, the objective of this study was to test the hypothesis that beta- and theta-band oscillations respectively reflect monetary rewards and punishments in a motor control task. Moreover, it was hypothesized that the use of monetary feedback would result in greater oscillatory activity than motor performance feedback alone. EEG was recorded while participants performed goal-directed reaching movements toward visual targets while the delivery of monetary feedback as well as its probability were manipulated based on behavioral performance. To investigate the possibility that monetary feedback processing entails lateralized responses, oscillatory activity was specifically assessed at electrodes overlaying the motor cortical regions bilaterally as well as over the fronto-central cortical regions.
Section snippets
Participants
Twenty-three self-reported right-handed human participants (16 females; 22.3 ± 0.4 years old; all reported values are means ± SEM) took part in the experiment. Participants were neurologically healthy with normal or corrected-to-normal vision. To ensure sufficient statistical power, the choice of the number of participants was based on an a priori power calculation (Button et al., 2013), which revealed that twenty-two participants were needed for analyses to be adequately powered (see below).
Endpoint accuracy
The ANOVA conducted on the endpoint accuracy data revealed a significant Outcome X Monetary Feedback X Probability three-way interaction (F (2,44) = 6.522, p = 0.003, = 0.23), a significant Outcome X Probability two-way interaction (F (1,22) = 29.955, p < 0.001, = 0.58), a main effect of Outcome (F (1,22) = 126.707, p < 0.001, = 0.85) and a main effect of Probability (F (1,22) = 81.402, p < 0.001, = 0.79). The analysis revealed no effect of Monetary Feedback (F
Greater beta-band power in left central ROI after target hits with monetary incentives
The first EEG analysis sought to determine if beta-band power was enhanced following target hits as compared to misses when monetary feedback was present (i.e., Gain and Loss conditions). The time-courses of beta-band modulations following target hits and misses in each ROI are presented in Fig. 2a. As can be seen, beta-band power was greater following hits than misses. This was confirmed statistically by the ANOVAs, which revealed a significant main effect of Outcome in the Left Central ROI
Addressing the possibility of learning or fatigue in kinematic and EEG data
To verify the possibility that learning or fatigue took place over the course of the experiment, pairwise comparisons (Early vs Late epochs) were conducted on the endpoint accuracy, RT, and MT data. Results revealed no difference for RT (Z = 0.091, p = 0.927, r = 0.01) and MT (t (22) = 0.329, p = 0.577, r = 0.07). As for endpoint accuracy, the analysis revealed a slight but significant difference across epochs (Z = 3.133, p = 0.002, r = 0.46), with participants being 0.5 ± 0.1 mm more accurate
Discussion
The present study sought to test the hypothesis that beta- and theta-band oscillations respectively reflect monetary rewards and punishments in a goal-directed reaching task, and that monetary feedback results in greater oscillatory activity than motor performance feedback alone. EEG time-frequency analyses revealed a double dissociation between target hits and misses when monetary incentives were provided. Namely, target hits associated with contextually positive outcomes (i.e., reward or
Conclusion
Overall, the present work characterizes the EEG oscillatory signatures of positive and negative monetary feedback processing in the context of goal-directed reaching movements. The identified changes in oscillatory power constitute plausible neural substrates for the documented effects of monetary incentives on motor learning and performance.
Acknowledgments
This work was supported by the Natural Sciences and Engineering Research Council (grant number 418589). The authors declare no competing financial interests.
References (128)
- et al.
Reward improves long-term retention of a motor memory through induction of offline memory gains
Curr. Biol.
(2011) - et al.
Diminished frontostriatal activity during processing of monetary rewards and losses in pathological gambling
Biol. Psychiatr.
(2012) - et al.
Short- and long-term dopamine depletion causes enhanced beta oscillations in the cortico-basal ganglia loop of parkinsonian rats
Exp. Neurol.
(2016) - et al.
Frontal theta links prediction errors to behavioral adaptation in reinforcement learning
Neuroimage
(2010) - et al.
Frontal theta as a mechanism for cognitive control
Trends Cognit. Sci.
(2014) - et al.
Topological FDR for neuroimaging
Neuroimage
(2010) - et al.
Reward expectation modulates feedback-related negativity and EEG spectra
Neuroimage
(2007) - et al.
A universal role of the ventral striatum in reward-based learning: evidence from human studies
Neurobiol. Learn. Mem.
(2014) - et al.
Oscillatory activity in basal ganglia and motor cortex in an awake behaving rodent model of Parkinson's disease
Basal Ganglia
(2014) - et al.
EEGLAB: an open source toolbox for analysis of single-trial EEG dynamics including independent component analysis
J. Neurosci. Meth.
(2004)
EEG oscillatory activity associated to monetary gain and loss signals in a learning task: effects of attentional impulsivity and learning ability
Int. J. Psychophysiol.
Demands on response inhibition processes determine modulations of theta band activity in superior frontal areas and correlations with pupillometry–Implications for the norepinephrine system during inhibitory control
Neuroimage
Magneto- and electroencephalographic manifestations of reward anticipation and delivery
Neuroimage
Neuronal activity in primate dorsal anterior cingulate cortex signals task conflict and predicts adjustments in pupil-linked arousal
Neuron
Thresholding of statistical maps in functional neuroimaging using the false discovery rate
Neuroimage
Sensitivity of frontal beta oscillations to reward valence but not probability
Neurosci. Lett.
The role of beta-gamma oscillations in unexpected rewards processing
Neuroimage
Dopaminergic modulation of motor maps in rat motor cortex: an in vivo study
Neuroscience
10/20, 10/10, and 10/5 systems revisited: their validity as relative head-surface-based positioning systems
Neuroimage
FMRI visualization of brain activity during a monetary incentive delay task
Neuroimage
The rewarding value of good motor performance in the context of monetary incentives
Neuropsychologia
Reward pays the cost of noise reduction in motor and cognitive control
Curr. Biol.
Human oscillatory activity associated to reward processing in a gambling task
Neuropsychologia
Genetic variability in the dopamine system (dopamine receptor D4, catechol-O-methyltransferase) modulates neurophysiological responses to gains and losses
Biol. Psychiatr.
Theta oscillations integrate functionally segregated sub-regions of the medial prefrontal cortex
Neuroimage
Beta oscillations and reward processing: coupling oscillatory activity and hemodynamic responses
Neuroimage
The norepinephrine system shows information-content specific properties during cognitive control–Evidence from EEG and pupillary responses
Neuroimage
The norepinephrine system and its relevance for multi-component behavior
Neuroimage
The human globus pallidus internus is sensitive to rewards–Evidence from intracerebral recordings
Brain Stimulation
Activity in human reward-sensitive brain areas is strongly context dependent
Neuroimage
Three-dimensional probabilistic anatomical cranio-cerebral correlation via the international 10-20 system oriented for transcranial functional brain mapping
Neuroimage
Critical roles for anterior insula and dorsal striatum in punishment-based avoidance learning
Neuron
The neural representation of unexpected uncertainty during value-based decision making
Neuron
From uncertainty to reward: BOLD characteristics differentiate signaling pathways
BMC Neurosci.
Interaction of synchronized dynamics in cortex and basal ganglia in Parkinson's disease
Eur. J. Neurosci.
Theta and high-beta networks for feedback processing: a simultaneous EEG-fMRI study in healthy male subjects
Transl. Psychiatry
EEG theta dynamics within frontal and parietal cortices for error processing during reaching movements in a prism adaptation study altering visuo-motor predictive planning
PLoS One
An integrative theory of locus coeruleus-norepinephrine function: adaptive gain and optimal performance
Annu. Rev. Neurosci.
Calculating and graphing within-subject confidence intervals for ANOVA
Behav. Res. Meth.
Controlling the false discovery rate: a practical and powerful approach to multiple testing
J. Roy. Stat. Soc.
Effector selection precedes reach planning in the dorsal parietofrontal cortex
J. Neurophysiol.
Anxious individuals have difficulty learning the causal statistics of aversive environments
Nat. Neurosci.
Power failure: why small sample size undermines the reliability of neuroscience
Nat. Rev. Neurosci.
Functional connectivity of reward processing in the brain
Front. Hum. Neurosci.
Relationship between oscillations in the basal ganglia and synchronization of cortical activity
Epileptic Disord. Int. Epilepsy J.
The norepinephrine system affects specific neurophysiological subprocesses in the modulation of inhibitory control by working memory demands
Hum. Brain Mapp.
Synchronous, focally modulated beta-band oscillations characterize local field potential activity in the striatum of awake behaving monkeys
J. Neurosci.
Stochastic reinforcement benefits skill acquisition
Learn. Mem.
Temporal dynamics of reward processing revealed by magnetoencephalography
Hum. Brain Mapp.
The effects of neural gain on attention and learning
Nat. Neurosci.
Cited by (16)
Punishments and rewards both modestly impair visuomotor memory retention
2021, Neurobiology of Learning and MemoryCitation Excerpt :The present results disconfirmed this work’s initial hypothesis by revealing that punishments and rewards both impaired retention. However, the qualitatively similar impairing influences of punishments and rewards on retention lends credence to the idea that punishments and rewards can yield similar effects at the neurophysiological levels (Hamel et al., 2018; Nieuwenhuis et al., 2005; Palminteri et al., 2015). The present retention impairments oppose previous lines of work showing that rewards enhance retention (Abe et al., 2011; Dayan et al., 2014; Galea et al., 2015; Hamel et al., 2019), but also resonate well with others showing impairing effects of rewards on adaptation (van der Kooij et al., 2018) and punishments on immediate retention (Hill et al., 2020).
Reward boosts reinforcement-based motor learning
2021, iScienceCitation Excerpt :Based on these elements and on the causal role of the striatum in reinforcement-based adjustments in motor commands (Nakamura and Hikosaka, 2006; Williams and Eskandar, 2006), we suspect that this region may be crucial for the beneficial effect of reward observed in the present study. Notably, the cerebellum (Carta et al., 2019; Heffley et al., 2018; Sendhilnathan et al., 2020; Vassiliadis et al., 2019; Wagner et al., 2017) and frontal areas (Dayan et al, 2014, 2018; Hamel et al., 2018; Palidis et al., 2019; Ramakrishnan et al., 2017; Sidarta et al., 2016) are also likely to contribute to reward-based motor learning. Further investigations are required to better delineate the neurophysiological bases of reward-related improvements in motor learning.
Association of short- and long-latency afferent inhibition with human behavior
2021, Clinical NeurophysiologyCitation Excerpt :Therefore, the reduction in SAI during reward presentation is likely not due to a change in corticospinal excitability. Alternatively, research has shown that presentation of monetary rewards increase beta power in M1 (Hamel et al. 2018). Given findings by Guerra et al. (2016), who showed that enhancing cortical beta power with transcranial alternating current stimulation (tACS) directed over M1 abolishes SAI, the reduction in SAI during reward presentation may be due to an increase in beta power.
Differential effects of reward and punishment on reinforcement-based motor learning and generalization
2023, Journal of Neurophysiology