Hippocampal activation during extinction learning predicts occurrence of the renewal effect in extinction recall
Introduction
Renewal in extinction learning describes the recovery of a response learned during acquisition in those cases when the context present during extinction is altered during extinction recall. The renewal effect was first demonstrated in an animal study (Bouton and Bolles, 1979), in which an association between a conditioned (CS) and an unconditioned stimulus (US) acquired in context A was extinguished by repeated presentations of the CS in context B, but recovered when the CS was presented again in context A. Variants of the renewal effect have been studied, with testing occurring in a third context (Bouton and Bolles, 1979, Harris et al., 2000) or in a novel context after acquisition and extinction in an identical context (Bouton and Ricker, 1994, Tamai and Nakajima, 2000). Among these versions, the ABA renewal effect described above is the most common (Bouton, 2004), observed in various different paradigms, such as appetitive conditioning (Bouton and Peck, 1989), fear conditioning (Bouton and King, 1983), and taste aversion learning (Rosas and Bouton, 1997) in rats, as well as in human fear conditioning (Vansteenwegen et al., 2005, Vansteenwegen et al., 2006) and predictive learning (Lachnit et al., 2008, Üngör and Lachnit, 2006, Üngör and Lachnit, 2008), where participants learn relations between cues and outcomes by making predictions and receiving feedback. The observation that extinguished responses can recover is interpreted as an indication that extinction does not erase an established association between CS and US but instead produces a competing CS–US association (Bouton, 2004, Myers and Davis, 2002). To evoke the renewal effect, the presented contexts presumably activate, or retrieve, the corresponding relation of the CS with the US (Bouton, 2004), illustrating the crucial role that context plays for extinction learning and recall. Thus it can be assumed that brain regions processing contextual information will have a prominent function for extinction learning and the renewal effect.
Among the brain structures considered to have a role in context-dependent memory are the hippocampus and prefrontal cortex. Animal studies on fear extinction that used permanent lesions (Good and Honey, 1991) or temporary inactivations of hippocampus (Corcoran and Maren, 2001, Corcoran et al., 2005, Hobin et al., 2006, Holt and Maren, 1999) demonstrated that hippocampal inactivation disrupts contextual retrieval of fear conditioning. The renewal effect in fear extinction in particular was found disrupted after lesions of the dorsal hippocampus (Ji and Maren, 2005). Lesions of ventromedial prefrontal cortex (vmPFC) led to disruption of extinction recall, suggesting a role for this region in consolidation of extinction or retrieval of contexts present during extinction (Quirk et al., 2000). Blockade of NMDA receptors in vmPFC caused deficits in consolidation of extinction (Burgos-Robles et al., 2007, Sotres-Bayon et al., 2009).
In general, vmPFC is thought to be involved in emotional decision-making (Deppe et al., 2005, Longe et al., 2009, Volz and von Cramon, 2009) under uncertainty (Northoff et al., 2006, Windmann et al., 2006) and under risk (Clark et al., 2008), in social decision-making, e.g. moral judgment (Koenigs et al., 2007) and economic games (Koenigs and Tranel, 2007). An overarching/related function of vmPFC that encompasses these functions might be encoding of stimulus value signals (FitzGerald et al., 2009, Hare et al., 2008). In this regard, vmPFC might have a role for context-based decisions.
Functional magnetic resonance imaging (fMRI) studies in humans have demonstrated differential activity of hippocampal and vmPFC regions during the phases of fear extinction learning. Analyses of functional connectivity found positive correlations between activation in hippocampus and vmPFC during extinction recall (Milad et al., 2007). A study by Kalisch et al. (2006) showed that activity in ventromedial prefrontal cortex (vmPFC) and hippocampus is context-dependent, as it was expressed only in the extinction but not in the acquisition context, furthermore that context-dependent fear extinction memory is correlated with activation in both the hippocampus and ventromedial prefrontal cortex. The authors suggest that hippocampus confers context dependence on vmPFC. Similarly, Corcoran and Quirk (2007) assumed that vmPFC receives contextual information from hippocampus to decide whether extinction or fear should be recalled. In contrast, a study by Lang et al. (2009) showed higher activation of hippocampus to the CS + than to the CS— during the acquisition phase of contextual fear extinction learning, but not during the extinction phase, hinting at hippocampal involvement during encoding of an association between context, CS and aversive US. Another study (Alvarez et al., 2008) also found higher hippocampal activation to the context that announced an aversive US than to the context that did not. In a similar line, Marschner et al. (2008) showed hippocampal activation during context-related but not cue-related acquisition of fear conditioning that decreased over time, suggesting a time-limited role for this region in associative learning. A recent study dissecting the roles of medial temporal lobe regions for processing item and context found parahippocampal cortex involved in initiating context-related recall/contextual processing and the hippocampus in item-context engram activation (Sadeh et al., 2012). Taken together, the results hint at an involvement of hippocampal structures both in associating contextual cues to a cue-outcome-coupling of CS and US as well as in delivering information about these contextual cues during recall of extinction, while vmPFC is active in the decision process regarding the proper response based on hippocampal information. Thus, both structures are presumably involved in processing the renewal effect of extinction.
A suggestion of why a change in context is important in extinction learning (Darby and Pearce, 1995) states that during initial conditioning context will be ignored, because it is irrelevant for solving the task. The unexpected change of contingencies between CS and US during extinction, however, will draw attention to the extinction context by means of an expectancy violation. Along these lines is an overarching view of hippocampus as a module that – triggered by violations of predictions – processes novelty signals (Kumaran and Maguire, 2007, Kumaran and Maguire, 2009) by performing match–mismatch computations in both the temporal and spatial domain, e.g. at the presentation of slightly altered stimuli or stimuli presented in a different order, which constitute a change to a previously presented sample. In this view, hippocampus is suggested to have a generic role in associative processing which might well account for its activation in contextual extinction learning.
Even though the vast majority of research on extinction learning has been done within the framework of fear extinction, several studies in both animals and humans suggest that brain areas which participate in fear extinction, i.e. prefrontal regions, hippocampus, and amygdala, are also involved in appetitive extinction learning. Lesions and inactivations of medial and orbital prefrontal regions impaired extinction of instrumental responses in macaque monkeys (Izquierdo and Murray, 2005) and rats (Chudasama and Robbins, 2003, Ghazizadeh et al., 2012, Peters and De Vries, 2013). In appetitive Pavlovian extinction, NA efflux in mPFC of rats was increased (Mingote et al., 2004). During successful operant extinction learning, activity in frontopolar OFC in humans increased while activity in mOFC decreased (Finger et al., 2008), compared to acquisition of the stimulus-reinforcement association. Hippocampal neurons respond to a conditioned stimulus whether it signals food or a shock, thus they are involved in both aversive and appetitive conditioning (Segal et al., 1972). Hippocampal lesions in rats impaired Pavlovian and operant appetitive extinction learning (Chan et al., 2003, Jarrard et al., 1986), in particular also contextual instrumental extinction learning (Good and Honey, 1991), in which a stimulus was reinforced in context A but not in context B. The amygdala appears to be necessary for processes that involve updating of representations of value, not only in fear extinction, but also in reinforcer devaluation (Morrison and Salzman, 2010). Accordingly, amygdala participation in appetitive extinction has been demonstrated for food-reward extinction learning (Judice-Daher et al., 2012, Portero-Tresserra et al., 2013), for extinction of responding to a visual CS which had been paired with food reward (Lindgren et al., 2003). Recent neuroimaging studies suggest that the human amygdala is involved in many reward-based decision-making tasks (Morrison and Salzman, 2010). These parallels between extinction with and without a fear component suggest that the neuronal mechanisms and the neurocircuitry mediating extinction are similar, irrespective of the valence of the conditioning process.
In the present study, we therefore used an associative learning task in which participants were required to learn relations between cues and outcomes presented in particular contexts, to investigate the neuronal correlates of the renewal effect outside a fear context. In a similar predictive learning task, the renewal effect was previously demonstrated in humans on a behavioral level (Üngör and Lachnit, 2006). We adapted this task for usage in an fMRI setting to compare brain activation patterns of participants who exhibited and did not exhibit the renewal effect.
We assumed that the brain regions involved in context processing in extinction would also figure prominently in the renewal effect. Our specific hypotheses were as follows: If hippocampus, as the literature suggests, is involved in associating a context with a changed cue-outcome relation, particularly if the context is of relevance for the organism, we expect stronger hippocampal activation in those extinction trials where the extinction context differs from that during acquisition than in trials where the context remains the same, since in the latter case the context is irrelevant. In particular, this stronger activation should occur in participants who actually process the context and subsequently show the renewal effect. If the renewal effect is mediated by hippocampus and ventromedial PFC, their activation in extinction learning and recall should distinguish activation patterns of participants who show the renewal effect from those who do not. If vmPFC is involved in retrieving contextual information during extinction recall, it should be activated predominantly in those participants who actually use the contextual information for their response, as opposed to those who do not.
Section snippets
Participants
28 healthy right-handed volunteers (14 females, 14 males), mean age 25,32 years +/− 3.878 years st.dev., range 20–30 years, without a history of neurological disorders, participated in this study after giving written informed consent. Prior to the experiments, participants received handouts informing them about the fMRI procedures. The protocol was approved by the Ethics Committee of the Ruhr-University Bochum. The study conforms to the Code of Ethics of the World Medical Association (Declaration of
Renewal-effect
The renewal effect (i.e. responding during the test phase according to the association learned during acquisition, taking context into consideration) in the complete group occurred – as expected – significantly more frequently in condition ABA than in condition AAA (t-test for matched samples t(28) = 4.559 p = .000). While in the ABA condition on average 2.448 (+/− 0.445 std. dev.) of 6 responses (40,8%) reported the association from the acquisition phase, in the AAA condition it was only 0.586 (+/−
Activation in hippocampus and vmPFC distinguishes between renewal and no renewal and thus mediates the renewal effect
Our results support our hypotheses that activation in hippocampus and vmPFC in extinction learning and extinction recall, respectively, are stronger in the REN group that uses the context for their decision than in the NOREN group that ignores or discards the contextual information, and thus the activation patterns in these regions distinguish between the occurrence of a renewal effect and the failure to show a renewal effect.
During extinction learning, the REN group exhibits higher activation
Acknowledgments
This work was supported by a grant from the DFG (FOR 1581 Extinction Learning). We thank Tobias Otto for programming the stimulus presentation software and Christine Westermann for support in performing the experiments.
Conflict of interest
The authors report no conflicts of interest.
References (109)
- et al.
The human amygdala plays a stimulus specific role in the detection of novelty
Neuroimage
(2011) - et al.
The role of the amygdala in the extinction of conditioned fear
Biol. Psychiatry
(2006) - et al.
The neural coding of expected and unexpected monetary performance outcomes: dissociations between active and observational learning
Behav. Brain Res.
(2012) - et al.
The influence of feedback valence in associative learning
Neuroimage
(2009) - et al.
A unique role for the human amygdala in novelty detection
Neuroimage
(2010) - et al.
Response and habituation of the human amygdala during visual processing of facial expression
Neuron
(1996) - et al.
Classical fear conditioning in functional neuroimaging
Curr. Opin. Neurobiol.
(2000) - et al.
Consolidation of fear extinction requires NMDA receptor-dependent bursting in the ventromedial prefrontal cortex
Neuron
(2007) - et al.
Human reversal learning under conditions of certain versus uncertain outcomes
Neuroimage
(2011) - et al.
Activation of different anterior cingulate foci in association with hypothesis testing and response selection
Neuroimage
(1998)