Affect-based impulsivity is defined as engaging in impulsive behaviors during an emotional state (Cyders & Smith, 2008). Affect-based impulsivity varies dimensionally in the population, and elevated levels have been observed in several psychiatric illnesses including borderline personality disorder, substance use disorders, bipolar disorder, and bulimia nervosa (Berg et al., 2015; Cyders & Smith, 2008). Longitudinal studies find that affect-based impulsivity predicts the onset and worsening of psychiatric symptoms (Cyders et al., 2010; Cyders & Coskunpinar, 2010; Manasse et al., 2018; VanderVeen et al., 2016). In addition, affect-based impulsivity portends risk for problematic drinking, pathological gambling, and compulsive shopping (Cyders & Smith, 2008). Affect-based impulsivity encompasses impulsive behaviors in response to both positive and negative emotions (Smith et al., 2007), forming a valence-independent factor that is more robustly associated with clinical outcomes than other facets of impulsivity (e.g., sensation seeking; Berg et al., 2015).
Research on the neurocognitive mechanisms of affect-based impulsivity has primarily considered four accounts (Johnson et al., 2020): (1) heightened emotion generation, (2) impaired emotion regulation, (3) risky decision-making, and (4) impaired response inhibition (see Fisher-Fox et al., 2024 for additional alternative accounts). Empirical studies have found little evidence of heightened emotion generation (Amlung et al., 2017; Cyders et al., 2010; Cyders & Coskunpinar, 2010, 2011; Johnson et al., 2017; Owens et al., 2018; Pearlstein et al., 2019; VanderVeen et al., 2016; Wise et al., 2015) and mixed evidence for risky decision-making (Cyders et al., 2010; Cyders & Coskunpinar, 2011; Johnson et al., 2016; Mackillop et al., 2014; MacKillop et al., 2016; Sharma et al., 2014; Wise et al., 2015). Conversely, affect-based impulsivity is reliably associated with the use of less effective emotion regulation strategies and weaker recruitment of brain regions involved in emotion regulation (Albein-Urios et al., 2013, 2014; King et al., 2018). Moreover, affect-based impulsivity is associated with impaired response inhibition (Cyders & Coskunpinar, 2011; Dekker & Johnson, 2018; Johnson et al., 2016; Sharma et al., 2014), particularly in clinical samples (Dekker & Johnson, 2018), and altered neural processing during inhibitory control tasks (Barkley-Levenson et al., 2018; Chester et al., 2016; Tervo-Clemmens et al., 2017; Wilbertz et al., 2014). Studies to date have primarily relied on summary indices of emotion processing or response inhibition (e.g., rate of inhibitory failures), which only provide indirect evidence about lower-level cognitive processes (Gureckis & Love, 2015; Love, 2015) and often have poor psychometric properties (Hedge et al., 2018). More recently, computational approaches have yielded insight into the generative processes that produce behavior (Huys, Guitart-Masip, et al., 2015; Montague et al., 2012), formally bridging behavior and the brain (Love, 2015; Palmeri et al., 2017). Even as research on the neurocognitive underpinnings of affect-based impulsivity has grown over the past 20 years, much less is known about how an emotional state shapes the neurocomputational processes that underpin impulsive decisions.
Here, we present a narrative review of affect-based impulsivity, propose an integrative decision neuroscience account of affect-based impulsivity, and outline a research agenda for testing central components of our account. We organize the review around the core components of our model. Given our focus, we primarily review human research that uses theory-based computational psychiatry methods (Bennett et al., 2019; Maia et al., 2017), as well as preclinical animal neuroscience studies that elucidate corresponding neural pathways (e.g., Cartoni et al., 2016; Glimcher, 2011; Niv et al., 2006). Nonetheless, we appreciate the value of other approaches, including individual differences research (Cyders & Smith, 2008; Fisher-Fox et al., 2024), ecological momentary assessment (Sperry et al., 2021), and cognitive and affective neuroscience (Johnson et al., 2020).
We narrow the scope of our review in a few key ways. First, although trait affect-based impulsivity varies across people (Cyders & Smith, 2008), our account focuses on the within-person effect of emotions on neurocomputational decision processes. That is, how do negative emotions alter neurocomputational processes, relative to that person’s baseline? We anticipate that this within-person account can inform our understanding of trait affect-based impulsivity (consistent with personality theories that view traits as a density distribution of states; Fleeson, 2001), since individual differences in computational and neural systems could lead to between-person differences (Huys, Guitart-Masip, et al., 2015). Second, impulsivity is not a unitary construct, referring to distinct behavioral tendencies that range from heightened delay discounting to inhibitory control deficits to an impulsive decision-making style (Caswell et al., 2015; Sharma et al., 2014). In our model, we focus on impulsive behaviors that are short-sighted and rash, reflecting a preference for immediate rewards, consistent with original psychological theories on trait affect-based impulsivity (Cyders & Smith, 2008; Smith et al., 2007).
Lastly, we narrow the scope of our account to focus on negative emotions, consistent with research finding distinct neural correlates for negative and positive emotions (Lindquist et al., 2012, 2016). Emotions are often associated with certain action tendencies (N. Frijda et al., 1989; N. H. Frijda, 1987; Izard, 2007; Lang & Bradley, 2013; Moors et al., 2013; Posner et al., 2005), and aversive states often inhibit behavior altogether (McNaughton & Corr, 2004). Given that negative emotions are often associated with inaction and withdrawal (e.g., de Berker et al., 2016), it is potentially counterintuitive that aversive internal states would promote pursuit of appetitive cues.
To explain such phenomena, we describe how emotion-dependent computations affect four aspects of decision-making: 1) whether to act, 2) what actions to consider, 3) which action to take, and 4) how vigorously to act. Although we describe these as sequential yet overlapping computations, we do not mean to imply that each stage involves conscious deliberation. Rather, we view these four aspects as interacting processes that may unfold rapidly and in parallel. We formalize key components of this model into two hypotheses: First, negative affect leads to an increase in circulating glucocorticoids (GCs) and norepinephrine (NE). GCs and NE blunt medial prefrontal cortex (mPFC) functioning (Arnsten, 2009; Schwabe et al., 2012), reducing goal-directed computations that support deliberative reasoning (Gläscher et al., 2010). Second, affect-related increases in GCs sensitize the mesolimbic dopamine (DA) pathway, enhancing the influence of Pavlovian reward cues on behavior (Peciña et al., 2006; P. V. Piazza & Le Moal, 1996). Altogether, this account provides new biological and neurocomputational targets for understanding impulsive decision processes in humans.
Aversive internal states shape decision-making
How might negative emotions enhance the pursuit of rewards? Let us consider an example. Ada and her boyfriend Darius are troubleshooting their TV connection. Ada considers different actions, with the goal of being able watch her favorite reality TV show, The Bachelor. Then, Ada receives a text asking her and Darius to bring a dessert to a potluck (Figure 1a). They disagree over what to bring, and this disagreement escalates into an argument. Ada notices a shift in her internal state (e.g., racing heart) and labels this state “shame”. She finds this state aversive and is now focused on reducing these feelings of shame. She decides to go on a walk, hoping that she will begin to feel calmer (Figure 1b). Her feelings of shame do not abate, and she finds it increasingly uncomfortable to feel this way. She encounters a sign for a bar. Having previously learned that alcohol can reduce feelings of shame, Ada enters the bar and imbibes heavily (Figure 1c). Though drinking reduces her aversive internal state in the short term, Ada returns home drunk, leading to further conflict with Darius.
Figure 1
Example of affect-based impulsivity. a) Baseline conditions. (i) Ada and Darius are watching a TV show but are experiencing technical difficulties. The picture resolution is poor due to the placement of the digital antenna. (ii) Ada considers different actions: watching the show on her laptop, fixing the TV, or continue watching the show on the TV (despite the poor resolution). (iii) Ada receives a text inviting her and Darius to a potluck the next evening. b) Interpersonal conflict leads to change in affective state. (i) Ada and Darius disagree over what baked good to bring to the potluck. (ii) Ada experiences an increase in negative affect and labels her emotional experience “shame.” (iii) Too distressed to continue the conversation with Darius, Ada decides to leave the apartment. c) Negative affect alters valuation of different actions and motivates impulsive behavior. (i) Ada’s intense emotional state persists, and she considers actions that she anticipates will reduce her current negative affective state: drinking alcohol, using a different emotion regulation skill (e.g., deep breathing), and returning home to talk through the conflict with Darius. (ii) While on her walk, Ada encounters a sign for a bar, which functions as a conditioned reward cue. (iii) Ada vigorously pursues the reward – alcohol – and quickly becomes drunk.
I. Deciding whether to act
When Ada left the apartment, she was acting to change her emotional state. We propose that Ada’s perception of whether she could change her state guided this decision (Table 1.1). Crucially, although we frame this as a ‘decision’ about perceived controllability, such computations are not necessarily deliberative and can be rapid and implicit, especially for more engrained behavioral patterns (Huys & Dayan, 2009). Whereas appetitive contexts promote approach (Panksepp, 2004; Wasserman et al., 1974), aversive contexts promote multifarious behaviors (Blanchard et al., 2005; Bolles, 1970; McNaughton & Corr, 2004) including fighting, fleeing, freezing, and passive avoidance (McNaughton & Gray, 2000). In an aversive context, the choice to act depends on how the threat is appraised: perceiving it as distant and beyond behavioral control elicits passive avoidance, while perceiving it as close and controllable (Boureau & Dayan, 2011) promotes active escape behaviors that exert control over the threat (e.g., fighting; Lloyd & Dayan, 2016; McNaughton & Gray, 2000).
Table 1
How does affective state shape decision processes across the four stages of a decision? N.B. Though our focus is on impulsive behaviors that are maladaptive, we anticipate that similar mechanisms may explain adaptive responses to an emotion (e.g., grizzly bear sighting ➔ fear ➔ freeze).
| STAGE OF DECISION-MAKING | COGNITIVE AND REINFORCEMENT LEARNING MECHANISMS | NEUROCOMPUTATIONAL AND NEUROENDOCRINE TARGETS | |
|---|---|---|---|
| i) | Whether to act? |
|
|
| ii) | Which actions to consider? |
|
|
| iii) | How to decide among actions? |
|
|
| iv) | How vigorously to engage in action? |
|
|
Deciding to act (i.e., active escape) versus not act (i.e., passive avoidance) recruits different learning systems. Passive avoidance largely depends on the Pavlovian system, which supports stimulus-outcome learning, such as which cues are associated with the threatening context (Sutton & Barto, 2018). By learning these associations, the organism can anticipate and avoid similar contexts in the future (Cartoni et al., 2016; Niv et al., 2006). In contrast, active escape principally depends on the instrumental system (Dorfman & Gershman, 2019; Moscarello & Hartley, 2017), which supports action-outcome learning (Sutton & Barto, 2018), including which behavior will eliminate a threat. From this vantage point, Ada’s belief that she can alter her feelings of shame motivates her to act, and her behavior is primarily under instrumental control.
II. Narrowing the set of actions actively considered
Emotions function as a form of metareasoning, shaping the states and actions an organism considers (Anderson & Adolphs, 2014; Huys & Renz, 2017; Levenson, 2011; Panksepp, 2004; Sander et al., 2018) and promoting emotion-congruent responses (Anderson & Adolphs, 2014; Levenson, 2011; Panksepp, 2004). We propose that features of internal states act as conditioned cues, shifting an organism’s goals and motivating actions to achieve these goals (Table 1.2; Cartoni et al., 2016). In our example, Ada’s shift in internal state (neutral to shame) redirects her goals from planning to watch The Bachelor (Figure 1a-ii) to reducing intense aversive feelings. When a walk fails to help, she considers additional emotion-congruent actions that align with her current goal – including (1) drinking alcohol, (2) using another emotion regulation skill, and (3) returning home to work through the argument with Darius (Figure 1c-i).
III. Evaluating the action set
Once the action set has been winnowed, an organism must arbitrate among the available actions. In RL terms, these instrumental actions are either under control of the habitual or goal-directed system (Dolan & Dayan, 2013). Goal-directed decisions depend on evaluating the expected value obtained by taking certain actions (Niv et al., 2006). In a simple environment, an organism learns the value of different actions using straightforward model-free computations, tracking the expected return for each action based on historical outcomes.1 However, in complex environments, like social interactions (FeldmanHall & Nassar, 2021), learning the value of different actions often relies on model-based algorithms (Dolan & Dayan, 2013) that are cognitively taxing (Otto, Gershman, et al., 2013). These algorithms require the organism to represent the environment’s structure, the relationship among states, and the action-outcome associations in each state (Daw et al., 2005; Gläscher et al., 2010). To illustrate, if guided by model-free learning, Ada would choose an action by comparing the cached values of actions that are under consideration. Under model-based learning, she would rely on her mental model of Darius to predict his likely response to each alternative action.
So how do emotions alter these RL systems? Negative emotions are often intensely aversive and drive an organism to quickly arbitrate among actions (McNaughton & Corr, 2004; Mobbs, 2018). In our view, emotions simplify both model-free and model-based components of this arbitration process (Table 1.3). First, strong negative emotions can blunt computationally expensive model-based reasoning (Otto, Raio, et al., 2013; Schwabe et al., 2010; Schwabe & Wolf, 2009), prompting a shift toward model-free learning (Mkrtchian et al., 2017). Second, vis-à-vis state-dependent learning (Dickinson & Balleine, 1994; Mollenauer, 1971; Tovote et al., 2015), the cached values of available actions are re-mapped based on what was learned about those actions in similar states. Instrumental outcomes that were learned to be valuable when in a similar state acquire incentive value that is contingent on state (e.g., learning to use a vending machine when feeling thirsty; Dickinson & Balleine, 1994). When arbitrating among available actions, this adjustment to the cached value alters the value gradient among available actions (e.g., using the vending machine is a more highly valued action than it is typically).
Third, even when an organism engages in model-based reasoning to consider the consequences of different actions, emotions may shape this simulation process itself (as suggested in Huys & Renz, 2017). Emotions are associated with attentional biases (MacLeod et al., 1986; Mathews & MacLeod, 2005), and action sequences that are incongruent with the emotion may be selectively ignored when simulating paths forward (Huys & Renz, 2017). For example, when thinking through the consequences of a particular action sequence, organisms frequently fail to consider the long-run value if they encounter an (imagined) large negative outcome in the sequence (Huys et al., 2012; Lally et al., 2017). This tendency to “prune” action sequences that involve imagined negative outcomes is greater in people with elevated anxiety and depressive symptoms (Huys et al., 2012; Lally et al., 2017), and attentional biases toward threat are frequently found in anxiety and depressive disorders (Mathews & MacLeod, 2005). In our example, Ada could simulate what would happen if she attempted to work through the argument with Darius. She predicts that they will both raise their voices and then storm off to separate rooms (an aversive outcome). She may stop the simulation there and choose among available actions based on their immediate value, rather than simulating how they could eventually make amends.
IV. Engaging in a selected action: the role of motivational vigor
Emotions not only alter the perceived value of different actions but also enhance the vigor of a selected emotion-congruent action. We propose that appetitive Pavlovian-to-Instrumental Transfer (PIT; Table 1.4) is a key pathway through which negative affect enhances the pursuit of rewards (Cartoni et al., 2016). Emerging evidence indicates that this PIT-related invigoration results from the “opportunity cost” of not acting (Boureau & Dayan, 2011; Niv et al., 2006). That is, when an organism perceives that an aversive state can be improved (e.g., active escape), then the gap between the current and preferred states is wide, making each moment of inaction feel costly.
Suppose Ada has had a history of tumultuous relationships, and she thus experiences relationship discord as particularly distressing (via an aversive Pavlovian association). Consequently, the argument with Darius is especially distressing to her. Each moment of inaction feels costly, driving actions to reduce negative affect. When she decides to drink alcohol, she does so vigorously (McNamara et al., 2024). Had Ada’s prior relationships been more stable, her distress in this situation might have been less acute. The perceived cost of inaction would have been lower, and her motivation to escape these feelings would be weaker.
Neurocomputational Account of Affect-Based Impulsivity
Thus far, we have described a model of affect-based impulsivity in cognitive terms: 1) Deciding to act depends on perceptions of controllability; 2) The Pavlovian system sculpts which actions are considered, enhancing emotion-congruent actions; 3) Intense negative emotions blunt model-based reasoning, altering valuation of the action set; and 4) Appetitive PIT explains enhanced motivational vigor for the selected emotion-congruent action. We now turn to the neurobiological basis of these affect2-related shifts in decision-making. First, we propose that the decision to act depends on computations of controllability that are encoded in the prefrontal cortex (PFC). Second, which actions are considered depends on affective state. Finally, we describe the roles of dopamine (DA), glucocorticoids (GCs), and norepinephrine (NE) in, third, altering the valuation of available actions and, fourth, amplifying vigor for selected actions.
I. The brain basis of controllability
Goal-directed decision-making relies on the expectation that an action will be instrumentally effective in achieving a desired outcome, such as alleviating a negative emotional state. Recent findings in computational cognitive neuroscience show that humans dynamically track the predictability (Dorfman & Gershman, 2019) and controllability (Ligneul et al., 2022) of an environment, and these computations occur in brain regions such as the PFC (Ligneul et al., 2022), bed nucleus of the stria terminalis (BNST), insula, and posterior cingulate gyrus (Limbachia et al., 2021). Critically, activity in these regions governs whether a stressor leads to action or inaction (i.e. “learned helplessness”; Amat et al., 2005; Table 1.1).
Building on these insights, we propose that computations of controllability are encoded in the PFC (Ligneul et al., 2022), as well as related brain regions (Limbachia et al., 2021), and gate the extent to which DA or serotonin predominates over behavior. Historically, DA has been thought to guide approach behavior in appetitive contexts and serotonin to inhibit punished behaviors3 in aversive contexts (Boureau & Dayan, 2011). However, DA is involved in motivating behavior in both appetitive and aversive contexts (Boureau & Dayan, 2011; Guitart-Masip et al., 2014). In aversive contexts, by virtue of bringing the organism into a more desirable state, escape is encoded as a reward, and DA invigorates behaviors that help the organism escape (Lloyd & Dayan, 2016). Crucially, these DA-dependent behaviors only occur when the outcome is perceived as controllable (for review, see Boureau & Dayan, 2011). When behavior has no effect on the aversive environment, serotonin is released into mPFC and inhibits behavior (e.g., in learned helplessness experiments; Bland et al., 2003; Boureau & Dayan, 2011).
II. Affective brain state shapes the action set
Once an organism decides to act, its emotional state promotes affect-congruent actions – a process that depends on a corresponding brain state. Though there is debate about how to conceptualize and study emotions (Anderson & Adolphs, 2014; Barrett et al., 2007; Levenson, 2011; Panksepp, 2004; Sander et al., 2018), certain neural systems are nonetheless consistently implicated in emotional experiences (Table 1.2). Emotional experiences often begin when a motivationally relevant stimulus is detected, engaging orienting processes (Sander et al., 2018) supported by the ventral attention system (e.g., temporoparietal junction; Kincade et al., 2005) and the salience network (e.g., dorsal anterior cingulate cortex, anterior insula; Seeley et al., 2007). Then, there is a cascade of neuropeptides that modulate neural activity and organize a shift in internal state (Flavell et al., 2022). In certain instances, such as the presence of a threat, corticotropin-releasing hormone (Vale et al., 1981) is released and facilitates the stress response.
In parallel, brain systems regulating the autonomic nervous system (e.g., hypothalamus) modulate sympathetic and parasympathetic activity (e.g., increased heart rate). These changes are detected through interoception (primary and secondary somatosensory cortices) and are integrated into a multimodal representation of the body (insula; Critchley et al., 2004). As additional information about the stimulus is gathered through sustained attention, the estimate of the stimulus’s relevance is adjusted, which may further modulate relevant circuits (Gross, 2015). Conceptual knowledge, prior experience, and language may also inform estimates of relevance (Barrett et al., 2007).
Once this affective brain state has been constructed, there are many pathways by which state shapes how the action set is narrowed (Table 1.2; Tovote et al., 2015). Some affect-congruent behaviors (e.g., freezing) are hard-wired, mediated by evolutionarily preserved midbrain structures (periaqueductal gray; Graybiel, 2008). Other pathways involve hippocampal replay, which helps retrieve actions that were effective when previously experiencing a similar emotional state (Carr et al., 2011). Co-activation (Tovote et al., 2015) of the same circuits that constitute an affective state (Barrett et al., 2007) may also shape the set of actions being considered (Huys & Renz, 2017).
III. Negative affect hampers goal-directed computations: the roles of glucocorticoids and norepinephrine
Once the action set has been narrowed, the organism selects among available actions using a combination of learning algorithms, which help the organism select the action associated with the highest expected value. Our model proposes that negative affective states lead the organism to rely primarily on model-free reasoning and to use shortcuts that simplify model-based computations. Building on the established links of stress and negative affect with GCs and NE (Blair et al., 2008; W. A. Brown & Heninger, 1975; Dickerson & Kemeny, 2004; J. R. Piazza et al., 2013), we consider how increases in GCs and NE may account for this shift in reasoning (Table 1.3).
Stress has widespread effects on cognition (Lupien et al., 2007; Starcke & Brand, 2012), including reduced cognitive performance for computations depending on mPFC functioning (Arnsten, 2009; de Berker et al., 2016; Schwabe & Wolf, 2013). These effects of stress are mediated by changes in intracellular signaling of PFC cells (Arnsten, 2009). NE enhances coherent firing of cells receiving similar information, thereby increasing the “signal” in brain regions performing a cognitive function. Conversely, DA in PFC decreases cell firing in response to motivationally irrelevant information, thus reducing the “noise” in surrounding brain regions. The interaction between NE and DA yields an inverted-U relationship between NE and DA on PFC functioning (Arnsten, 2009). At moderate levels of prefrontal NE and DA, neuronal ensembles can effectively communicate to perform complex computations. Yet, when NE and DA are very high, altered signaling in PFC interferes with effective communication among ensembles needed for complex computations. This observation aligns with the broader literature on stress-induced modulation of GCs and NE that blunt goal-directed decision-making and reduce model-based learning (Otto, Raio, et al., 2013; Schwabe et al., 2010, 2011, 2012). We suggest that affect-dependent reductions in model-based reasoning are mediated by these stress-related changes in PFC functioning.
IV. Affect-related changes in the mesolimbic dopamine reward circuit account for invigorated pursuit of reward
In conjunction with these functional changes in PFC, we further propose that negative affect enhances Pavlovian influences on behavior – including invigorated pursuit of rewards – through its effects on the mesolimbic reward system (Table 1.4). To provide context, let us first consider how DA invigorates behavior.
Enhanced pursuit of rewards: the role of dopamine. Heightened mesolimbic DA reactivity to reward cues increases experiences of “wanting,” or craving (Berridge & Robinson, 2016). “Wanting” of the cue, which is related to sign-tracking, typically involves increased behavioral engagement with the reward cue, partly via Pavlovian mechanisms (Anselme et al., 2013; Morrison et al., 2015). Individual differences in sign-tracking predict greater PIT-dependent (Garofalo & di Pellegrino, 2015) approach toward and actions involving the reward cue itself (e.g., a rat licking the lever that predicts food). In the context of aversive states where the outcome is perceived as controllable, DA invigorates escape behaviors that promote safety and that reduce the aversiveness of the organism’s internal state (Boureau & Dayan, 2011).
The nucleus accumbens (NAcc) is a central node of the mesolimbic reward circuit involved in learning from surprise: DA-related modulation of NAcc scales parametrically with the extent to which the outcome is better than expected (Glimcher, 2011). NAcc has two subregions, core and shell, which exhibit dissociable roles in learning. Whereas the core tracks the reward rate of the environment, enhances motivation, and invigorates behavior toward any reward cue (e.g., sign for bar increasing pursuit of substances), the shell motivates behavior toward rewards that are associated with specific cues (e.g., sign for bar increasing pursuit of alcohol; Corbit & Balleine, 2005; Floresco, 2015). Indeed, sign-tracking depends on DA reactivity in the NAcc shell (DiFeliceantonio & Berridge, 2012; Mahler & Berridge, 2012; Morrison et al., 2015; Warlow et al., 2017), consistent with its broader role in PIT (Corbit & Balleine, 2005).
How does negative affect alter dopamine activity in reward circuits? Stressors frequently elicit negative affect (Dickerson & Kemeny, 2004) and alter reactivity to reward cues. Stress sensitizes DA receptors in NAcc shell, boosting firing rate in response to reward cues and increasing pursuit of specific rewards (Marinelli & Piazza, 2002; Peciña et al., 2006; P. V. Piazza & Le Moal, 1996). The sensitization of mesolimbic reward circuitry is partly driven by glucocorticoids (GCs): high levels of circulating GCs activate NAcc shell glucocorticoid receptors (GRs; Marinelli & Piazza, 2002), enhancing DA-dependent activity and increasing the Pavlovian influence of reward cues (Marinelli & Piazza, 2002). Thus, during negative emotional states, concurrent increases in GCs enhance sensitivity to reward cues and motivate vigorous pursuit of these rewards.
Paths Forward
We have articulated a decision neuroscience account of affect-based impulsivity that provides specific testable hypotheses regarding neuroendocrine systems, neural substrates, and neurocomputational processes. As shown in Figure 2, negative affect leads to increases in GCs and NE, which in turn dampen goal-directed decision-making by impairing mPFC functioning (Arnsten, 2009; Schwabe et al., 2010, 2011, 2012; Schwabe & Wolf, 2013). Second, and in parallel, GCs sensitize the mesolimbic DA pathway (Marinelli & Piazza, 2002; P. V. Piazza & Le Moal, 1996; Rougé-Pont et al., 1993), thereby increasing the Pavlovian influence of reward cues on decision-making (Berridge & Robinson, 2016; Peciña et al., 2006) and invigorating pursuit of outcomes associated with these cues.
Figure 2
A stressor (1) induces negative affect. Increase in negative affect elicits concomitant changes in circulating levels of (2) norepinephrine (NE) and glucocorticoids (GCs). (3) GCs enhance dopamine (DA) reactivity in nucleus accumbens (NAcc) shell, and NE and GCs blunt medial prefrontal cortex (mPFC) functioning. (4) Altered functioning in mPFC and NAcc shell results in altered balance of Pavlovian and goal-directed decision systems. Heightened influence of Pavlovian systems on decision-making amplifies influence of reward cues and (5) invigorates pursuit of rewards.
Investigating these hypotheses requires advanced methods that can dissect a decision into latent components and corresponding neural circuits. Although many neuroscientific approaches may be used to study the neural bases of affect-based impulsivity, we consider decision neuroscience to be particularly promising. Decision neuroscience integrates Bayesian decision theory (Dayan & Daw, 2008; Huys, Daw, et al., 2015) with model-based cognitive neuroscience (Palmeri et al., 2017) to explore the neurocomputational mechanisms of behavioral phenomena (Dreher & Tremblay, 2016). This approach has advanced our understanding of the mechanisms involved in perceptual, value-based, and social decision-making (Dreher & Tremblay, 2016). Pharmacological and neuromodulatory methods promise to further strengthen the inferences of this research. Whereas traditional clinical neuroimaging methods are correlational (e.g., fMRI; Vytal & Hamann, 2010), transcranial direct-current stimulation (Stagg & Nitsche, 2011), transcranial magnetic stimulation (Wassermann et al., 2008), and pharmacological manipulations allow researchers to manipulate neural systems to test the causal role of a circuit or a brain region, potentially advancing mechanistic accounts of psychological phenomena (Allen et al., 2020).
Using these methods, our hypotheses could be tested effectively in four stages: First, test effects of a negative affect induction on learning and decision-making (Figure 2-1) in a task that can distinguish between Pavlovian and goal-directed decision systems (Figure 2-4; Gläscher et al., 2010; Huys et al., 2012). Second, link concomitant changes in GCs and NE to the balance of these decision systems (Figure 2-2). Third, use fMRI to identify neural correlates (Figure 2-3). Finally, employ a pharmacological manipulation of GCs and NE to characterize their role in shifting network dynamics and neural circuits. Together, these steps could reveal new insights into the neural bases of how negative affect alters decision-making, inform existing neurocognitive accounts of affect-based impulsivity, and yield findings with translational implications.
Limitations and Future Directions
Our account focuses on the within-person effect of negative affect on decision-making, but does not address why people vary in proneness to affect-based impulsivity (Berg et al., 2015; Cyders & Smith, 2008). We anticipate that our model could be expanded to address this open question. For instance, people vary in their appraisals of controllability, with low perceptions of control over threat linked to anxiety and depressive disorders (Cheng et al., 2013). How people tend to appraise controllability could impact whether they act to alter their emotion. Relatedly, people vary in the extent to which they tend to rely on model-based reasoning (Gillan et al., 2016; Patzelt et al., 2019), and trait affect-based impulsivity is associated with lower model-based reasoning – independent of a person’s current emotional state (Patzelt et al., 2019). This deficit in model-based reasoning could compound the effect of negative emotions on action selection, perhaps via the mechanisms we proposed above. Third, our account emphasizes that affect-related increases in GCs lead to heightened DA reactivity in the mesolimbic reward circuit (Figure 2). There are between-person differences in GR sensitivity to GCs, reflecting effects of genetics and chronic stress (Kosten et al., 2002; Ortiz et al., 1996; Rougé-Pont et al., 1993), and this sensitivity impacts the potency of GCs on DA reactivity (P. V. Piazza & Le Moal, 1996). A natural next step in developing our account would be to examine whether GR sensitivity is related to trait affect-based impulsivity.
It is also worth highlighting constructs and phenomena that are not addressed in the present account of affect-based impulsivity. First, our proposal focuses on how negative affect – not positive affect – enhances impulsive behaviors. Second, our proposal does not directly consider explicit emotion regulation strategies that people commonly deploy, including suppression and reappraisal (Gross, 2015). Indeed, enacting effective emotion regulation strategies may fall within the set of options a person evaluates when considering how to respond to their emotional state. Third, negative affect may increase habitual behavior (Schwabe et al., 2010, 2012; Schwabe & Wolf, 2009, 2013), yet our model does not directly address the role of habit. Notably, habitual control relies on model-free learning systems (for review, see Dolan & Dayan, 2013). Thus, it may be fruitful to extend our account to consider habitual impulsive behaviors. Such an extension would require further consideration of how repeated experiences consolidate model-free representations into inflexible stimulus-response policies, and whether this process is dependent on affect-based impulsivity. Fourth, certain emotions affect the decisiveness with which a person takes action (e.g., anger promotes decisiveness; Lerner & Tiedens, 2006). Altered decisiveness may be related to lower-level cognitive processes like decision threshold in drift diffusion models (Ratcliff & McKoon, 2008). Fifth, we primarily focus on the role of GCs and NE, yet other neuroendocrine systems (e.g., oxytocin) interact with GCs and NE, affect DA reactivity, and may separately alter the same decision systems through other pathways (Crockett & Fehr, 2013; Huys et al., 2012). Finally, even as we have described how emotions impact distinct stages of decision-making, which are relevant to impulsive behaviors, not all actions that are a response to an emotion lie within the scope of the model. For example, some affect-congruent responses are reflexive or hard-wired (e.g., freezing). As these behaviors are not learned, our hypotheses on arbitration are irrelevant. Similarly, the ways in which emotions alter simulations in model-based control is only relevant in environments that are complex enough to necessitate model-based reasoning (though it’s worth noting that model-based computations are evident even in circumstances previously assumed to not necessitate model-based reasoning; Collins & Frank, 2012).
There are several methodological limitations that we must also acknowledge. Extant research on the neurocognitive substrates of trait affect-based impulsivity has been conducted in humans, and we must contend with the limitations present within the field of human neuroscience, including high type-I error rate (Szucs & Ioannidis, 2020), small clinical samples (Marek et al., 2020; Poldrack et al., 2017; Szucs & Ioannidis, 2020), different preprocessing and analysis methods (Collaboration, 2015; Esteban et al., 2019; Ioannidis, 2005; Power et al., 2012), and often poor reliability of behavioral tasks (Hedge et al., 2018). Use of larger samples and of state-of-the-art acquisition, preprocessing (Esteban et al., 2019), and analysis (V. M. Brown et al., 2020; Haines et al., 2020; Price et al., 2019; Rouder & Haaf, 2019) methods would help address these concerns. Finally, additional challenges arise when employing a decision neuroscience approach. Researchers must specify, estimate, and test plausible computational models of behavior (Daunizeau et al., 2014; Kruschke, 2014).
Conclusion
We propose a model of affect-based impulsivity that explains behavior during negative affective states in terms of Pavlovian and goal-directed decision systems. Our model proposes that affect-related increases in glucocorticoids (GCs) and norepinephrine (NE) shift the balance of these decision systems in two key ways: (1) they blunt mPFC functioning, reducing goal-directed decision-making, and (2) they sensitize the mesolimbic DA pathway, enhancing vigorous pursuit of rewards. Decision neuroscience methods provide a framework for testing this account (Gureckis & Love, 2015; Love, 2015). Pharmacological and neuromodulatory methods are well suited for interrogating related neurocomputational systems (Allen et al., 2020). Altogether, a decision neuroscience account of affect-based impulsivity can advance our understanding of this transdiagnostic construct and yield new treatment targets for several psychiatric disorders.
Notes
[1] Model-based and model-free learning are not categorically different, but are instead thought to lie on a continuum. These systems operate independently and in parallel, with computations from both systems guiding decision-making.
[2] We use affect and emotion interchangeably when discussing the brain basis for how internal state alters decision processes, though we recognize that emotion theorists argue for a distinction between the two (Barrett et al., 2007).
[3] Though serotonin is most consistently implicated in behavioral inhibition, this effect is not universal. For example, SSRIs can increase escape behaviors during a forced swim test in rats (Detke et al., 1995).
Acknowledgements
We would like to thank Eric A. Youngstrom, Stacey B. Daughters, Timothy A. Allen, and Alexandre Y. Dombrovski for offering helpful feedback on earlier versions of this manuscript. This manuscript is published as a preprint on PsyArXiv: https://osf.io/preprints/psyarxiv/jvmzs.
Author Contributions
AMS – conceptualization, visualization, original draft, and editing; MNH – review and editing.
submission-comments
In this review article, the authors propose an overarching theory of impulsive behavior in response to negative affect through the lens of decision neuroscience. There is much to like about this paper. It is well written, it clearly presents its ideas and provides a bird’s eye view of how neural systems could be contributing to supporting these processes, which I think would be useful to a wide audience. While I find this narrative review compelling and worth publishing, I do have a few questions about the logic and the claims. I think addressing these could be helpful in improving its impact.
Major points:
I find the core of this conceptualization compelling. That is, that emotional state shapes arbitration between model-free and model-based processing. In the manuscript this component corresponds to section III. The authors explain three mechanisms that possibly explain a shift from model-based to model-free decision-making. First, emotion-driven restriction of computationally-expensive processes; second, re-mapping of cached values for available actions based on previous learning in the same emotional state; and third, negative emotions can themselves tint the possible outcomes and values simulated in a model-based way. In my opinion, these three mechanisms should be better explained, expounded on, and substantiated with available evidence. Particularly, the second mechanism seems very superficially described. I am not sure what is specifically meant by “re-mapped” cached action values. Perhaps, providing concrete examples of this and even a descriptive figure would be ideal to clarify these ideas.
As I mentioned, section III on action set evaluation seems particularly sound to me. However, I am less convinced about sections I and II are necessary processes. That is, if reacting to negative affect is already a learned response, the agent need not “decide whether to act”. Similarly, the learned reactive behavior may be so automatic that the agent may not even consider alternative responses, thereby bypassing the “narrowing action set” step. Can the authors clarify the importance of these steps in their general conceptual model or alternatively, explain that these steps are relevant for the specific example used to illustrate it? Could it be that these processes are relevant before behavioral patterns are learned and become the default?
Section I on deciding whether to act makes this process sound deliberative. The agent has some belief over controllability and selects a type of behavior on the basis of this evaluated belief. Is there evidence that this is conscious or intentional? It is also possible that the selection of different models of behavioral control (passive versus active) could be associated with the presence of other existing comorbidities, for example, anxiety, which this section does not discuss.
Emotion regulation processes like reappraisal and suppression are not really part of the neurocomputational process proposed here but are undoubtedly important (and even present in the example story of Ada, i.e. taking a walk, working things out with Darious, etc). I think the authors should acknowledge that this is a missing piece or explain why it is beyond the purpose of the review.
Minor points
I suggest expanding the references cited the section on the neurocomputational bases of controllability to include Bland et al., 2003 https://www.nature.com/articles/1300206, and to mention that the circuit extends beyond prefrontal cortex (Limbachia et al., 2021 https://www.nature.com/articles/s42003-020-01537-5?fromPaywallRec=true).
What exactly is meant by impulsivity in this paper? A rash decision? A model-free decision? A risky decision? A suboptimal decision? A shift in delay discounting? I think it would be good to define impulsive behavior a bit better here.
Typo on page 13: “These effects of stress are (be) mediated by changes…”
peer-review-recommendation
Revisions Required
submission-comments
Schreiber and Hallquist review literature on computational mechanisms underlying affect based impulsivity. Briefly, the review seems to revolve around the same conundrum as that outlined by Huys and Renz, namely that pruning the model-based system provides an elegant account of the impact of emotion on decision making but that it is somewhat underdetermined, and valuation changes could have similar impact. Overall, the review is difficult to read, achieving baffling levels of self-contradiction. It is true that there are some interesting review articles written at the interface of phenomenological and behavioral (e.g. decision making paradigms) research which can reach across traditional research ‘silos’ and inspire fresh thinking. Whether or not such laudable aims drove the present work, perhaps some reconsideration of the authors’ objectives might be worthwhile.
Stylistically, the authors allow themselves to cite selectively - key empirical work on impulsivity and model-based decision making is not cited (e.g. Gillan/Daw elife, Patzelt/Gershman Biological Psychiatry) - and yet rule out a priori discussion of the strongest body of empirical work (response inhibition), on the grounds that it traditionally not analyzed using generative models. If generative models are so important, why not cite key work on impulsivity which has actually used them? If generative models are not important, why not cite the strongest body of empirical work? And then, what if generative models (e.g. accumulator models) were to be used to analyze response inhibition data? Are these data allowed back ‘in the fold’?
At some point the authors state that they are most interested in within-subject changes in emotion. But between subject difference are still apparent, at least implicitly, to derive a contrast. If the Pavlovian system is ‘more’ engaged, it is more engaged relative to what? A past version of the individual, or another person? Balanced discussion of within and between participant effects is generally typical throughout psychiatry, and individual differences are at the heart of psychiatry - this is a psychiatry journal.
On page 4 the modelling approach is described as ‘new’, but it draws heavily on existing accounts, many of which have been well rehearsed in the literature. The models in question are explanatorily powerful, meaning they can be applied widely as other authors have found.
On page 5, the authors ‘appreciate’ the value of cognitive neuroscience, and thankfully proceed to cite numerous such studies. Animal and computational models have great value in building accounts of impulsivity, but ideally we would want human cognitive neuroscience also to have a central role within a translational research program of impulsivity.
It isn’t counter-intuitive that aversive states should promote pursuit of appetitive cues - mood repair is obvious and well established within the addiction literature (e.g. by Koob and others).
Page 12 - serotonin doesn’t inhibit behavior, at least not universally. SSRIs increase escape behavior on a forced swim test. A citation is needed for DA increasing approach behavior when the outcome is perceived as controllable - it is true that DA does interact with contingency but approach behavior could simply mean approaching Pavlovian cues.
On page 19, the authors claim that their model does not address the role of habit, yet discuss cached values and a model-free system. It is possible to differentiate model free and habitual systems, but ultimately these terms arise from different paradigms but refer to something broadly similar. Overall, the authors should choose whether they are committing to a rather specific language in which e.g. model-free learning and habit can been distinguished, or a rather loose language in which ‘Pavlovian learning’ is unitary.
The review doesn’t appear to distinguish between impulsive action or choice, and within choice, risk taking and delay discounting. Again, this may be as a result of a desire for internal consistency with the cost of generalizability/scope.
peer-review-recommendation
Resubmit for Review
submission-comments
The authors have addressed all my suggestions satisfactorily.
peer-review-recommendation
Accept Submission
submission-comments
The authors have made some improvement to the work. While these ideas, and similar ones, are popular in the field, I struggle with the fact that there seems to be several distinct computational routes to achieve the same outcome in this account. Emotions could bias decision making via a conditioned stimulus-response pathway; they could prune the potential model-based action repertoire to be considered; they could change the valuation of model-based actions; they could bias towards model free over model based control. Maybe this ambiguity serves as a form of encouragement for experimentalists to pin these ideas down, but usually you would look to theory to help with this rather than the other way around.
There is work by Brown/Price and others showing improved psychometrics with computational methods (already cited), but overall there is no guarantee that computational modelling offers any psychometric advantage. We would expect a psychometric improvement, of course, but in practice this is not necessarily the case. For example, Collins et al have had difficulty demonstrating generalizability across different assessments of learning rate parameters. I suppose one could say that you could keep fitting different models until psychometric improvement was observed, given that if structured variance that would drive test-retest reliability is ignored by one set of models, other models could be found which capture this variation. But it could be A) modest or B) irrelevant to the construct of interest. At bottom, then, psychometrics seems to be a non-sequitur and I’m concerned that any claim of a priori improvement is potentially misleading. It’s fairly obvious to the reader that the authors didn’t want to talk about response inhibition because it was outside of their focus on decision making rather than any psychometric issue.
The within/between person issue still stands - if a claim is made, i.e. Ada drinks vigorously, this implies a high intake on a relative or an absolute scale. How could such an absolute scale be defined? Generally we think of alcohol consumption within (human) culturally specified terms - hence, a relative scale - so, between subjects. It’s possible to think about purely in terms of pharmacological indices, but how could we define what ‘vigorous’ is in these terms? I don’t see how a purely within-subject account can be achieved in this context. I suppose an analogy would be sporting achievement - one could say that X improved their running time by some amount, but it would be natural to understand that achievement by reference to others. It might be possible to build a bioengineering account of human capacities, and show that this time is close to what can be achieved given biological constraints. But in this manuscript, almost all of the background work, citations etc, is all from a between-subject point of view, and the kind of framework that would be needed to establish within-subject assessment in absolute terms (i.e. independent of norming to a population) is not introduced. Concretely - let’s say I don’t drink anything all year, but then have a few glasses of wine at Christmas - in percentage terms, my intake and blood alcohol have gone up enormously (potentially infinite if it is normally zero). But I would guess this isn’t what the authors are getting at.
peer-review-recommendation
Revisions Required
Reviewer 1:
Reviewer E
In this review article, the authors propose an overarching theory of impulsive behavior in response to negative affect through the lens of decision neuroscience. There is much to like about this paper. It is well written, it clearly presents its ideas and provides a bird’s eye view of how neural systems could be contributing to supporting these processes, which I think would be useful to a wide audience. While I find this narrative review compelling and worth publishing, I do have a few questions about the logic and the claims. I think addressing these could be helpful in improving its impact.
Author’s response
Thank you for your thoughtful comments. We appreciate the critical points you raise below, as well as the positive comments about the strengths of the paper you highlight above. Altogether, we believe your comments have substantively strengthened our paper, and we are hopeful that we have sufficiently addressed your concerns.
Reviewer E
Major points:
1. I find the core of this conceptualization compelling. That is, that emotional state shapes arbitration between model-free and model-based processing. In the manuscript this component corresponds to section III. The authors explain three mechanisms that possibly explain a shift from model-based to model-free decision-making. First, emotion-driven restriction of computationally-expensive processes; second, re-mapping of cached values for available actions based on previous learning in the same emotional state; and third, negative emotions can themselves tint the possible outcomes and values simulated in a model-based way. In my opinion, these three mechanisms should be better explained, expounded on, and substantiated with available evidence. Particularly, the second mechanism seems very superficially described. I am not sure what is specifically meant by “re-mapped” cached action values. Perhaps, providing concrete examples of this and even a descriptive figure would be ideal to clarify these ideas.
Author’s response
We are heartened to hear that you find core features of our model compelling, and we appreciate the importance of fully describing each pathway by which emotions shape the arbitration of actions. In the revised text, we have significantly expanded on this section (pp. 10-11):
First, strong negative emotions can blunt computationally expensive model-based reasoning (Otto, Raio, et al., 2013; Schwabe et al., 2010; Schwabe & Wolf, 2009), prompting a shift toward model-free learning (Mkrtchian et al., 2017). Second, vis-à-vis state-dependent learning (Dickinson & Balleine, 1994; Mollenauer, 1971; Tovote et al., 2015), the cached values of available actions are re-mapped based on what was learned about those actions in similar states. Instrumental outcomes that were learned to be valuable when in a similar state acquire incentive value that is contingent on state (e.g., learning to use a vending machine when feeling thirsty; Dickinson & Balleine, 1994). When arbitrating among available actions, this adjustment to the cached value alters the value gradient among available actions (e.g., using the vending machine is a more highly valued action than it is typically).
Third, even when an organism engages in model-based reasoning to consider the consequences of different actions, emotions may shape this simulation process itself (as suggested in Huys & Renz, 2017). Emotions are associated with attentional biases (MacLeod et al., 1986; Mathews & MacLeod, 2005), and action sequences that are incongruent with the emotion may be selectively ignored when simulating paths forward (Huys & Renz, 2017). For example, when thinking through the consequences of a particular action sequence, organisms frequently fail to consider the long-run value if they encounter an (imagined) large negative outcome in the sequence (Huys et al., 2012; Lally et al., 2017). This tendency to “prune” action sequences that involve imagined negative outcomes is greater in people with elevated anxiety and depressive symptoms (Huys et al., 2012; Lally et al., 2017), and attentional biases toward threat are frequently found in anxiety and depressive disorders (Mathews & MacLeod, 2005).
Reviewer E
2. As I mentioned, section III on action set evaluation seems particularly sound to me. However, I am less convinced about sections I and II are necessary processes. That is, if reacting to negative affect is already a learned response, the agent need not “decide whether to act”. Similarly, the learned reactive behavior may be so automatic that the agent may not even consider alternative responses, thereby bypassing the “narrowing action set” step. Can the authors clarify the importance of these steps in their general conceptual model or alternatively, explain that these steps are relevant for the specific example used to illustrate it? Could it be that these processes are relevant before behavioral patterns are learned and become the default?
Author’s response
We agree that certain components of our model are more relevant when a behavioral pattern is emerging and has not yet become fully engrained. For example, model-based learning is more common early in learning. Consequently, the effects of emotions on the simulation process may be more pronounced early in learning.
We nonetheless contend that these steps can unfold very quickly and do not necessarily require deliberative reasoning or conscious thought. In our view, it is likely the case that each stage of the decision process is relevant for most instrumental actions that occur in response to a negative emotion. For example, if a person sees a grizzly bear 100 yards away, they will quickly appraise whether that bear is close enough to be within behavior control. Conversely, if the bear is quite close, then they must act swiftly to save their life. In a state of fear, their goal shifts (to staying alive). Fear-congruent responses (flight, flee, freezing) predominate the action set, and they evaluate which action to select using simplified computations. Whichever action they chose, it is likely they do so vigorously (given the urgency of the situation). Thus, even in this scenario where a person must respond very quickly, each stage of the decision process still unfolds. We now include text to more fully explicate ours views on this topic:
Although we describe these as sequential yet overlapping computations, we do not mean to imply that each stage involves conscious deliberation. Rather, we view these four aspects as interacting processes that may unfold rapidly and in parallel. pg. 6
Crucially, although we frame this as a ‘decision’ about perceived controllability, such computations are not necessarily deliberative and can be rapid and implicit, especially for more engrained behavioral patterns (Huys & Dayan, 2009). pg. 7
We note one important exception. Some affect-congruent responses are hardwired (e.g., freezing) or are reflexive, relying on evolutionarily preserved midbrain structures (e.g., PAG). Reflexes do not depend on evaluation of values. Thus, the action valuation and selection components of our model do not apply to reflexes.
To your broader point, it is true that some sub-components of our model may not always be relevant, even if each stage of the decision is still relevant. For example, we have hypothesized that emotions shape the simulation process during model-based planning. We predict that this will only occur in instances where the environment is complex enough to necessitate model-based control. Should the environment be simple, with model-free computations sufficing, such computations may be unnecessary. We now directly address this issue as a caveat under Limitations and Future Directions (pg. 22):
Finally, even as we have described how emotions impact distinct stages of decisionmaking, not all actions that are a response to an emotion lie within the scope of the model. For example, some affect-congruent responses are reflexive or hard-wired (e.g., freezing). As these behaviors are not learned, our hypotheses on arbitration are irrelevant. Similarly, the ways in which emotions alter simulations in model-based control is only relevant in environments that are complex enough to necessitate model-based reasoning (though it’s worth noting that model-based computations are evident even in circumstances previously assumed to not necessitate model-based reasoning; Collins & Frank, 2012).
Reviewer E
3. Section I on deciding whether to act makes this process sound deliberative. The agent has some belief over controllability and selects a type of behavior on the basis of this evaluated belief. Is there evidence that this is conscious or intentional? It is also possible that the selection of different models of behavioral control (passive versus active) could be associated with the presence of other existing comorbidities, for example, anxiety, which this section does not discuss.
Author’s response
We agree that beliefs around controllability can be conscious, slow, and deliberative (e.g., a depressed person thinking “nothing I do ever matters”), as well as fast and seemingly automatic (e.g., once you see a grizzly bear one hundred yards away, you know there’s nothing you can do to exert control over the bear). In our view, both cases still require an appraisal about the stressor’s controllability, even if that computation occurs quickly or defaults to a person’s typical perception of controllability (i.e., a prior). We now explicitly address this consideration on pg. 7 of the manuscript:
Crucially, although we frame this as a ‘decision’ about perceived controllability, such computations are not necessarily deliberative and can be rapid and implicit, especially for more engrained behavioral patterns (Huys & Dayan, 2009).
We appreciate that psychiatric comorbidities will impact people’s priors on controllability. We now directly address this issue on pg. 20 of the manuscript:
For instance, people vary in their appraisals of controllability, with low perceptions of control over threat linked to anxiety and depressive disorders (Cheng et al., 2013). How people tend to appraise controllability could impact whether they act to alter their emotion.
Reviewer E
4. Emotion regulation processes like reappraisal and suppression are not really part of the neurocomputational process proposed here but are undoubtedly important (and even present in the example story of Ada, i.e. taking a walk, working things out with Darious, etc). I think the authors should acknowledge that this is a missing piece or explain why it is beyond the purpose of the review.
Author’s response
We agree that emotion regulation processes are relevant to affect-based impulsivity and even referenced in our example. Even as these processes are highly relevant to affectbased impulsivity, we still view them as outside the explanatory scope of the review and proposed model. We articulate our reasoning on pg. 21:
Second, our proposal does not directly consider explicit emotion regulation strategies that people commonly deploy, including suppression and reappraisal (Gross, 2015). Indeed, enacting effective emotion regulation strategies may fall within the set of options a person evaluates when considering how to respond to their emotional state.
We also note that we indicate the potential use of our model for describing adaptive responses to emotions on pg. 49 of the manuscript (Table 1):
N.B. Though our focus is on impulsive behaviors that are maladaptive, we anticipate that similar mechanisms may explain adaptive responses to an emotion (e.g., grizzly bear sighting → fear → freeze).
Reviewer E
Minor points
1. I suggest expanding the references cited the section on the neurocomputational bases of controllability to include Bland et al., 2003 https://www.nature.com/articles/1300206, and to mention that the circuit extends beyond prefrontal cortex (Limbachia et al., 2021 https://www.nature.com/articles/s42003-020-01537-5?fromPaywallRec=true).
Author’s response
We appreciate these excellent reference suggestions. We recognize that controllability computations are not solely encoded in mPFC and have amended this section to highlight additional regions that are modulated by controllability. We also appreciate the suggestion to include the Bland article, which provides a mechanistic account for how controllability alters levels of DA and serotonin in mPFC. Below is a revised version of this section (pp. 13-14):
Recent findings in computational cognitive neuroscience show that humans dynamically track the predictability (Dorfman & Gershman, 2019) and controllability (Ligneul et al., 2022) of an environment, and these computations occur in brain regions such as the PFC (Ligneul et al., 2022), bed nucleus of the stria terminalis (BNST), insula, and posterior cingulate gyrus (Limbachia et al., 2021). Critically, activity in these regions governs whether a stressor leads to action or inaction (i.e. “learned helplessness”; Amat et al., 2005; Table 1.1).
Building on these insights, we propose that computations of controllability are encoded in the PFC (Ligneul et al., 2022), as well as related brain regions (Limbachia et al., 2021), and gate the extent to which DA or serotonin predominates over behavior. Historically, DA has been thought to guide approach behavior in appetitive contexts and serotonin to inhibit punished behaviors in aversive contexts (Boureau & Dayan, 2011). However, DA is involved in motivating behavior in both appetitive and aversive contexts (Boureau & Dayan, 2011; Guitart-Masip et al., 2014). In aversive contexts, by virtue of bringing the organism into a more desirable state, escape is encoded as a reward, and DA invigorates behaviors that help the organism escape (Lloyd & Dayan, 2016). Crucially, these DA-dependent behaviors only occur when the outcome is perceived as controllable (for review, see Boureau & Dayan, 2011). When behavior has no effect on the aversive environment, serotonin is released into mPFC and inhibits behavior (e.g., in learned helplessness experiments; Bland et al., 2003; Boureau & Dayan, 2011).
Reviewer E
2. What exactly is meant by impulsivity in this paper? A rash decision? A model-free decision? A risky decision? A suboptimal decision? A shift in delay discounting? I think it would be good to define impulsive behavior a bit better here.
Author’s response
It is certainly true that impulsivity refers to many behavioral tendencies, including heightened delay discounting, risky preferences, suboptimal decisions, and rash actions. We agree that is important to more clearly delineate what impulsivity means in the context of our manuscript. As such, we have added the following text to the introduction (pg. 5):
Second, impulsivity is not a unitary construct, referring to distinct behavioral tendencies ranging from heightened delay discounting to inhibitory control deficits to impulsive decision-making style (Caswell et al., 2015; Sharma et al., 2014). In our model, we focus on impulsive behaviors that are short-sighted and rash, reflecting a preference for immediate rewards, consistent with original psychological theories on trait affect-based impulsivity (Cyders & Smith, 2008; Smith et al., 2007).
Reviewer E
3. Typo on page 13: “These effects of stress are (be) mediated by changes…”
Author’s response
Thank you for bringing our attention to this issue. We have corrected the grammar of this sentence in the revised manuscript.
Henry Chase
Reviewer H
Schreiber and Hallquist review literature on computational mechanisms underlying affect based impulsivity. Briefly, the review seems to revolve around the same conundrum as that outlined by Huys and Renz, namely that pruning the model-based system provides an elegant account of the impact of emotion on decision making but that it is somewhat underdetermined, and valuation changes could have similar impact. Overall, the review is difficult to read, achieving baffling levels of self-contradiction. It is true that there are some interesting review articles written at the interface of phenomenological and behavioral (e.g. decision making paradigms) research which can reach across traditional research ‘silos’ and inspire fresh thinking. Whether or not such laudable aims drove the present work, perhaps some reconsideration of the authors’ objectives might be worthwhile.
Author’s response
You are correct that our goal in writing this article was to synthesize literature across diverse research traditions, with the goal of inspiring fresh thinking for how to study affect-based impulsivity. As you imply, cross-talk between research ‘silos’ is often limited, stymying progress in developing a mechanistic understanding for how affectbased impulsivity arises. We have made significant revisions throughout the manuscript (summarized above and below), which we hope more clearly communicates the aims of this manuscript, as well as delineating the scope of the narrative review.
Reviewer H
• Stylistically, the authors allow themselves to cite selectively - key empirical work on impulsivity and model-based decision making is not cited (e.g. Gillan/Daw elife, Patzelt/Gershman Biological Psychiatry) - and yet rule out a priori discussion of the strongest body of empirical work (response inhibition), on the grounds that it traditionally not analyzed using generative models. If generative models are so important, why not cite key work on impulsivity which has actually used them? If generative models are not important, why not cite the strongest body of empirical work? And then, what if generative models (e.g. accumulator models) were to be used to analyze response inhibition data? Are these data allowed back ‘in the fold’?
Author’s response
We appreciate that our original manuscript did not include seminal papers on model-based control, which are relevant to the paper. We have thus expanded the reference list to include Gillan et al. (2016) and Patzelt et al. (2019) on pg. 20.
We also appreciate the sizeable literature linking response inhibition deficits to trait affect-based impulsivity. We note that this literature is cited on pg. 4 of the manuscript. As you keenly point out, much of this work relies on summary statistics of task behavior, which have significant psychometric limitations – especially when the goal is to link summary indices to traits (Hedge et al. 2018). Indeed, a growing body of work shows that a domain-general bias toward inefficient evidence accumulation is associated with psychopathology (Sripada & Weigard, 2021), including impulsivity (Hall et al., 2021; Schreiber et al., 2025), and this bias can account for poor performance on inhibitory control tasks, as well as other neurocognitive tasks (Weigard et al., 2021). In light of this literature, inefficient evidence accumulation could account for many of the observed findings that link trait affect-based impulsivity with response inhibition. Given our focus on underlying computational processes that generate impulsive behavior, we do not believe citing this literature more thoroughly is appropriate. Nonetheless, we agree with your suggestion that generative models can be applied to response inhibition paradigms and may yield important insights that are relevant for affect-based impulsivity. We note that we mention this possibility as a future direction on pg. 21 of the manuscript:
Fourth, certain emotions affect the decisiveness with which a person takes action (e.g., anger promotes decisiveness; Lerner & Tiedens, 2006). Altered decisiveness may be related to lower-level cognitive processes like decision threshold in drift diffusion models (Ratcliff & McKoon, 2008).
We appreciate that we did not clearly state our concerns with the extant literature and have thus expanded on this issue in the introduction:
Studies to date have primarily relied on summary indices of emotion processing or response inhibition (e.g., rate of inhibitory failures), which only provide indirect evidence about lower-level cognitive processes (Gureckis & Love, 2015; Love, 2015) and often have poor psychometric properties (Hedge et al., 2018).
Reviewer H
• At some point the authors state that they are most interested in within-subject changes in emotion. But between subject difference are still apparent, at least implicitly, to derive a contrast. If the Pavlovian system is ‘more’ engaged, it is more engaged relative to what? A past version of the individual, or another person? Balanced discussion of within and between participant effects is generally typical throughout psychiatry, and individual differences are at the heart of psychiatry - this is a psychiatry journal.
Author’s response
We certainly agree that both within- and between-person effects are relevant to affect-based impulsivity. Our model is focused on within-person effects, with the hope that the model can be extended to capture between-person differences. Though we view this as a critical next step, there is not sufficient space in the current manuscript to consider all potential pathways that would lead someone to be prone to affect-based impulsivity. Nonetheless, we have extended our discussion of how our model might be extended to account for between-person differences under Limitations and Future Directions. Below is a revised version of this section:
We anticipate that our model could be expanded to address this open question. For instance, people vary in their appraisals of controllability, with low perceptions of control over threat linked to anxiety and depressive disorders (Cheng et al., 2013). How people tend to appraise controllability could impact whether they act to alter their emotion. Relatedly, people vary in the extent to which they tend to rely on model-based reasoning (Gillan et al., 2016; Patzelt et al., 2019), and trait affect-based impulsivity is associated with lower model-based reasoning – independent of a person’s current emotional state (Patzelt et al., 2019). This deficit in model-based reasoning could compound the effect of negative emotions on action selection, perhaps via the mechanisms we proposed above. Third, our account emphasizes that affect-related increases in GCs lead to heightened DA reactivity in the mesolimbic reward circuit (Figure 2). There are between-person differences in GR sensitivity to GCs, reflecting effects of genetics and chronic stress (Kosten et al., 2002; Ortiz et al., 1996; Rougé- Pont et al., 1993), and this sensitivity impacts the potency of GCs on DA reactivity (P. V. Piazza & Le Moal, 1996). A natural next step in developing our account would be to examine whether GR sensitivity is related to trait affect-based impulsivity.
We appreciate that we may not have made our focus on within-person effects sufficiently clear throughout the paper. We now clarify in the introduction that our focus is on how negative emotions shape computational processes, relative to that person’s baseline (pg. 5).
Reviewer H
• On page 4 the modelling approach is described as ‘new’, but it draws heavily on existing accounts, many of which have been well rehearsed in the literature. The models in question are explanatorily powerful, meaning they can be applied widely as other authors have found.
Author’s response
You are correct that decision neuroscience is by no means a ‘new’ venture, and you are right that our model draws heavily on existing work. We nonetheless believe our proposed model of affect-based impulsivity is novel, in that it synthesizes literature from disparate fields including preclinical models of addiction, ethological models of threat responses, affective neuroscience, and reinforcement learning. We have amended the text in the manuscript accordingly:
Here, we present a narrative review of affect-based impulsivity, propose an integrative decision neuroscience account of affect-based impulsivity, and outline a research agenda for testing central components of our account.
Reviewer H
• On page 5, the authors ‘appreciate’ the value of cognitive neuroscience, and thankfully proceed to cite numerous such studies. Animal and computational models have great value in building accounts of impulsivity, but ideally we would want human cognitive neuroscience also to have a central role within a translational research program of impulsivity.
Author’s response
We agree that human cognitive neuroscience has a central role in research on impulsivity, and we apologize that we seemed to have implied otherwise. As you point out, we cite human cognitive neuroscience studies throughout the manuscript. Moreover, the proposed model of affect-based impulsivity and corresponding hypotheses are designed to be tested using human neuroscience methods, as highlighted on pg. 19 and pg. 22-23 of the manuscript.
Reviewer H
• It isn’t counter-intuitive that aversive states should promote pursuit of appetitive cues - mood repair is obvious and well established within the addiction literature (e.g. by Koob and others).
Author’s response
We agree that it is well-established that negative emotions can lead to the pursuit of appetitive cues. Yet, from the perspective of action tendencies, the decision to pursue rewards when in duress is a bit surprising in that aversive states typically promote inhibition. We have thus revised this sentence to clarify the vantage point for our comment:
Given that negative emotions are often associated with inaction and withdrawal (e.g., de Berker et al., 2016), it is potentially counterintuitive that aversive internal states would promote pursuit of appetitive cues.
Reviewer H
• Page 12 - serotonin doesn’t inhibit behavior, at least not universally. SSRIs increase escape behavior on a forced swim test. A citation is needed for DA increasing approach behavior when the outcome is perceived as controllable - it is true that DA does interact with contingency but approach behavior could simply mean approaching Pavlovian cues.
Author’s response
We appreciate that our original wording oversimplified the complex interaction between serotonin and DA, as well as the role of each in behavior during aversive contexts. To provide additional nuance, we have added a footnote on pg. 13:
Historically, DA has been thought to guide approach behavior in appetitive contexts and serotonin to inhibit punished behaviors3 in aversive contexts (Boureau & Dayan, 2011).
3Though serotonin is most consistently implicated in behavioral inhibition, this effect is not universal. For example, SSRIs can increase escape behaviors during a forced swim test in rats (Detke et al., 1995).
We apologize for not including the citation for the role of DA in escape behaviors. We have amended that sentence accordingly (pg. 14):
Crucially, these DA-dependent behaviors only occur when the outcome is perceived as controllable (for review, see Boureau & Dayan, 2011).
Reviewer H
• On page 19, the authors claim that their model does not address the role of habit, yet discuss cached values and a model-free system. It is possible to differentiate model free and habitual systems, but ultimately these terms arise from different paradigms but refer to something broadly similar. Overall, the authors should choose whether they are committing to a rather specific language in which e.g. model-free learning and habit can been distinguished, or a rather loose language in which ‘Pavlovian learning’ is unitary.
Author’s response
You are correct that habit and model-free systems are tightly intertwined, with habitual action reflecting the use of cached values to make a current decision. Nonetheless, we are hesitant to claim that our model addresses the role of habit, particularly in light of the proposed neurobiological targets in our model (NAcc, mPFC). Given the anatomical dissociation in circuits that govern habitual versus goal-directed control, with the dorsolateral striatum supporting habit and dorsomedial striatum involved in goal-directed action, greater consideration of how these circuits interact with NAcc and mPFC is warranted. Moreover, the paradigms that are used to assess whether an action is under habitual or goal-directed control often differ, with devaluation procedures being central to studies of habit. As such, when considering concrete next steps for testing this account, there is a need to consider experimental methods that are wellsuited for disentangling habit and goal-directed action, as well as Pavlovian influences.
We agree that clarifying the role of habit is an important future direction, especially because of model-free learning processes may underlie habitual behavior. As such, we have amended the following sentences on pp. 21:
Third, negative affect may increase habitual behavior (Schwabe et al., 2010, 2012; Schwabe & Wolf, 2009, 2013), yet our model does not directly address the role of habit. Notably, habitual control relies on model-free learning systems (for review, see Dolan & Dayan, 2013). Thus, it may be fruitful to extend our account to consider more habitual impulsive behaviors. Such an extension would require further consideration of how repeated experiences consolidate model-free representations into inflexible stimulus-response policies, and whether this process is dependent on affect-based impulsivity.
Reviewer H
• The review doesn’t appear to distinguish between impulsive action or choice, and within choice, risk taking and delay discounting. Again, this may be as a result of a desire for internal consistency with the cost of generalizability/scope.
Author’s response
Thank you for pointing out this issue. You are correct that our manuscript, as originally written, did not delineate between different types of impulsivity, nor describe which are most relevant to our model. As such, we have added the following text to the introduction on pg. 5:
Second, impulsivity is not a unitary construct, referring to distinct behavioral tendencies that range from heightened delay discounting to inhibitory control deficits to impulsive decision-making style (Caswell et al., 2015; Sharma et al., 2014). In our model, we focus on impulsive behaviors that are short-sighted and rash, reflecting a preference for immediate rewards, consistent with original psychological theories on trait affect-based impulsivity (Cyders & Smith, 2008; Smith et al., 2007).
References:
Hall, N. T., Schreiber, A. M., Allen, T. A., & Hallquist, M. N. (2021). Disentangling cognitive processes in externalizing psychopathology using drift diffusion modeling: Antagonism, but not disinhibition, is associated with poor cognitive control. Journal of Personality, 89(5), 970-985.
Schreiber, A. M., Hall, N. T., Parr, D. F., & Hallquist, M. N. (2025). Impulsive adolescents exhibit inefficient processing and a low decision threshold when decoding facial expressions of emotions. Psychological Medicine, 55, e105.
Sripada, C., & Weigard, A. (2021). Impaired evidence accumulation as a transdiagnostic vulnerability factor in psychopathology. Frontiers in psychiatry, 12, 627179.
Weigard, A., Clark, D. A., & Sripada, C. (2021). Cognitive efficiency beats top-down control as a reliable individual difference dimension relevant to selfcontrol. Cognition, 215, 104818.