Figure 1
Schematic diagram illustrating the RL framework. At every time step, the agent receives information about the state they are in (i.e., a representation of their current environment, such as the speed and position of the car when driving) and the amount of reward they have received. Based on this feedback, the agent aims to adjust its action selection policy to maximise the amount of reward obtained in the future.
Table 1
Table summarising the evidence implicating the various proposed computations performed by dopamine and serotonin in depression and antidepressant treatment.
| PROPOSED ROLES OF NEUROTRANSMITTERS IN RL | IS THERE EVIDENCE IMPLICATING THESE PROCESSES IN ANTIDEPRESSANT TREATMENT? | ||||
|---|---|---|---|---|---|
| EVIDENCE FOR MODIFICATION BY PHARMACOLOGICAL INTERVENTION | EVIDENCE FOR ASSOCIATION WITH DEPRESSION | EVIDENCE FOR MODIFICATION BY ANTIDEPRESSANT TREATMENT IN DEPRESSED PATIENTS | EVIDENCE THAT RESPONSE TO ANTIDEPRESSANT TREATMENT IS MEDIATED BY PROCESS | ||
| Dopamine | Dopamine as RPE Signal | Yes, e.g., Pessiglione et al., (2006) | Yes for behaviour, mixed neural evidence, e.g., Gradin et al., (2011), Rutledge et al., (2017) | Mixed evidence (Admon et al., 2017; Walsh et al., 2018; Whitton et al., 2020) | – |
| Dopamine and Effort-Based Computations | Yes, e.g., Beierholm et al. (2013) | Yes, e.g., Treadway et al. (2012) | – | – | |
| Dopamine and Model-Based RL | Yes, e.g., Wunderlich et al. (2012) | Yes, but only under stress induction (Heller et al., 2018) | – | – | |
| Serotonin | Serotonin and Aversive Processing | Yes, in punishment learning (Cools et al., 2008) and inhibiting vigour (Crockett et al., 2009) | Yes, e.g., Dombrovski et al. (2013) | – | – |
| Serotonin and Reward Processing | Yes, in reward learning (e.g., Scholl et al., 2017) and ‘momentum’ (e.g., Michely, Eldar, Martin et al., 2020) | Yes (see dopamine and RPEs) | – | – | |
| Serotonin and Temporal Discounting | Yes, e.g., Schweighofer et al. (2008) | Mixed evidence, e.g., Pulcu et al. (2014), Lempert & Pizzagali (2010) | – | – | |