Have a personal or library account? Click to login
What Can Reinforcement Learning Models of Dopamine and Serotonin Tell Us about the Action of Antidepressants? Cover

What Can Reinforcement Learning Models of Dopamine and Serotonin Tell Us about the Action of Antidepressants?

Computational Psychiatry
Volume 6 (2022): Issue 1
By: Denis C. L. Lan and  Michael Browning  
Open Access
|Jul 2022

References

  1. 1Aberman, J. E., & Salamone, J. D. (1999). Nucleus accumbens dopamine depletions make rats more sensitive to high ratio requirements but do not impair primary food reinforcement. Neuroscience, 92(2), 545–552. DOI: 10.1016/s0306-4522(99)00004-4
    Back to article
  2. 2Admon, R., Kaiser, R. H., Dillon, D. G., Beltzer, M., Goer, F., Olson, D. P., Vitaliano, G., & Pizzagalli, D. A. (2017). Dopaminergic Enhancement of Striatal Response to Reward in Major Depression. The American Journal of Psychiatry, 174(4), 378–386. DOI: 10.1176/appi.ajp.2016.16010111
    Back to article
  3. 3Amlung, M., Marsden, E., Holshausen, K., Morris, V., Patel, H., Vedelago, L., Naish, K. R., Reed, D. D., & McCabe, R. E. (2019). Delay Discounting as a Transdiagnostic Process in Psychiatric Disorders: A Meta-analysis. JAMA Psychiatry, 76(11), 1176–1186. DOI: 10.1001/jamapsychiatry.2019.2102
    Back to article
  4. 4Andrews, P. W., Bharwani, A., Lee, K. R., Fox, M., & Thomson, J. A. (2015). Is serotonin an upper or a downer? The evolution of the serotonergic system and its role in depression and the antidepressant response. Neuroscience and Biobehavioral Reviews, 51, 164–188. DOI: 10.1016/j.neubiorev.2015.01.018
    Back to article
  5. 5Artigas, F., Romero, L., de Montigny, C., & Blier, P. (1996). Acceleration of the effect of selected antidepressant drugs in major depression by 5-HT1A antagonists. Trends in Neurosciences, 19(9), 378–383. DOI: 10.1016/S0166-2236(96)10037-0
    Back to article
  6. 6Aston-Jones, G., Rajkowski, J., & Cohen, J. (1999). Role of locus coeruleus in attention and behavioral flexibility. Biological Psychiatry, 46(9), 1309–1320. DOI: 10.1016/s0006-3223(99)00140-7
    Back to article
  7. 7Balleine, B. W., & O’Doherty, J. P. (2010). Human and Rodent Homologies in Action Control: Corticostriatal Determinants of Goal-Directed and Habitual Action. Neuropsychopharmacology, 35(1), 48–69. DOI: 10.1038/npp.2009.131
    Back to article
  8. 8Bayer, H. M., & Glimcher, P. W. (2005). Midbrain Dopamine Neurons Encode a Quantitative Reward Prediction Error Signal. Neuron, 47(1), 129–141. DOI: 10.1016/j.neuron.2005.05.020
    Back to article
  9. 9Beierholm, U., Guitart-Masip, M., Economides, M., Chowdhury, R., Düzel, E., Dolan, R., & Dayan, P. (2013). Dopamine Modulates Reward-Related Vigor. Neuropsychopharmacology, 38(8), 1495–1503. DOI: 10.1038/npp.2013.48
    Back to article
  10. 10Berwian, I. M., Wenzel, J. G., Collins, A. G. E., Seifritz, E., Stephan, K. E., Walter, H., & Huys, Q. J. M. (2020). Computational Mechanisms of Effort and Reward Decisions in Patients With Depression and Their Association With Relapse After Antidepressant Discontinuation. JAMA Psychiatry, 77(5), 1–10. DOI: 10.1001/jamapsychiatry.2019.4971
    Back to article
  11. 11Bolger, N., Zee, K., Rossignac-Milon, M., & Hassin, R. (2019). Causal processes in psychology are heterogeneous. Journal of Experimental Psychology: General, 148, 601–618. DOI: 10.1037/xge0000558
    Back to article
  12. 12Bromberg-Martin, E. S., Hikosaka, O., & Nakamura, K. (2010). Coding of Task Reward Value in the Dorsal Raphe Nucleus. Journal of Neuroscience, 30(18), 6262–6272. DOI: 10.1523/JNEUROSCI.0015-10.2010
    Back to article
  13. 13Brown, V. M., Zhu, L., Solway, A., Wang, J. M., McCurry, K. L., King-Casas, B., & Chiu, P. H. (2021). Reinforcement Learning Disruptions in Individuals With Depression and Sensitivity to Symptom Change Following Cognitive Behavioral Therapy. JAMA Psychiatry, 78(10), 1113–1122. DOI: 10.1001/jamapsychiatry.2021.1844
    Back to article
  14. 14Carr, G. D., & White, N. M. (1987). Effects of systemic and intracranial amphetamine injections on behavior in the open field: A detailed analysis. Pharmacology, Biochemistry, and Behavior, 27(1), 113–122. DOI: 10.1016/0091-3057(87)90485-0
    Back to article
  15. 15Castrén, E. (2005). Is mood chemistry? Nature Reviews. Neuroscience, 6(3), 241–246. DOI: 10.1038/nrn1629
    Back to article
  16. 16Chang, C. Y., Gardner, M., Di Tillio, M. G., & Schoenbaum, G. (2017). Optogenetic Blockade of Dopamine Transients Prevents Learning Induced by Changes in Reward Features. Current Biology: CB, 27(22), 3480–3486.e3. DOI: 10.1016/j.cub.2017.09.049
    Back to article
  17. 17Cipriani, A., Furukawa, T. A., Salanti, G., Chaimani, A., Atkinson, L. Z., Ogawa, Y., Leucht, S., Ruhe, H. G., Turner, E. H., Higgins, J. P. T., Egger, M., Takeshima, N., Hayasaka, Y., Imai, H., Shinohara, K., Tajika, A., Ioannidis, J. P. A., & Geddes, J. R. (2018). Comparative efficacy and acceptability of 21 antidepressant drugs for the acute treatment of adults with major depressive disorder: A systematic review and network meta-analysis. The Lancet, 391(10128), 1357–1366. DOI: 10.1016/S0140-6736(17)32802-7
    Back to article
  18. 18Cools, R., Nakamura, K., & Daw, N. D. (2011). Serotonin and Dopamine: Unifying Affective, Activational, and Decision Functions. Neuropsychopharmacology, 36(1), 98–113. DOI: 10.1038/npp.2010.121
    Back to article
  19. 19Cools, R., Robinson, O. J., & Sahakian, B. (2008). Acute Tryptophan Depletion in Healthy Volunteers Enhances Punishment Prediction but Does not Affect Reward Prediction. Neuropsychopharmacology, 33(9), 2291–2299. DOI: 10.1038/sj.npp.1301598
    Back to article
  20. 20Cowen, P. J., & Browning, M. (2015). What has serotonin to do with depression? World Psychiatry, 14(2), 158–160. DOI: 10.1002/wps.20229
    Back to article
  21. 21Crockett, M. J., Clark, L., & Robbins, T. W. (2009). Reconciling the Role of Serotonin in Behavioral Inhibition and Aversion: Acute Tryptophan Depletion Abolishes Punishment-Induced Inhibition in Humans. The Journal of Neuroscience, 29(38), 11993–11999. DOI: 10.1523/JNEUROSCI.2513-09.2009
    Back to article
  22. 22Daw, N. D., Gershman, S. J., Seymour, B., Dayan, P., & Dolan, R. J. (2011). Model-based influences on humans’ choices and striatal prediction errors. Neuron, 69(6), 1204–1215. DOI: 10.1016/j.neuron.2011.02.027
    Back to article
  23. 23Daw, N. D., Kakade, S., & Dayan, P. (2002). Opponent interactions between serotonin and dopamine. Neural Networks, 15(4), 603–616. DOI: 10.1016/S0893-6080(02)00052-7
    Back to article
  24. 24de Jong, J. W., Afjei, S. A., Dorocic, I. P., Peck, J. R., Liu, C., Kim, C. K., Tian, L., Deisseroth, K., & Lammel, S. (2019). A neural circuit mechanism for encoding aversive stimuli in the mesolimbic dopamine system. Neuron, 101(1), 133–151.e7. DOI: 10.1016/j.neuron.2018.11.005
    Back to article
  25. 25Deakin, J. F. W., & Graeff, F. G. (1991). 5-HT and mechanisms of defence. Journal of Psychopharmacology, 5(4), 305–315. DOI: 10.1177/026988119100500414
    Back to article
  26. 26Delgado, P. L. (2000). Depression: The case for a monoamine deficiency. The Journal of Clinical Psychiatry, 61 Suppl 6, 7–11.
    Back to article
  27. 27Dell’Osso, B., Palazzo, M. C., Oldani, L., & Altamura, A. C. (2010). The Noradrenergic Action in Antidepressant Treatments: Pharmacological and Clinical Aspects. CNS Neuroscience & Therapeutics, 17(6), 723–732. DOI: 10.1111/j.1755-5949.2010.00217.x
    Back to article
  28. 28Deserno, L., Huys, Q. J. M., Boehme, R., Buchert, R., Heinze, H.-J., Grace, A. A., Dolan, R. J., Heinz, A., & Schlagenhauf, F. (2015). Ventral striatal dopamine reflects behavioral and neural signatures of model-based control during sequential decision making. Proceedings of the National Academy of Sciences, 112(5), 1595–1600. DOI: 10.1073/pnas.1417219112
    Back to article
  29. 29Dombrovski, A. Y., Szanto, K., Clark, L., Aizenstein, H. J., Chase, H. W., Reynolds, C. F., & Siegle, G. J. (2015). Corticostriatothalamic reward prediction error signals and executive control in late-life depression. Psychological Medicine, 45(7), 1413–1424. DOI: 10.1017/S0033291714002517
    Back to article
  30. 30Dombrovski, A. Y., Szanto, K., Clark, L., Reynolds, C. F., & Siegle, G. J. (2013). Reward Signals, Attempted Suicide, and Impulsivity in Late-Life Depression. JAMA Psychiatry, 70(10), 1020. DOI: 10.1001/jamapsychiatry.2013.75
    Back to article
  31. 31Eckstein, M. K., Master, S. L., Xia, L., Dahl, R. E., Wilbrecht, L., & Collins, A. G. E. (2021). Learning Rates Are Not All the Same: The Interpretation of Computational Model Parameters Depends on the Context (p. 2021.05.28.446162). DOI: 10.1101/2021.05.28.446162
    Back to article
  32. 32Eckstein, M. K., Wilbrecht, L., & Collins, A. G. (2021). What do reinforcement learning models measure? Interpreting model parameters in cognition and neuroscience. Current Opinion in Behavioral Sciences, 41, 128–137. DOI: 10.1016/j.cobeha.2021.06.004
    Back to article
  33. 33Eldar, E., & Niv, Y. (2015). Interaction between emotional state and learning underlies mood instability. Nature Communications, 6(1), 6149. DOI: 10.1038/ncomms7149
    Back to article
  34. 34Eldar, E., Roth, C., Dayan, P., & Dolan, R. J. (2018). Decodability of Reward Learning Signals Predicts Mood Fluctuations. Current Biology, 28(9), 1433–1439.e7. DOI: 10.1016/j.cub.2018.03.038
    Back to article
  35. 35Eldar, E., Rutledge, R. B., Dolan, R. J., & Niv, Y. (2016). Mood as Representation of Momentum. Trends in Cognitive Sciences, 20(1), 15–24. DOI: 10.1016/j.tics.2015.07.010
    Back to article
  36. 36Fletcher, P. J., & Korth, K. M. (1999). Activation of 5-HT1B receptors in the nucleus accumbens reduces amphetamine-induced enhancement of responding for conditioned reward. Psychopharmacology, 142(2), 165–174. DOI: 10.1007/s002130050876
    Back to article
  37. 37Fletcher, P. J., Ming, Z.-H., & Higgins, G. A. (1993). Conditioned place preference induced by microinjection of 8-OH-DPAT into the dorsal or median raphe nucleus. Psychopharmacology, 113(1), 31–36. DOI: 10.1007/BF02244330
    Back to article
  38. 38Fletcher, P. J., Tampakeras, M., & Yeomans, J. S. (1995). Median raphe injections of 8-OH-DPAT lower frequency thresholds for lateral hypothalamic self-stimulation. Pharmacology Biochemistry and Behavior, 52(1), 65–71. DOI: 10.1016/0091-3057(94)00441-K
    Back to article
  39. 39Frey, A.-L., & McCabe, C. (2020). Effects of serotonin and dopamine depletion on neural prediction computations during social learning. Neuropsychopharmacology, 45(9), 1431–1437. DOI: 10.1038/s41386-020-0678-z
    Back to article
  40. 40Gillan, C. M., Kosinski, M., Whelan, R., Phelps, E. A., & Daw, N. D. (2016). Characterizing a psychiatric symptom dimension related to deficits in goal-directed control. ELife, 5, e11305. DOI: 10.7554/eLife.11305
    Back to article
  41. 41Grace, A. A. (1991). Phasic versus tonic dopamine release and the modulation of dopamine system responsivity: A hypothesis for the etiology of schizophrenia. Neuroscience, 41(1), 1–24. DOI: 10.1016/0306-4522(91)90196-u
    Back to article
  42. 42Grace, A. A. (2000). The tonic/phasic model of dopamine system regulation and its implications for understanding alcohol and psychostimulant craving. Addiction (Abingdon, England), 95 Suppl 2, S119–128. DOI: 10.1080/09652140050111690
    Back to article
  43. 43Gradin, V. B., Kumar, P., Waiter, G., Ahearn, T., Stickle, C., Milders, M., Reid, I., Hall, J., & Steele, J. D. (2011). Expected value and prediction error abnormalities in depression and schizophrenia. Brain: A Journal of Neurology, 134(Pt 6), 1751–1764. DOI: 10.1093/brain/awr059
    Back to article
  44. 44Greenberg, T., Chase, H. W., Almeida, J. R., Stiffler, R., Zevallos, C. R., Aslam, H. A., Deckersbach, T., Weyandt, S., Cooper, C., Toups, M., Carmody, T., Kurian, B., Peltier, S., Adams, P., McInnis, M. G., Oquendo, M. A., McGrath, P. J., Fava, M., Weissman, M., … Phillips, M. L. (2015). Moderation of the Relationship Between Reward Expectancy and Prediction Error-Related Ventral Striatal Reactivity by Anhedonia in Unmedicated Major Depressive Disorder: Findings From the EMBARC Study. The American Journal of Psychiatry, 172(9), 881–891. DOI: 10.1176/appi.ajp.2015.14050594
    Back to article
  45. 45Haber, S. N. (2014). The place of dopamine in the cortico-basal ganglia circuit. Neuroscience, 282, 248–257. DOI: 10.1016/j.neuroscience.2014.10.008
    Back to article
  46. 46Halahakoon, D. C., Kieslich, K., O’Driscoll, C., Nair, A., Lewis, G., & Roiser, J. P. (2020). Reward-Processing Behavior in Depressed Participants Relative to Healthy Volunteers. JAMA Psychiatry, 77(12), 1286–1295. DOI: 10.1001/jamapsychiatry.2020.2139
    Back to article
  47. 47Harmer, C. J., Goodwin, G. M., & Cowen, P. J. (2009). Why do antidepressants take so long to work? A cognitive neuropsychological model of antidepressant drug action. The British Journal of Psychiatry, 195(2), 102–108. DOI: 10.1192/bjp.bp.108.051193
    Back to article
  48. 48Harmer, C. J., O’Sullivan, U., Favaron, E., Massey-Chase, R., Ayres, R., Reinecke, A., Goodwin, G. M., & Cowen, P. J. (2009). Effect of acute antidepressant administration on negative affective bias in depressed patients. The American Journal of Psychiatry, 166(10), 1178–1184. DOI: 10.1176/appi.ajp.2009.09020149
    Back to article
  49. 49Harrison, A. A., Everitt, B. J., & Robbins, T. W. (1997). Central 5-HT depletion enhances impulsive responding without affecting the accuracy of attentional performance: Interactions with dopaminergic mechanisms. Psychopharmacology, 133(4), 329–342. DOI: 10.1007/s002130050410
    Back to article
  50. 50Harrison, A. A., Everitt, B. J., & Robbins, T. W. (1999). Central serotonin depletion impairs both the acquisition and performance of a symmetrically reinforced go/no-go conditional visual discrimination. Behavioural Brain Research, 100(1–2), 99–112. DOI: 10.1016/s0166-4328(98)00117-x
    Back to article
  51. 51Hauser, T. U., Eldar, E., & Dolan, R. J. (2017). Separate mesocortical and mesolimbic pathways encode effort and reward learning signals. Proceedings of the National Academy of Sciences, 114(35), E7395–E7404. DOI: 10.1073/pnas.1705643114
    Back to article
  52. 52Heller, A. S., Ezie, C. E. C., Otto, A. R., & Timpano, K. R. (2018). Model-based learning and individual differences in depression: The moderating role of stress. Behaviour Research and Therapy, 111, 19–26. DOI: 10.1016/j.brat.2018.09.007
    Back to article
  53. 53Hollerman, J. R., & Schultz, W. (1998). Dopamine neurons report an error in the temporal prediction of reward during learning. Nature Neuroscience, 1(4), 304–309. DOI: 10.1038/1124
    Back to article
  54. 54Huang, Y., Yaple, Z. A., & Yu, R. (2020). Goal-oriented and habitual decisions: Neural signatures of model-based and model-free learning. NeuroImage, 215, 116834. DOI: 10.1016/j.neuroimage.2020.116834
    Back to article
  55. 55Huys, Q. J. M., Daw, N. D., & Dayan, P. (2015). Depression: A Decision-Theoretic Analysis. Annual Review of Neuroscience, 38(1), 1–23. DOI: 10.1146/annurev-neuro-071714-033928
    Back to article
  56. 56Huys, Q. J. M., Guitart-Masip, M., Dolan, R. J., & Dayan, P. (2015). Decision-Theoretic Psychiatry. Clinical Psychological Science, 3(3), 400–421. DOI: 10.1177/2167702614562040
    Back to article
  57. 57Huys, Q. J. M., Pizzagalli, D. A., Bogdan, R., & Dayan, P. (2013). Mapping anhedonia onto reinforcement learning: A behavioural meta-analysis. Biology of Mood & Anxiety Disorders, 3(1), 12. DOI: 10.1186/2045-5380-3-12
    Back to article
  58. 58Jackson, D. M., Andén, N. E., & Dahlström, A. (1975). A functional effect of dopamine in the nucleus accumbens and in some other dopamine-rich parts of the rat brain. Psychopharmacologia, 45(2), 139–149. DOI: 10.1007/BF00429052
    Back to article
  59. 59Kanen, J. W., Apergis-Schoute, A. M., Yellowlees, R., Arntz, F. E., van der Flier, F. E., Price, A., Cardinal, R. N., Christmas, D. M., Clark, L., Sahakian, B. J., Crockett, M. J., & Robbins, T. W. (2021). Serotonin depletion impairs both Pavlovian and instrumental reversal learning in healthy humans. Molecular Psychiatry, 1–11. DOI: 10.1038/s41380-021-01240-9
    Back to article
  60. 60Kapur, S., & Remington, G. (1996). Serotonin-dopamine interaction and its relevance to schizophrenia. The American Journal of Psychiatry, 153(4), 466–476. DOI: 10.1176/ajp.153.4.466
    Back to article
  61. 61Kool, W., Gershman, S. J., & Cushman, F. A. (2017). Cost-Benefit Arbitration Between Multiple Reinforcement-Learning Systems. Psychological Science, 28(9), 1321–1333. DOI: 10.1177/0956797617708288
    Back to article
  62. 62Kranz, G. S., Kasper, S., & Lanzenberger, R. (2010). Reward and the serotonergic system. Neuroscience, 166(4), 1023–1035. DOI: 10.1016/j.neuroscience.2010.01.036
    Back to article
  63. 63Krystal, A. D., Pizzagalli, D. A., Smoski, M., Mathew, S. J., Nurnberger, J., Lisanby, S. H., Iosifescu, D., Murrough, J. W., Yang, H., Weiner, R. D., Calabrese, J. R., Sanacora, G., Hermes, G., Keefe, R. S. E., Song, A., Goodman, W., Szabo, S. T., Whitton, A. E., Gao, K., & Potter, W. Z. (2020). A randomized proof-of-mechanism trial applying the ‘fast-fail’ approach to evaluating κ-opioid antagonism as a treatment for anhedonia. Nature Medicine, 26(5), 760–768. DOI: 10.1038/s41591-020-0806-7
    Back to article
  64. 64Krystal, J. H., Sanacora, G., Blumberg, H., Anand, A., Charney, D. S., Marek, G., Epperson, C. N., Goddard, A., & Mason, G. F. (2002). Glutamate and GABA systems as targets for novel antidepressant and mood-stabilizing treatments. Molecular Psychiatry, 7(1), S71–S80. DOI: 10.1038/sj.mp.4001021
    Back to article
  65. 65Krystal, J. H., Sanacora, G., & Duman, R. S. (2013). Rapid-acting glutamatergic antidepressants: The path to ketamine and beyond. Biological Psychiatry, 73(12), 1133–1141. DOI: 10.1016/j.biopsych.2013.03.026
    Back to article
  66. 66Kumar, P., Goer, F., Murray, L., Dillon, D. G., Beltzer, M. L., Cohen, A. L., Brooks, N. H., & Pizzagalli, D. A. (2018). Impaired reward prediction error encoding and striatal-midbrain connectivity in depression. Neuropsychopharmacology, 43(7), 1581–1588. DOI: 10.1038/s41386-018-0032-x
    Back to article
  67. 67Langdon, A. J., Sharpe, M. J., Schoenbaum, G., & Niv, Y. (2018). Model-based predictions for dopamine. Current Opinion in Neurobiology, 49, 1–7. DOI: 10.1016/j.conb.2017.10.006
    Back to article
  68. 68Lee, S. W., Shimojo, S., & O’Doherty, J. P. (2014). Neural Computations Underlying Arbitration between Model-Based and Model-free Learning. Neuron, 81(3), 687–699. DOI: 10.1016/j.neuron.2013.11.028
    Back to article
  69. 69Lempert, K. M., & Pizzagalli, D. A. (2010). Delay Discounting and Future-directed Thinking in Anhedonic Individuals. Journal of Behavior Therapy and Experimental Psychiatry, 41(3), 258–264. DOI: 10.1016/j.jbtep.2010.02.003
    Back to article
  70. 70Li, Y., Zhong, W., Wang, D., Feng, Q., Liu, Z., Zhou, J., Jia, C., Hu, F., Zeng, J., Guo, Q., Fu, L., & Luo, M. (2016). Serotonin neurons in the dorsal raphe nucleus encode reward signals. Nature Communications, 7(1), 10503. DOI: 10.1038/ncomms10503
    Back to article
  71. 71Lukinova, E., Wang, Y., Lehrer, S. F., & Erlich, J. C. (2019). Time preferences are reliable across time-horizons and verbal versus experiential tasks. ELife, 8, e39656. DOI: 10.7554/eLife.39656
    Back to article
  72. 72Matsumoto, M., & Hikosaka, O. (2009). Two types of dopamine neuron distinctly convey positive and negative motivational signals. Nature, 459(7248), 837–841. DOI: 10.1038/nature08028
    Back to article
  73. 73Maya Vetencourt, J. F., Sale, A., Viegi, A., Baroncelli, L., De Pasquale, R., O’Leary, O. F., Castrén, E., & Maffei, L. (2008). The antidepressant fluoxetine restores plasticity in the adult visual cortex. Science (New York, N.Y.), 320(5874), 385–388. DOI: 10.1126/science.1150516
    Back to article
  74. 74Meyniel, F., Goodwin, G. M., Deakin, J. W., Klinge, C., MacFadyen, C., Milligan, H., Mullings, E., Pessiglione, M., & Gaillard, R. (2016). A specific role for serotonin in overcoming effort cost. ELife, 5, e17282. DOI: 10.7554/eLife.17282
    Back to article
  75. 75Michely, J., Eldar, E., Erdman, A., Martin, I. M., & Dolan, R. J. (2020). SSRIs modulate asymmetric learning from reward and punishment. BioRxiv, 2020.05.21.108266. DOI: 10.1101/2020.05.21.108266
    Back to article
  76. 76Michely, J., Eldar, E., Martin, I. M., & Dolan, R. J. (2020). A mechanistic account of serotonin’s impact on mood. Nature Communications, 11(1), 2335. DOI: 10.1038/s41467-020-16090-2
    Back to article
  77. 77Miyazaki, K., Miyazaki, K. W., & Doya, K. (2011). Activation of Dorsal Raphe Serotonin Neurons Underlies Waiting for Delayed Rewards. Journal of Neuroscience, 31(2), 469–479. DOI: 10.1523/JNEUROSCI.3714-10.2011
    Back to article
  78. 78Niv, Y., Daw, N. D., Joel, D., & Dayan, P. (2007). Tonic dopamine: Opportunity costs and the control of response vigor. Psychopharmacology, 191(3), 507–520. DOI: 10.1007/s00213-006-0502-4
    Back to article
  79. 79Otto, A. R., Gershman, S. J., Markman, A. B., & Daw, N. D. (2013). The curse of planning: Dissecting multiple reinforcement-learning systems by taxing the central executive. Psychological Science, 24(5), 751–761. DOI: 10.1177/0956797612463080
    Back to article
  80. 80Palminteri, S., Wyart, V., & Koechlin, E. (2017). The Importance of Falsification in Computational Cognitive Modeling. Trends in Cognitive Sciences, 21(6), 425–433. DOI: 10.1016/j.tics.2017.03.011
    Back to article
  81. 81Paul, E. D., & Lowry, C. A. (2013). Functional topography of serotonergic systems supports the Deakin/Graeff hypothesis of anxiety and affective disorders. Journal of Psychopharmacology (Oxford, England), 27(12), 1090–1106. DOI: 10.1177/0269881113490328
    Back to article
  82. 82Pessiglione, M., Seymour, B., Flandin, G., Dolan, R. J., & Frith, C. D. (2006). Dopamine-dependent prediction errors underpin reward-seeking behaviour in humans. Nature, 442(7106), 1042–1045. DOI: 10.1038/nature05051
    Back to article
  83. 83Pizzagalli, D. A., Smoski, M., Ang, Y.-S., Whitton, A. E., Sanacora, G., Mathew, S. J., Nurnberger, J., Lisanby, S. H., Iosifescu, D. V., Murrough, J. W., Yang, H., Weiner, R. D., Calabrese, J. R., Goodman, W., Potter, W. Z., & Krystal, A. D. (2020). Selective kappa-opioid antagonism ameliorates anhedonic behavior: Evidence from the Fast-fail Trial in Mood and Anxiety Spectrum Disorders (FAST-MAS). Neuropsychopharmacology, 45(10), 1656–1663. DOI: 10.1038/s41386-020-0738-4
    Back to article
  84. 84Pulcu, E., Trotter, P. D., Thomas, E. J., McFarquhar, M., Juhasz, G., Sahakian, B. J., Deakin, J. F. W., Zahn, R., Anderson, I. M., & Elliott, R. (2014). Temporal discounting in major depressive disorder. Psychological Medicine, 44(9), 1825–1834. DOI: 10.1017/S0033291713002584
    Back to article
  85. 85Rutledge, R. B., Moutoussis, M., Smittenaar, P., Zeidman, P., Taylor, T., Hrynkiewicz, L., Lam, J., Skandali, N., Siegel, J. Z., Ousdal, O. T., Prabhu, G., Dayan, P., Fonagy, P., & Dolan, R. J. (2017). Association of Neural and Emotional Impacts of Reward Prediction Errors With Major Depression. JAMA Psychiatry, 74(8), 790–797. DOI: 10.1001/jamapsychiatry.2017.1713
    Back to article
  86. 86Rutledge, R. B., Skandali, N., Dayan, P., & Dolan, R. J. (2014). A computational and neural model of momentary subjective well-being. Proceedings of the National Academy of Sciences, 111(33), 12252–12257. DOI: 10.1073/pnas.1407535111
    Back to article
  87. 87Salamone, J. D., & Correa, M. (2002). Motivational views of reinforcement: Implications for understanding the behavioral functions of nucleus accumbens dopamine. Behavioural Brain Research, 137(1–2), 3–25. DOI: 10.1016/s0166-4328(02)00282-6
    Back to article
  88. 88Salinas-Hernández, X. I., Vogel, P., Betz, S., Kalisch, R., Sigurdsson, T., & Duvarci, S. (2018). Dopamine neurons drive fear extinction learning by signaling the omission of expected aversive outcomes. ELife, 7, e38818. DOI: 10.7554/eLife.38818
    Back to article
  89. 89Saunders, B. T., Richard, J. M., Margolis, E. B., & Janak, P. H. (2018). Dopamine neurons create Pavlovian conditioned stimuli with circuit-defined motivational properties. Nature Neuroscience, 21(8), 1072–1083. DOI: 10.1038/s41593-018-0191-4
    Back to article
  90. 90Schneier, F. R., Slifstein, M., Whitton, A. E., Pizzagalli, D. A., Reinen, J., McGrath, P. J., Iosifescu, D. V., & Abi-Dargham, A. (2018). Dopamine Release in Antidepressant-Naive Major Depressive Disorder: A Multimodal [11C]-(+)-PHNO Positron Emission Tomography and Functional Magnetic Resonance Imaging Study. Biological Psychiatry, 84(8), 563–573. DOI: 10.1016/j.biopsych.2018.05.014
    Back to article
  91. 91Scholl, J., Kolling, N., Nelissen, N., Browning, M., Rushworth, M. F. S., & Harmer, C. J. (2017). Beyond negative valence: 2-week administration of a serotonergic antidepressant enhances both reward and effort learning signals. PLOS Biology, 15(2), e2000756. DOI: 10.1371/journal.pbio.2000756
    Back to article
  92. 92Schultz, W. (1998). Predictive Reward Signal of Dopamine Neurons. Journal of Neurophysiology, 80(1), 1–27. DOI: 10.1152/jn.1998.80.1.1
    Back to article
  93. 93Schultz, W., Dayan, P., & Montague, P. R. (1997). A Neural Substrate of Prediction and Reward. Science, 275(5306), 1593–1599. DOI: 10.1126/science.275.5306.1593
    Back to article
  94. 94Schweighofer, N., Bertin, M., Shishida, K., Okamoto, Y., Tanaka, S. C., Yamawaki, S., & Doya, K. (2008). Low-serotonin levels increase delayed reward discounting in humans. The Journal of Neuroscience: The Official Journal of the Society for Neuroscience, 28(17), 4528–4532. DOI: 10.1523/JNEUROSCI.4982-07.2008
    Back to article
  95. 95Schweighofer, N., Tanaka, S. C., & Doya, K. (2007). Serotonin and the Evaluation of Future Rewards. Annals of the New York Academy of Sciences, 1104(1), 289–300. DOI: 10.1196/annals.1390.011
    Back to article
  96. 96Seymour, B., Daw, N. D., Roiser, J. P., Dayan, P., & Dolan, R. (2012). Serotonin Selectively Modulates Reward Value in Human Decision-Making. Journal of Neuroscience, 32(17), 5833–5842. DOI: 10.1523/JNEUROSCI.0053-12.2012
    Back to article
  97. 97Sharp, M. E., Foerde, K., Daw, N. D., & Shohamy, D. (2016). Dopamine selectively remediates ‘model-based’ reward learning: A computational approach. Brain, 139(2), 355–364. DOI: 10.1093/brain/awv347
    Back to article
  98. 98Sharpe, M. J., Chang, C. Y., Liu, M. A., Batchelor, H. M., Mueller, L. E., Jones, J. L., Niv, Y., & Schoenbaum, G. (2017). Dopamine transients are sufficient and necessary for acquisition of model-based associations. Nature Neuroscience, 20(5), 735–742. DOI: 10.1038/nn.4538
    Back to article
  99. 99Smittenaar, P., FitzGerald, T. H. B., Romei, V., Wright, N. D., & Dolan, R. J. (2013). Disruption of Dorsolateral Prefrontal Cortex Decreases Model-Based in Favor of Model-free Control in Humans. Neuron, 80(4), 914–919. DOI: 10.1016/j.neuron.2013.08.009
    Back to article
  100. 100Sokolowski, J. D., & Salamone, J. D. (1998). The role of accumbens dopamine in lever pressing and response allocation: Effects of 6-OHDA injected into core and dorsomedial shell. Pharmacology, Biochemistry, and Behavior, 59(3), 557–566. DOI: 10.1016/s0091-3057(97)00544-3
    Back to article
  101. 101Soubrié, P. (1986). Reconciling the role of central serotonin neurons in human and animal behavior. Behavioral and Brain Sciences, 9(2), 319–335. DOI: 10.1017/S0140525X00022871
    Back to article
  102. 102Stauffer, W. R., Lak, A., Yang, A., Borel, M., Paulsen, O., Boyden, E. S., & Schultz, W. (2016). Dopamine Neuron-Specific Optogenetic Stimulation in Rhesus Macaques. Cell, 166(6), 1564–1571.e6. DOI: 10.1016/j.cell.2016.08.024
    Back to article
  103. 103Takahashi, Y. K., Batchelor, H. M., Liu, B., Khanna, A., Morales, M., & Schoenbaum, G. (2017). Dopamine Neurons Respond to Errors in the Prediction of Sensory Features of Expected Rewards. Neuron, 95(6), 1395–1405.e3. Scopus. DOI: 10.1016/j.neuron.2017.08.025
    Back to article
  104. 104Treadway, M. T., Bossaller, N., Shelton, R. C., & Zald, D. H. (2012). Effort-Based Decision-Making in Major Depressive Disorder: A Translational Model of Motivational Anhedonia. Journal of Abnormal Psychology, 121(3), 553–558. DOI: 10.1037/a0028813
    Back to article
  105. 105Treadway, M. T., & Zald, D. H. (2011). Reconsidering Anhedonia in Depression: Lessons from Translational Neuroscience. Neuroscience and Biobehavioral Reviews, 35(3), 537–555. DOI: 10.1016/j.neubiorev.2010.06.006
    Back to article
  106. 106Tsypes, A., Szanto, K., Bridge, J. A., Brown, V. M., Keilp, J. G., & Dombrovski, A. Y. (2022). Delay discounting in suicidal behavior: Myopic preference or inconsistent valuation? Journal of Psychopathology and Clinical Science, 131(1), 34–44. DOI: 10.1037/abn0000717
    Back to article
  107. 107Ubl, B., Kuehner, C., Kirsch, P., Ruttorf, M., Diener, C., & Flor, H. (2015). Altered neural reward and loss processing and prediction error signalling in depression. Social Cognitive and Affective Neuroscience, 10(8), 1102–1112. DOI: 10.1093/scan/nsu158
    Back to article
  108. 108Viglione, A., Chiarotti, F., Poggini, S., Giuliani, A., & Branchi, I. (2019). Predicting antidepressant treatment outcome based on socioeconomic status and citalopram dose. The Pharmacogenomics Journal, 19(6), 538–546. DOI: 10.1038/s41397-019-0080-6
    Back to article
  109. 109Walsh, A. E. L., Browning, M., Drevets, W. C., Furey, M., & Harmer, C. J. (2018). Dissociable temporal effects of bupropion on behavioural measures of emotional and reward processing in depression. Philosophical Transactions of the Royal Society of London. Series B, Biological Sciences, 373(1742). DOI: 10.1098/rstb.2017.0030
    Back to article
  110. 110Weissengruber, S., Lee, S. W., O’Doherty, J. P., & Ruff, C. C. (2019). Neurostimulation Reveals Context-Dependent Arbitration Between Model-Based and Model-Free Reinforcement Learning. Cerebral Cortex, 29(11), 4850–4862. DOI: 10.1093/cercor/bhz019
    Back to article
  111. 111Westbrook, A., Bosch, R. van den, Määttä, J. I., Hofmans, L., Papadopetraki, D., Cools, R., & Frank, M. J. (2020). Dopamine promotes cognitive effort by biasing the benefits versus costs of cognitive work. Science, 367(6484), 1362–1366. DOI: 10.1126/science.aaz5891
    Back to article
  112. 112Westbrook, A., Frank, M. J., & Cools, R. (2021). A mosaic of cost–benefit control over cortico-striatal circuitry. Trends in Cognitive Sciences, 25(8), 710–721. DOI: 10.1016/j.tics.2021.04.007
    Back to article
  113. 113Whitton, A. E., Reinen, J. M., Slifstein, M., Ang, Y.-S., McGrath, P. J., Iosifescu, D. V., Abi-Dargham, A., Pizzagalli, D. A., & Schneier, F. R. (2020). Baseline reward processing and ventrostriatal dopamine function are associated with pramipexole response in depression. Brain, 143(2), 701–710. DOI: 10.1093/brain/awaa002
    Back to article
  114. 114Wogar, M. A., Bradshaw, C. M., & Szabadi, E. (1993). Effect of lesions of the ascending 5-hydroxytryptaminergic pathways on choice between delayed reinforcers. Psychopharmacology, 111(2), 239–243. DOI: 10.1007/BF02245530
    Back to article
  115. 115Wunderlich, K., Smittenaar, P., & Dolan, R. J. (2012). Dopamine Enhances Model-Based over Model-Free Choice Behavior. Neuron, 75(3–4), 418–424. DOI: 10.1016/j.neuron.2012.03.042
    Back to article
  116. 116Zimmerman, M., Ellison, W., Young, D., Chelminski, I., & Dalrymple, K. (2015). How many different ways do patients meet the diagnostic criteria for major depressive disorder? Comprehensive Psychiatry, 56, 29–34. DOI: 10.1016/j.comppsych.2014.09.007
    Back to article
DOI: https://doi.org/10.5334/cpsy.83 | Journal eISSN: 2379-6227
Journal RSS Feed
Language: English
Submitted on: Sep 30, 2021
Accepted on: Jun 29, 2022
Published on: Jul 20, 2022
Published by: Ubiquity Press
In partnership with: Paradigm Publishing Services
Publication frequency: 1 issue per year
Keywords:
serotonin,
dopamine,
depression,
antidepressants,
reinforcement learning

© 2022 Denis C. L. Lan, Michael Browning, published by Ubiquity Press
This work is licensed under the Creative Commons Attribution 4.0 License.

Previous articleVolume 6 (2022): Issue 1Next article