References
- 1Bayer, H. M., & Glimcher, P. W. (2005). Midbrain dopamine neurons encode a quantitative reward prediction error signal. Neuron, 47(1), 129–141. DOI: 10.1016/j.neuron.2005.05.020
- 2Bechara, A., Damasio, A. R., Damasio, H., & Anderson, S. W. (1994). Insensitivity to future consequences following damage to human prefrontal cortex. Cognition, 50, 1–3. DOI: 10.1016/0010-0277(94)90018-3
- 3Cazé, R. D., & van der Meer, M. A. (2013). Adaptive properties of differential learning rates for positive and negative outcomes. Biological cybernetics, 107(6), 711–719. DOI: 10.1007/s00422-013-0571-5
- 4Chambon, V., Théro, H., Vidal, M., Vandendriessche, H., Haggard, P., & Palminteri, S. (2020). Information about action outcomes differentially affects learning from self-determined versus imposed choices. Nature Human Behaviour, 4(10), 1067–1079. DOI: 10.1038/s41562-020-0919-5
- 5Chiu, Y.-C., Huang, J.-T., Duann, J.-R., & Lin, C.-H. (2018). Twenty years after the iowa gambling task: rationality, emotion, and decision-making. Frontiers in psychology, 8, 2353. DOI: 10.3389/fpsyg.2017.02353
- 6Cohen, J. Y., Haesler, S., Vong, L., Lowell, B. B., & Uchida, N. (2012). Neuron-type-specific signals for reward and punishment in the ventral tegmental area. nature, 482(7383), 85. DOI: 10.1038/nature10754
- 7Collins, A. G., & Frank, M. J. (2014). Opponent actor learning (opal): Modeling interactive effects of striatal dopamine on reinforcement learning and choice incentive. Psychological review, 121(3), 337. DOI: 10.1037/a0037015
- 8Cools, R., Nakamura, K., & Daw, N. D. (2011). Serotonin and dopamine: unifying affective, activational, and decision functions. Neuropsychopharmacology, 36(1), 98–113. DOI: 10.1038/npp.2010.121
- 9Dabney, W., Kurth-Nelson, Z., Uchida, N., Starkweather, C. K., Hassabis, D., Munos, R., & Botvinick, M. (2020). A distributional code for value in dopamine-based reinforcement learning. Nature, 577(7792), 671–675. DOI: 10.1038/s41586-019-1924-6
- 10d’Acremont, M., Lu, Z.-L., Li, X., Van der Linden, M., & Bechara, A. (2009). Neural correlates of risk prediction error during reinforcement learning in humans. Neuroimage, 47(4), 1929–1939. DOI: 10.1016/j.neuroimage.2009.04.096
- 11Daw, N. D., Gershman, S. J., Seymour, B., Dayan, P., & Dolan, R. J. (2011). Model-based influences on humans’ choices and striatal prediction errors. Neuron, 69(6), 1204–1215. DOI: 10.1016/j.neuron.2011.02.027
- 12Daw, N. D., Kakade, S., & Dayan, P. (2002). Opponent interactions between serotonin and dopamine. Neural Networks, 15(4–6), 603–616. DOI: 10.1016/S0893-6080(02)00052-7
- 13Dayan, P., & Huys, Q. J. (2008). Serotonin, inhibition, and negative mood. PLoS Comput Biol, 4(2), e4. DOI: 10.1371/journal.pcbi.0040004
- 14Dayan, P., & Huys, Q. J. (2009). Serotonin in affective control. Annual review of neuroscience, 32. DOI: 10.1146/annurev.neuro.051508.135607
- 15Deakin, J. (1983). Roles of serotonergic systems in escape, avoidance and other behaviours. Theory in psychopharmacology, 2, 149–193.
- 16Deakin, J. W., & Graeff, F. G. (1991). 5-ht and mechanisms of defence. Journal of psychopharmacology, 5(4), 305–315. DOI: 10.1177/026988119100500414
- 17Fontanesi, L., Gluth, S., Spektor, M. S., & Rieskamp, J. (2019). A reinforcement learning diffusion decision model for value-based decisions. Psychonomic bulletin & review, 26(4), 1099–1121. DOI: 10.3758/s13423-018-1554-2
- 18Gershman, S. J., Monfils, M.-H., Norman, K. A., & Niv, Y. (2017). The computational nature of memory modification. Elife, 6,
e23763 . DOI: 10.7554/eLife.23763.019 - 19Glimcher, P. W. (2011). Understanding dopamine and reinforcement learning: the dopamine reward prediction error hypothesis. Proceedings of the National Academy of Sciences, 108(Supplement 3), 15647–15654. DOI: 10.1073/pnas.1014269108
- 20Glimcher, P. W., & Rustichini, A. (2004). Neuroeconomics: the consilience of brain and decision. Science, 306(5695), 447–452. DOI: 10.1126/science.1102566
- 21Grupe, D. W., & Nitschke, J. B. (2013). Uncertainty and anticipation in anxiety: an integrated neurobiological and psychological perspective. Nature Reviews Neuroscience, 14(7), 488–501. DOI: 10.1038/nrn3524
- 22Hauser, T. U., Iannaccone, R., Walitza, S., Brandeis, D., & Brem, S. (2015). Cognitive flexibility in adolescence: neural and behavioral mechanisms of reward prediction error processing in adaptive decision making during development. Neuroimage, 104, 347–354. DOI: 10.1016/j.neuroimage.2014.09.018
- 23Hirsh, J. B., Mar, R. A., & Peterson, J. B. (2012). Psychological entropy: A framework for understanding uncertainty-related anxiety. Psychological review, 119(2), 304. DOI: 10.1037/a0026767
- 24Huang, H., Thompson, W., & Paulus, M. P. (2017). Computational dysfunctions in anxiety: Failure to differentiate signal from noise. Biological psychiatry, 82(6), 440–446. DOI: 10.1016/j.biopsych.2017.07.007
- 25Jepma, M., Schaaf, J. V., Visser, I., & Huizenga, H. M. (2020). Uncertainty-driven regulation of learning and exploration in adolescents: A computational account. PLoS computational biology, 16(9),
e1008276 . DOI: 10.1371/journal.pcbi.1008276 - 26Kahneman, D., & Tversky, A. (2013).
Choices, values, and frames . In Handbook of the fundamentals of financial decision making: Part i (pp. 269–278). World Scientific. DOI: 10.1142/9789814417358_0016 - 27Kilpatrick, Z. P., Holmes, W. R., Eissa, T. L., & Josić, K. (2019). Optimal models of decision-making in dynamic environments. Current Opinion in Neurobiology, 58, 54–60. DOI: 10.1016/j.conb.2019.06.006
- 28Kishida, K. T., Saez, I., Lohrenz, T., Witcher, M. R., Laxton, A. W., Tatter, S. B., … Montague, P. R. (2016). Subsecond dopamine fluctuations in human striatum encode superposed error signals about actual and counterfactual reward. Proceedings of the National Academy of Sciences, 113(1), 200–205. DOI: 10.1073/pnas.1513619112
- 29Korn, C., Sharot, T., Walter, H., Heekeren, H., & Dolan, R. J. (2014). Depression is related to an absence of optimistically biased belief updating about future life events. Psychological medicine, 44(3), 579–592. DOI: 10.1017/S0033291713001074
- 30Lefebvre, G., Lebreton, M., Meyniel, F., Bourgeois-Gironde, S., & Palminteri, S. (2017). Behavioural and neural characterization of optimistic reinforcement learning. Nature Human Behaviour, 1(4), 1–9. DOI: 10.1038/s41562-017-0067
- 31Lefebvre, G., Summerfield, C., & Bogacz, R. (2020). A normative account of confirmatory biases during reinforcement learning. BioRxiv. DOI: 10.1101/2020.05.12.090134
- 32Li, J., Schiller, D., Schoenbaum, G., Phelps, E. A., & Daw, N. D. (2011). Differential roles of human striatum and amygdala in associative learning. Nature neuroscience, 14(10), 1250. DOI: 10.1038/nn.2904
- 33Lin, C.-H., Chiu, Y.-C., Lee, P.-L., & Hsieh, J.-C. (2007, Mar 15). Is deck b a disadvantageous deck in the iowa gambling task? Behavioral and Brain Functions, 3(1), 16. DOI: 10.1186/1744-9081-3-16
- 34Luhmann, C. C., Ishida, K., & Hajcak, G. (2011). Intolerance of uncertainty and decisions about delayed, probabilistic rewards. Behavior Therapy, 42(3), 378–386. DOI: 10.1016/j.beth.2010.09.002
- 35Mihatsch, O., & Neuneier, R. (2002). Risk-sensitive reinforcement learning. Machine learning, 49(2–3), 267–290. DOI: 10.1023/A:1017940631555
- 36Mikhael, J. G., & Bogacz, R. (2016). Learning reward uncertainty in the basal ganglia. PLoS computational biology, 12(9),
e1005062 . DOI: 10.1371/journal.pcbi.1005062 - 37Montague, P. R., Dayan, P., & Sejnowski, T. J. (1996). A framework for mesencephalic dopamine systems based on predictive hebbian learning. Journal of neuroscience, 16(5), 1936–1947. DOI: 10.1523/JNEUROSCI.16-05-01936.1996
- 38Montague, P. R., Kishida, K. T., Moran, R. J., & Lohrenz, T. M. (2016). An efficiency framework for valence processing systems inspired by soft cross-wiring. Current opinion in behavioral sciences, 11, 121–129. DOI: 10.1016/j.cobeha.2016.08.002
- 39Moran, R. J., Kishida, K. T., Lohrenz, T., Saez, I., Laxton, A. W., Witcher, M. R., … Montague, P. R. (2018). The protective action encoding of serotonin transients in the human brain. Neuropsychopharmacology, 43(6), 1425. DOI: 10.1038/npp.2017.304
- 40Niv, Y., Duff, M. O., & Dayan, P. (2005). Dopamine, uncertainty and td learning. Behavioral and brain Functions, 1(1), 6. DOI: 10.1186/1744-9081-1-6
- 41Niv, Y., Edlund, J. A., Dayan, P., & O’Doherty, J. P. (2012). Neural prediction errors reveal a risk-sensitive reinforcement-learning process in the human brain. Journal of Neuroscience, 32(2), 551–562. DOI: 10.1523/JNEUROSCI.5498-10.2012
- 42Pan, W.-X., Schmidt, R., Wickens, J. R., & Hyland, B. I. (2005). Dopamine cells respond to predicted events during classical conditioning: evidence for eligibility traces in the reward-learning network. Journal of Neuroscience, 25(26), 6235–6242. DOI: 10.1523/JNEUROSCI.1478-05.2005
- 43Pedersen, M. L., Frank, M. J., & Biele, G. (2017). The drift diffusion model as the choice rule in reinforcement learning. Psychonomic bulletin & review, 24(4), 1234–1251. DOI: 10.3758/s13423-016-1199-y
- 44Platt, M. L., & Huettel, S. A. (2008). Risky business: the neuroeconomics of decision making under uncertainty. Nature neuroscience, 11(4), 398. DOI: 10.1038/nn2062
- 45Preuschoff, K., Quartz, S. R., & Bossaerts, P. (2008). Human insula activation reflects risk prediction errors as well as risk. Journal of Neuroscience, 28(11), 2745–2752. DOI: 10.1523/JNEUROSCI.4286-07.2008
- 46Priyadharsini, B. P., Ravindran, B., & Chakravarthy, V. S. (2012). Understanding the role of serotonin in basal ganglia through a unified model. In International conference on artificial neural networks (pp. 467–473). DOI: 10.1007/978-3-642-33269-2_59
- 47Redish, A. D., Jensen, S., Johnson, A., & Kurth-Nelson, Z. (2007). Reconciling reinforcement learning models with behavioral extinction and renewal: implications for addiction, relapse, and problem gambling. Psychological review, 114(3), 784. DOI: 10.1037/0033-295X.114.3.784
- 48Rescorla, R. A., & Wagner, A. R. (1972). A theory of pavlovian conditioning: Variations in the effectiveness of reinforcement and nonreinforcement. Classical conditioning II: Current research and theory, 2, 64–99.
- 49Rogers, R. D. (2011). The roles of dopamine and serotonin in decision making: evidence from pharmacological experiments in humans. Neuropsychopharmacology, 36(1), 114–132. DOI: 10.1038/npp.2010.165
- 50Ross, M. C., Lenow, J. K., Kilts, C. D., & Cisler, J. M. (2018). Altered neural encoding of prediction errors in assault-related posttraumatic stress disorder. Journal of psychiatric research, 103, 83–90. DOI: 10.1016/j.jpsychires.2018.05.008
- 51Rouhani, N., & Niv, Y. (2019). Depressive symptoms bias the prediction-error enhancement of memory towards negative events in reinforcement learning. Psychopharmacology, 236(8), 2425–2435. DOI: 10.1007/s00213-019-05322-z
- 52Schultz, W., Apicella, P., & Ljungberg, T. (1993). Responses of monkey dopamine neurons to reward and conditioned stimuli during successive steps of learning a delayed response task. Journal of neuroscience, 13(3), 900–913. DOI: 10.1523/JNEUROSCI.13-03-00900.1993
- 53Schultz, W., Dayan, P., & Montague, P. R. (1997). A neural substrate of prediction and reward. Science, 275(5306), 1593–1599. DOI: 10.1126/science.275.5306.1593
- 54Steingroever, H., Wetzels, R., & Wagenmakers, E.-J. (2013). A comparison of reinforcement learning models for the iowa gambling task using parameter space partitioning. Journal of Problem Solving, 5(2). DOI: 10.7771/1932-6246.1150
- 55Sutton, R. S., & Barto, A. G. (2018). Reinforcement learning: An introduction. MIT press.
- 56Veliz-Cuba, A., Kilpatrick, Z. P., & Josic, K. (2016). Stochastic models of evidence accumulation in changing environments. SIAM Review, 58(2), 264–289. DOI: 10.1137/15M1028443
- 57Worbe, Y., Savulich, G., Voon, V., Fernandez-Egea, E., & Robbins, T. W. (2014). Serotonin depletion induces ‘waiting impulsivity’ on the human four-choice serial reaction time task: cross-species translational significance. Neuropsychopharmacology, 39(6), 1519–1526. DOI: 10.1038/npp.2013.351
- 58Yu, A. J., & Dayan, P. (2005). Uncertainty, neuromodulation, and attention. Neuron, 46(4), 681–692. DOI: 10.1016/j.neuron.2005.04.026
- 59Zaghloul, K. A., Blanco, J. A., Weidemann, C. T., McGill, K., Jaggi, J. L., Baltuch, G. H., & Kahana, M. J. (2009). Human substantia nigra neurons encode unexpected financial rewards. Science, 323(5920), 1496–1499. DOI: 10.1126/science.1167342