Learning and Choice in Mood Disorders: Searching for the Computational Parameters of Anhedonia
By: Oliver J. Robinson and Henry W. Chase
References
- Admon, R. , Kaiser, R. H. , Dillon, D. G. , Beltzer, M. , Goer, F. , Olson, D. P. , … Pizzagalli, D. A. (2017). Dopaminergic enhancement of striatal response to reward in major depression. American Journal of Psychiatry, 174, 378–386. 10.1176/appi.ajp.2016.16010111
- Admon, R. , Nickerson, L. D. , Dillon, D. G. , Holmes, A. J. , Bogdan, R. , Kumar, P. , … Pizzagalli, D. A. (2015). Dissociable cortico-striatal connectivity abnormalities in major depression in response to monetary gains and penalties. Psychological Medicine, 45, 121–131. doi:10.1017/S0033291714001123
- Admon, R. , & Pizzagalli, D. A. (2015). Corticostriatal pathways contribute to the natural time course of positive mood. Nature Communications, 6, Article 10065. doi:10.1038/ncomms10065
- Argyropoulos, S. V. , & Nutt, D. J. (2013). Anhedonia revisited: Is there a role for dopamine-targeting drugs for depression? Journal of Psychopharmacology, 27, 86–77. doi:10.1177/0269881113494104
- Arrondo, G. , Segarra, N. , Metastasio, A. , Ziauddeen, H. , Spencer, J. , Reinders, N. R. , … Murray, G. K. (2015). Reduction in ventral striatal activity when anticipating a reward in depression and schizophrenia: A replicated cross-diagnostic finding. Frontiers in Psychology, 6, Article 1280. doi:10.3389/fpsyg.2015.01280
- Aylward, J. , Hales, C. , Robinson, E. , & Robinson, O. J. (2017). Back-translating a rodent measure of negative bias into humans: The impact of induced anxiety and unmedicated mood and anxiety disorders. bioRxiv. doi:10.1101/143453
- Beddington, J. , Cooper, C. L. , Field, J. , Goswami, U. , Huppert, F. A. , Jenkins, R. , … Thomas, S. M. (2008). The mental wealth of nations. Nature, 455, 1057–1060. doi:10.1038/4551057a
- Beevers, C. G. , Worthy, D. A. , Gorlick, M. A. , Nix, B. , Chotibut, T. , & Todd Maddox, W. (2013). Influence of depression symptoms on history-independent reward and punishment processing. Psychiatry Research, 207(1–2), 53–60. doi:10.1016/j.psychres.2012.09.054
- Berridge, K. C. , & Robinson, T. E. (2003). Parsing reward. Trends in Neuroscience, 26, 507–513. doi:10.1016/S0166-2236(03)00233-9
- Blanco, N. J. , Otto, A. R. , Maddox, W. T. , Beevers, C. G. , & Love, B. C. (2013). The influence of depression symptoms on explor atory decision-making. Cognition, 129, 563–568. doi:10.1016/j.cognition.2013.08.018
- Bogacz, R. , Brown, E. , Moehlis, J. , Holmes, P. , & Cohen, J. D. (2006). The physics of optimal decision making: A formal analysis of models of performance in two-alternative forced-choice tasks. Psychological Review, 113, 700–765. doi:10.1037/0033-295X.113.4.700
- Box, G. E. P. (1976). Science and statistics. Journal of the American Statistical Association, 71, 791–799. doi:10.2307/2286841
- Browning, M. , Behrens, T. E. , Jocham, G. , O’Reilly, J. X. , & Bishop, S. J. (2015). Anxious individuals have difficulty learning the causal statistics of aversive environments. Nature Neuroscience, 18, 590–596. doi:10.1038/nn.3961
- Bush, R. R. , & Mosteller, F. (1955). Stochastic models for learning. Oxford, England: John Wiley.
- Bylsma, L. M. , Morris, B. H. , & Rottenberg, J. (2008). A meta-analysis of emotional reactivity in major depressive disorder. Clinical Psychology Review, 28, 676–691. doi:10.1016/j.cpr.2007.10.001
- Cavanagh, J. F. , Bismark, A. J. , Frank, M. J. , & Allen, J. J. (2011). Larger error signals in major depression are associated with better avoidance learning. Frontiers in Psychology, 2, Article 331. doi:10.3389/fpsyg.2011.00331
- Cavanagh, J. F. , Eisenberg, I. , Guitart-Masip, M. , Huys, Q. , & Frank, M. J. (2013). Frontal theta overrides Pavlovian learning biases. Journal of Neuroscience, 33, 8541–8548.
- Charpentier, C. J. , Aylward, J. , Roiser, J. P. , & Robinson, O. J. (2016). Enhanced risk aversion, but not loss aversion, in unmedicated pathological anxiety. Biological Psychiatry, 81, 1014–1022. doi:10.1016/j.biopsych.2016.12.010
- Chase, H. (2017). Risk simulations. figshare. https://doi.org/10.6084/m9.figshare.5103322.v1. Retrieved: Nov 28, 2017.
- Chase, H. W. , Fournier, J. C. , Bertocci, M. A. , Greenberg, T. , Aslam, H. , Stiffler, R. , (2017). A pathway linking reward circuitry, impulsive sensation-seeking and risky decision-making in young adults: Identifying neural markers for new interventions. Translational Psychiatry, 7(4), e1096. doi:10.1038/tp.2017.60
- Chase, H. W. , Frank, M. J. , Michael, A. , Bullmore, E. T. , Sahakian, B. J. , & Robbins, T. W. (2010). Approach and avoidance learning in patients with major depression and healthy controls: Relation to anhedonia. Psychological Medicine, 40, 433–440. doi:10.1017/S0033291709990468
- Chase, H. W. , Michael, A. , Bullmore, E. T. , Sahakian, B. J. , & Robbins, T. W. (2010). Paradoxical enhancement of choice reaction time performance in patients with major depression. Journal of Psychopharmacology, 24, 471–479. doi:10.1177/0269881109104883
- Chase, H. W. , Nusslock, R. , Almeida, J. R. , Forbes, E. E. , LaBarbara, E. J. , & Phillips, M. L. (2013). Dissociable patterns of abnormal frontal cortical activation during anticipation of an uncertain reward or loss in bipolar versus major depression. Bipolar Disorders, 15, 839–854. doi:10.1111/bdi.12132
- Christianson, J. P. , Paul, E. D. , Irani, M. , Thompson, B. M. , Kubala, K. H. , Yirmiya, R. , (2008). The role of prior stressor controllability and the dorsal raphe nucleus in sucrose preference and social exploration. Behavioural Brain Research, 193, 87–93. doi:10.1016/j.bbr.2008.04.024
- Chung, D. , Kadlec, K. , Aimone, J. A. , McCurry, K. , King-Casas, B. , & Chiu, P. H. (2017). Valuation in major depression is intact and stable in a non-learning environment. Scientific Reports, 7, Article 44374. doi:10.1038/srep44374
- Clery-Melin, M. L. , Schmidt, L. , Lafargue, G. , Baup, N. , Fossati, P. , & Pessiglione, M. (2011). Why don’t you try harder? An investigation of effort production in major depression. PLoS ONE, 6(8), e23178. doi:10.1371/journal.pone.0023178
- Collins, A. G. , Brown, J. K. , Gold, J. M. , Waltz, J. A. , & Frank, M. J. (2014). Working memory contributions to reinforcement learning impairments in schizophrenia. Journal of Neuroscience, 34, 13747–13756. doi:10.1523/JNEUROSCI.0989-14.2014
- Collins, A. G. , & Frank, M. J. (2012). How much of reinforcement learning is working memory, not reinforcement learning? A behavioral, computational, and neurogenetic analysis. European Journal of Neuroscience, 35, 1024–1035. doi:10.1111/j.1460-9568.2011.07980.x
- Cools, R. , Barker, R. A. , Sahakian, B. J. , & Robbins, T. W. (2001). Enhanced or impaired cognitive function in Parkinson’s disease as a function of dopaminergic medication and task de mands. Cerebral Cortex, 11, 1136–1143. Retrieved from http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=Retrieve&db=PubMed&dopt=Citation&list_uids=11709484
- Culver, N. C. , Vervliet, B. , & Craske, M. G. (2015). Compound extinction using the Rescorla–Wagner model to maximize exposure therapy effects for anxiety disorders. Clincial Psychological Science, 3, 335–348.
- Davis, M. , Walker, D. L. , Miles, L. , & Grillon, C. (2010). Phasic vs. sustained fear in rats and humans: Role of the extended amygdala in fear vs anxiety. Neuropsychopharmacology, 35, 105–135. doi:10.1038/npp.2009.109
- Daw N. D. (2011). Trial-by-trial data analysis using computational models. In M. R. Delgado , E. A. Phelps , & T. W. Robbins (Eds.), Decision making, affect, and learning: Attention and performance XXIII (pp. 3–38). Oxford, England: Oxford University Press.
- Daw, N. D. , Kakade, S. , & Dayan, P. (2002). Opponent interactions between serotonin and dopamine. Neural Networks, 15, 603–616. doi:10.1016/S0893-6080(02)00052-7
- DelDonno, S. R. , Weldon, A. L. , Crane, N. A. , Passarotti, A. M. , Pruitt, P. J. , Gabriel, L. B. , … Langenecker, S. A. (2015). Affective personality predictors of disrupted reward learning and pursuit in major depressive disorder. Psychiatry Research, 230, 56–64. doi:10.1016/j.psychres.2015.08.011
- Delgado, M. R. , Li, J. , Schiller, D. , & Phelps, E. A. (2008). The role of the striatum in aversive learning and aversive prediction errors. Philosophical Transactions of the Royal Society, Series B, 363, 3787–3800. doi:10.1098/rstb.2008.0161
- den Ouden, H. E. M. , Daw, N. D. , Fernandez, G. , Elshout, J. A. , Rijpkema, M. , Hoogman, M. , (2013). Dissociable effects of dopamine and serotonin on reversal learning. Neuron, 80, 1090–1100. doi:10.1016/j.neuron.2013.08.030
- Dichter, G. S. , Kozink, R. V. , McClernon, F. J. , & Smoski, M. J. (2012). Remitted major depression is characterized by reward network hyperactivation during reward anticipation and hypoactivation during reward outcomes. Journal of Affective Disorders, 136, 1126–1134. doi:10.1016/j.jad.2011.09.048
- Dillon, D. G. , Wiecki, T. , Pechtel, P. , Webb, C. , Goer, F. , Murray, L. , … Pizzagalli, D. A. (2015). A computational analysis of flanker interference in depression. Psychological Medicine, 45, 2333–2344. doi:10.1017/S0033291715000276
-
Dombrovski, A. Y.
,
Szanto, K.
,
Clark, L.
,
Aizenstein, H. J.
,
Chase, H. W.
,
Reynolds, C. F.,
III , & Siegle, G. J. (2015). Corticostriatothalamic reward prediction error signals and executive control in late-life depression. Psychological Medicine, 45, 1413–1424. doi:10.1017/S0033291714002517 - Dombrovski, A. Y. , Szanto, K. , Clark, L. , Reynolds, C. F. , & Siegle, G. J. (2013). Reward signals, attempted suicide, and impulsiv ity in late-life depression. JAMA Psychiatry, 70, 1020–1030. doi:10.1001/jamapsychiatry.2013.75
- Engelmann, J. B. , Meyer, F. , Fehr, E. , & Ruff, C. C. (2015). Anticipatory anxiety disrupts neural valuation during risky choice. Journal of Neuroscience, 35, 3085–3099.
- Fawcett, J. , Rush, A. J. , Vukelich, J. , Diaz, S. H. , Dunklee, L. , Romo, P. , … Escalona, R. (2016). Clinical experience with high-dosage pramipexole in patients with treatment-resistant depressive epi sodes in unipolar and bipolar depression. American Journal of Psychiatry, 173, 107–111. doi:10.1176/appi.ajp.2015.15060788
- Flagel, S. B. , Watson, S. J. , Robinson, T. E. , & Akil, H. (2007). Individual differences in the propensity to approach signals vs goals promote different adaptations in the dopamine system of rats. Psychopharmacology (Berlin), 191, 599–607. doi:10.1007/s00213-006-0535-8
- Frank, M. J. , Doll, B. B. , Oas-Terpstra, J. , & Moreno, F. (2009). Prefrontal and striatal dopaminergic genes predict individual differences in exploration and exploitation. Nature Neuroscience, 12, 1062–1068. doi:10.1038/nn.2342
- Franken, I. H. , Rassin, E. , & Muris, P. (2007). The assessment of anhedonia in clinical and non-clinical populations: Further validation of the Snaith–Hamilton Pleasure Scale (SHAPS). Journal of Affective Disorders, 99(1–3), 83–89. Retrieved from http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=Retrieve&db=PubMed&dopt=Citation&list_uids=16996138
- Gershman, S. J. (2016). Empirical priors for reinforcement learning models. Journal of Mathematical Psychology, 71, 1–6.
- Gradin, V. B. , Kumar, P. , Waiter, G. , Ahearn, T. , Stickle, C. , Milders, M. , … Steele, J. D. (2011). Expected value and prediction error abnormalities in depression and schizophrenia. Brain, 134(Pt 6), 1751–1764. doi:10.1093/brain/awr059
- Gradin, V. B. , Perez, A. , MacFarlane, J. A. , Cavin, I. , Waiter, G. , Engelmann, J. , … Steele, J. D. (2015). Abnormal brain responses to social fairness in depression: An fMRI study using the Ultimatum Game. Psychological Medicine, 45, 1241–1251. doi:10.1017/S0033291714002347
- Greenberg, T. , Chase, H. W. , Almeida, J. R. , Stiffler, R. , Zevallos, C. R. , Aslam, H. A. , … Phillips, M. L. (2015). Moderation of the relationship between reward expectancy and prediction error-related ventral striatal reactivity by anhedonia in unmedicated major depressive disorder: Findings from the EMBARC study. American Journal of Psychiatry, 172, 881–891. doi:10.1176/appi.ajp.2015.14050594
- Grillon, C. (2008). Models and mechanisms of anxiety: Evidence from startle studies. Psychopharmacology, 199, 421–437. doi:10.1007/s00213-007-1019-1
- Guitart-Masip, M. , Fuentemilla, L. , Bach, D. R. , Huys, Q. J. , Dayan, P. , Dolan, R. J. , (2011). Action dominates valence in antici patory representations in the human striatum and dopaminergic midbrain. Journal of Neuroscience, 31, 7867–7875. doi:10.1523/JNEUROSCI.6376-10.2011
- Hafeman, D. M. , Chang, K. D. , Garrett, A. S. , Sanders, E. M. , & Phillips, M. L. (2012). Effects of medication on neuroimaging findings in bipolar disorder: An updated review. Bipolar Disorders, 14, 375–410. doi:10.1111/j.1399-5618.2012.01023.x
- Hagele, C. , Schlagenhauf, F. , Rapp, M. , Sterzer, P. , Beck, A. , Bermpohl, F. , (2015). Dimensional psychiatry: Reward dysfunction and depressive mood across psychiatric disorders. Psychopharmacology (Berlin), 232, 331–341. doi:10.1007/s00213-014-3662-7
- Hamid, A. A. , Pettibone, J. R. , Mabrouk, O. S. , Hetrick, V. L. , Schmidt, R. , Vander Weele, C. M. , … Berke, J. D. (2016). Mesolimbic dopamine signals the value of work. Nature Neuroscience, 19, 117–126. doi:10.1038/nn.4173
- Heller, A. S. , Johnstone, T. , Light, S. N. , Peterson, M. J. , Kolden, G. G. , Kalin, N. H. , & Davidson, R. J. (2013). Relationships between changes in sustained fronto-striatal connectivity and positive affect in major depression resulting from antidepressant treatment. American Journal of Psychiatry, 170, 197–206. doi:10.1176/appi.ajp.2012.12010014
- Hershenberg, R. , Satterthwaite, T. D. , Daldal, A. , Katchmar, N. , Moore, T. M. , Kable, J. W. , & Wolf, D. H. (2016). Diminished effort on a progressive ratio task in both unipolar and bipolar depression. Journal of Affective Disorders, 196, 97–100. doi:10.1016/j.jad.2016.02.003
- Herzallah, M. M. , Moustafa, A. A. , Natsheh, J. Y. , Abdellatif, S. M. , Taha, M. B. , Tayem, Y. I. , … Gluck, M. A. (2013). Learning from negative feedback in patients with major depressive disorder is attenuated by SSRI antidepressants. Frontiers in Integrative Neuroscience, 7, Article 67. doi:10.3389/fnint.2013.00067
- Holland, P. C. (2004). Relations between Pavlovian-instrumental transfer and reinforcer devaluation. Journal of Experimental Psychology: Animal Behavior Processes, 30, 104–117. doi:10.1037/0097-7403.30.2.104
- Huys, Q. J. , Eshel, N. , O’Nions, E. , Sheridan, L. , Dayan, P. , & Roiser, J. P. (2012). Bonsai trees in your head: How the Pavlovian system sculpts goal-directed choices by pruning decision trees. PLoS Computational Biology, 8(3), e1002410. doi:10.1371/journal.pcbi.1002410
- Huys, Q. J. , Golzer, M. , Friedel, E. , Heinz, A. , Cools, R. , Dayan, P. , (2016). The specificity of Pavlovian regulation is associated with recovery from depression. Psychological Medicine, 46, 1027–1035. doi:10.1017/S0033291715002597
- Huys, Q. J. , Maia, T. V. , & Frank, M. J. (2016). Computational psy chiatry as a bridge from neuroscience to clinical applications. Nature Neuroscience, 19, 404–413. doi:10.1038/nn.4238
- Huys, Q. J. , Pizzagalli, D. A. , Bogdan, R. , & Dayan, P. (2013). Mapping anhedonia onto reinforcement learning: A behavioural meta-analysis. Biology of Mood & Anxiety Disorders, 3(1). doi:10.1186/2045-5380-3-12
- Kendler, K. S. , & Halberstadt, L. J. (2013). The road not taken: Life experiences in monozygotic twin pairs discordant for major depression. Molecular Psychiatry, 18, 975–984. doi:10.1038/mp.2012.55
- Kishida, K. T. , Saez, I. , Lohrenz, T. , Witcher, M. R. , Laxton, A. W. , Tatter, S. B. , … Montague, P. R. (2016). Subsecond dopamine fluctuations in human striatum encode superposed error signals about actual and counterfactual reward. Proceedings of the National Academy of Science of the United States of America, 113, 200–205. doi:10.1073/pnas.1513619112
- Knox, W. B. , Otto, A. R. , Stone, P. , & Love, B. C. (2011). The nature of belief-directed exploratory choice in human decision-making. Frontiers in Psychology, 2, Article 398. doi:10.3389/fpsyg.2011.00398
- Knutson, B. , Bhanji, J. P. , Cooney, R. E. , Atlas, L. Y. , & Gotlib, I. H. (2008). Neural responses to monetary incentives in major depression. Biological Psychiatry, 63, 686–692. doi:10.1016/j.biopsych.2007.07.023
- Knutson, B. , Fong, G. W. , Adams, C. M. , Varner, J. L. , & Hommer, D. (2001). Dissociation of reward anticipation and outcome with event-related fMRI. Neuroreport, 12, 3683–3687. Retrieved from http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=Retrieve&db=PubMed&dopt=Citation&list_uids=11726774
- Knutson, B. , & Heinz, A. (2015). Probing psychiatric symptoms with the monetary incentive delay task. Biological Psychiatry, 77, 418–420. doi:10.1016/j.biopsych.2014.12.022
- Kozak, M. J. , & Cuthbert, B. N. (2016). The nimh research domain criteria initiative: Background, issues, and pragmatics. Psychophysiology, 53, 286–297. doi:10.1111/psyp.12518
- Kumar, P. , Waiter, G. , Ahearn, T. , Milders, M. , Reid, I. , & Steele, J. D. (2008). Abnormal temporal difference reward-learning signals in major depression. Brain, 131(Pt. 8), 2084–2093. doi:10.1093/brain/awn136
- Kunisato, Y. , Okamoto, Y. , Ueda, K. , Onoda, K. , Okada, G. , Yoshimura, S. , … Yamawaki, S. (2012). Effects of depression on reward-based decision making and variability of action in probabilistic learning. Journal of Behavior Therapy & Experimental Psychiatry, 43, 1088–1094. doi:10.1016/j.jbtep.2012.05.007
- Lempert, K. M. , & Pizzagalli, D. A. (2010). Delay discounting and future-directed thinking in anhedonic individuals. Journal of Behavior Therapy & Experimental Psychiatry, 41, 258–264. doi:10.1016/j.jbtep.2010.02.003
- Le Pelley, M. E. , Suret, M. B. , & Beesley, T. (2009). Learned predictiveness effects in humans: A function of learning, performance, or both? Journal of Experimental Psychology: Animal Behavior Processes, 35, 312–327. doi:10.1037/a0014315
- Lohrenz, T. , Kishida, K. T. , & Montague, P. R. (2016). BOLD and its connection to dopamine release in human striatum: A cross-cohort comparison. Philosophical Transactions of the Royal Society of London, Series B, 371(1705). doi:10.1098/rstb.2015.0352
- Maddox, W. T. , Gorlick, M. A. , Worthy, D. A. , & Beevers, C. G. (2012). Depressive symptoms enhance loss-minimization, but attenuate gain-maximization in history-dependent decision-making. Cognition, 125, 118–124. doi:10.1016/j.cognition.2012.06.011
- Martinez-Molina, N. , Mas-Herrero, E. , Rodriguez-Fornells, A. , Zatorre, R. J. , & Marco-Pallares, J. (2016). Neural correlates of specific musical anhedonia. Proceedings of the National Academy of Sciences of the United States of America, 113, E7337–E7345. doi:10.1073/pnas.1611211113
- Misaki, M. , Suzuki, H. , Savitz, J. , Drevets, W. C. , & Bodurka, J. (2016). Individual variations in nucleus accumbens responses associated with major depressive disorder symptoms. Scientific Reports, 6, 21227. doi:10.1038/srep21227
- Mkrtchian, A. , Aylward, J. , Dayan, P. , Roiser, J. P. , & Robinson, O. J. (2017). Modeling avoidance in mood and anxiety disorders using reinforcement learning. Biological Psychiatry, 82, 532–539. doi:10.1016/j.biopsych.2017.01.017
- Montague, P. R. (2012). The scylla and charybdis of neuroeconomic approaches to psychopathology. Biological Psychiatry, 72, 80–81. doi:10.1016/j.biopsych.2012.05.010
- Montague, P. R. , Dolan, R. J. , Friston, K. J. , & Dayan, P. (2012). Computational psychiatry. Trends in Cognitive Sciences, 16, 72–80. doi:10.1016/j.tics.2011.11.018
- Mori, A. , Okamoto, Y. , Okada, G. , Takagaki, K. , Jinnin, R. , Takamura, M. , … Yamawaki, S. (2016). Behavioral activation can normalize neural hypoactivation in subthreshold depression during a monetary incentive delay task. Journal of Affective Disorders, 189, 254–262. doi:10.1016/j.jad.2015.09.036
- Muller, V. I. , Cieslik, E. C. , Serbanescu, I. , Laird, A. R. , Fox, P. T. , & Eickhoff, S. B. (2017). Altered brain activity in unipolar depression revisited: Meta-analyses of neuroimaging studies. JAMA Psychiatry, 74(1), 47–55. doi:10.1001/jamapsychiatry.2016.2783
- Murphy, F. C. , Michael, A. , Robbins, T. W. , & Sahakian, B. J. (2003). Neuropsychological impairment in patients with major depressive disorder: The effects of feedback on task performance. Psychological Medicine, 33, 455–467. Retrieved from http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=Retrieve&db=PubMed&dopt=Citation&list_uids=12701666
- Must, A. , Szabo, Z. , Bodi, N. , Szasz, A. , Janka, Z. , & Keri, S. (2006). Sensitivity to reward and punishment and the prefrontal cortex in major depression. Journal of Affective Disorders, 90, 209–215. doi:10.1016/j.jad.2005.12.005
- Myers, C. E. , Sheynin, J. , Balsdon, T. , Luzardo, A. , Beck, K. D. , Hogarth, L. , … Moustafa, A. A. (2016). Probabilistic reward- and punishment-based learning in opioid addiction: Experimental and computational data. Behavioural Brain Research, 296, 240–248. doi:10.1016/j.bbr.2015.09.018
- NICE. (2014). Anxiety disorders NICE quality standard (Standard No. QS53). Retrieved from https://www.nice.org.uk/guidance/qs53 .
- Niv, Y. , Daw, N. D. , Joel, D. , & Dayan, P. (2007). Tonic dopamine: Opportunity costs and the control of response vigor. Psychopharmacology (Berlin), 191, 507–520. Retrieved from http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=Retrieve&db=PubMed&dopt=Citation&list_uids=17031711
- Oleson, E. B. , Gentry, R. N. , Chioma, V. C. , & Cheer, J. F. (2012). Subsecond dopamine release in the nucleus accumbens predicts conditioned punishment and its successful avoidance. Journal of Neuroscience, 32, 14804–14808. doi:10.1523/jneurosci.3087-12.2012
- Palminteri, S. , Wyart, V. , & Koechlin, E. (2017). The importance of falsification in computational cognitive modeling. Trends in Cognitive Sciences, 21, 425–433. doi:10.1016/j.tics.2017.03.011
- Pe, M. L. , Vandekerckhove, J. , & Kuppens, P. (2013). A diffusion model account of the relationship between the emotional flanker task and rumination and depression. Emotion, 13, 739–747.
- Pechtel, P. , Dutra, S. J. , Goetz, E. L. , & Pizzagalli, D. A. (2013). Blunted reward responsiveness in remitted depression. Journal of Psychiatric Research, 47, 1864–1869. doi:10.1016/j.jpsychires.2013.08.011
- Pedersen, M. L. , Frank, M. J. , & Biele, G. (2016). The drift diffusion model as the choice rule in reinforcement learning. Psychonomic Bulletin & Review, 24, 1234–1251. doi:10.3758/s13423-016-1199-y
- Pessiglione, M. , Seymour, B. , Flandin, G. , Dolan, R. J. , & Frith, C. D. (2006). Dopamine-dependent prediction errors underpin reward-seeking behaviour in humans. Nature, 442, 1042–1045. Retrieved from http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=Retrieve&db=PubMed&dopt=Citation&list_uids=16929307
- Pizzagalli, D. A. (2010). The “anhedonia paradox” in schizophrenia: Insights from affective neuroscience. Biological Psychiatry, 67, 899–901. doi:10.1016/j.biopsych.2010.02.022
- Pizzagalli, D. A. , Holmes, A. J. , Dillon, D. G. , Goetz, E. L. , Birk, J. L. , Bogdan, R. , … Fava, M. (2009). Reduced caudate and nucleus accumbens response to rewards in unmedicated individuals with major depressive disorder. American Journal of Psychiatry, 166, 702–710. doi:10.1176/appi.ajp.2008.08081201
- Pizzagalli, D. A. , Jahn, A. L. , & O’Shea, J. P. (2005). Toward an objective characterization of an anhedonic phenotype: A signal-detection approach. Biological Psychiatry, 57, 319–327. Retrieved from http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=Retrieve&db=PubMed&dopt=Citation&list_uids=15705346
- Poldrack, R. A. (2011). Inferring mental states from neuroimaging data: From reverse inference to large-scale decoding. Neuron, 72, 692–697. doi:10.1016/j.neuron.2011.11.001
- Pulcu, E. , Trotter, P. D. , Thomas, E. J. , McFarquhar, M. , Juhasz, G. , Sahakian, B. J. , … Elliott, R. (2014). Temporal discounting in major depressive disorder. Psychological Medicine, 44, 1825–1834. doi:10.1017/S0033291713002584
- Pulcu, E. , Zahn, R. , Moll, J. , Trotter, P. D. , Thomas, E. J. , Juhasz, G. , … Elliott, R. (2014). Enhanced subgenual cingulate response to altruistic decisions in remitted major depressive disorder. NeuroImage: Clinical, 4, 701–710. doi:10.1016/j.nicl.2014.04.010
- Racagni, G. , Canonico, P. L. , Ravizza, L. , Pani, L. , & Amore, M. (2004). Consensus on the use of substituted benzamides in psychiatric patients. Neuropsychobiology, 50, 134–143. doi:10.1159/000079104
- Radke, S. , Guths, F. , Andre, J. A. , Muller, B. W. , & de Bruijn, E. R. (2014). In action or inaction? Social approach-avoidance tendencies in major depression. Psychiatry Research, 219, 513–517. doi:10.1016/j.psychres.2014.07.011
- Ratcliff, R. , Smith, P. L. , Brown, S. D. , & McKoon, G. (2016). Diffusion decision model: Current issues and history. Trends in Cognitive Sciences, 20, 260–281.
- Raymond, J. G. , Steele, J. D. , & Series, P. (2017). Modeling trait anxiety: From computational processes to personality. Frontiers in Psychiatry, 8, Article 1. doi:10.3389/fpsyt.2017.00001
- Rescorla, R. A. , & Wagner, A. R. (1972). A theory of Pavlovian conditioning: Variations in the effectiveness of reinforcement and nonreinforcement. In A. H. Black & W. F. Prokasy (Eds.), Classical conditioning II (pp. 64–99). New York, NY: Appleton-Century-Crofts.
- Rizvi, S. J. , Pizzagalli, D. A. , Sproule, B. A. , & Kennedy, S. H. (2016). Assessing anhedonia in depression: Potentials and pit falls. Neuroscience & Biobehavioral Reviews, 65, 21–35. doi:10.1016/j.neubiorev.2016.03.004
- Robinson, O. J. , Bond, R. L. , & Roiser, J. P. (2015). The impact of threat of shock on the framing effect and temporal discounting: Executive functions unperturbed by acute stress? Frontiers in Psychology, 6, Article 1315. doi:10.3389/fpsyg.2015.01315
- Robinson, O. J. , Cools, R. , Carlisi, C. O. , Sahakian, B. J. , & Drevets, W. C. (2012). Ventral striatum response during reward and punishment reversal learning in unmedicated major depressive dis order. American Journal of Psychiatry, 169, 152–159. doi:10.1176/appi.ajp.2011.11010137
- Robinson, O. J. , Frank, M. J. , Sahakian, B. J. , & Cools, R. (2010). Dissociable responses to punishment in distinct striatal regions during reversal learning. Neuroimage, 51, 1459–1467. doi:10.1016/j.neuroimage.2010.03.036
- Robinson, O. J. , Overstreet, C. , Charney, D. S. , Vytal, K. , & Grillon, C. (2013). Stress increases aversive prediction error signal in the ventral striatum. Proceedings of the National Academy of Sciences of the United States of America, 110, 4129–4133. doi:10.1073/pnas.1213923110
- Robinson, O. J. , Vytal, K. , Cornwell, B. R. , & Grillon, C. (2013). The impact of anxiety upon cognition: Perspectives from human threat of shock studies. Frontiers in Human Neuroscience, 7, Article 203. doi:10.3389/fnhum.2013.00203
- Rock, P. L. , Roiser, J. P. , Riedel, W. J. , & Blackwell, A. D. (2014). Cognitive impairment in depression: A systematic review and meta-analysis. Psychological Medicine, 44, 2029–2040. doi:10.1017/S0033291713002535
- Roiser, J. P. , Elliott, R. , & Sahakian, B. J. (2012). Cognitive mechanisms of treatment in depression. Neuropsychopharmacology, 37, 117–136. doi:10.1038/npp.2011.183
- Rothkirch, M. , Tonn, J. , Kohler, S. , & Sterzer, P. (2017). Neural mechanisms of reinforcement learning in unmedicated patients with major depressive disorder. Brain, 140, 1147–1157. doi:10.1093/brain/awx025
- Schlagenhauf, F. , Rapp, M. A. , Huys, Q. J. , Beck, A. , Wustenberg, T. , Deserno, L. , … Heinz, A. (2013). Ventral striatal prediction error signaling is associated with dopamine synthesis capacity and fluid intelligence. Human Brain Mapping, 34, 1490–1499. doi:10.1002/hbm.22000
- Schultz, W. , Dayan, P. , & Montague, P. R. (1997). A neural substrate of prediction and reward. Science, 275, 1593–1599. Retrieved from http://www.nc bi.nlm.nih.gov/pubmed/9054347
- Segarra, N. , Metastasio, A. , Ziauddeen, H. , Spencer, J. , Reinders, N. R. , Dudas, R. B. , … Murray, G. K. (2015). Abnormal frontostriatal activity during unexpected reward receipt in depression and schizophrenia: Relationship to anhedonia. Neuropsychopharmacology, 41, 2001–2010. doi:10.1038/npp.2015.370
- Seymour, B. , Daw, N. , Dayan, P. , Singer, T. , & Dolan, R. (2007). Differential encoding of losses and gains in the human striatum. Journal of Neuroscience, 27, 4826–4831. doi:10.1523/jneurosci.0400-07.2007
- Shanks, D. R. , & St. John, M. F. (1994). Characteristics of dissociable human learning-systems. Behavioral and Brain Sciences, 17, 367–395.
- Sherdell, L. , Waugh, C. E. , & Gotlib, I. H. (2012). Anticipatory pleasure predicts motivation for reward in major depression. Journal of Abnormal Psychology, 121, 51–60. doi:10.1037/a0024945
- Smoski, M. J. , Rittenberg, A. , & Dichter, G. S. (2011). Major de pressive disorder is characterized by greater reward network activation to monetary than pleasant image rewards. Psychiatry Research, 194, 263–270. doi:10.1016/j.pscychresns.2011.06.012
- Sokol-Hessner, P. , Hsu, M. , Curley, N. G. , Delgado, M. R. , Camerer, C. F. , & Phelps, E. A. (2009). Thinking like a trader selectively reduces individuals’ loss aversion. Proceedings of the National Academy of Sciences of the United States of America, 106, 5035–5040.
- Steele, J. D. , Kumar, P. , & Ebmeier, K. P. (2007). Blunted response to feedback information in depressive illness. Brain, 130(Pt. 9), 2367–2374. doi:10.1093/brain/awm150
- Stoy, M. , Schlagenhauf, F. , Sterzer, P. , Bermpohl, F. , Hagele, C. , Suchotzki, K. , … Strohle, A. (2012). Hyporeactivity of ven tral striatum towards incentive stimuli in unmedicated depressed patients normalizes after treatment with escitalopram. Journal of Psychopharmacology, 26, 677–688. doi:10.1177/0269881111416686
- Stringaris, A. , Vidal-Ribas Belil, P. , Artiges, E. , Lemaitre, H. , Gollier-Briant, F. , Wolke, S. , … Consortium, I. (2015). The brain’s response to reward anticipation and depression in adolescence: Dimensionality, specificity, and longitudinal predictions in a community-based sample. American Journal of Psychiatry, 172, 1215–1223. doi:10.1176/appi.ajp.2015.14101298
- Sutton, R. S. , & Barto, A. G. (Eds.). (1998). Reinforcement learning: An introduction. Cambridge, MA: MIT Press.
- Taylor Tavares, J. V. , Clark, L. , Furey, M. L. , Williams, G. B. , Sahakian, B. J. , & Drevets, W. C. (2008). Neural basis of abnormal response to negative feedback in unmedicated mood disorders. Neuroimage, 42, 1118–1126. doi:10.1016/j.neuroimage.2008.05.049
- Treadway, M. T. , Bossaller, N. A. , Shelton, R. C. , & Zald, D. H. (2012).Effort-based decision-making in major depressive dis order: A translational model of motivational anhedonia. Journal of Abnormal Psychology, 121, 553–558. doi:10.1037/a0028813
- Treadway, M. T. , & Zald, D. H. (2011). Reconsidering anhedonia in depression: Lessons from translational neuroscience. Neuroscience & Biobehavioral Reviews, 35, 537–555. doi:10.1016/j.neubiorev.2010.06.006
- Tsetsos, K. , Gao, J. , McClelland, J. L. , & Usher, M. (2012). Using time-varying evidence to test models of decision dynamics: Bounded diffusion vs. the leaky competing accumulator model. Frontiers in Neuroscience, 6, Article 79. doi:10.3389/fnins.2012.00079
- Tversky, A. , & Kahneman, D. (1992). Advances in prospect theory: Cumulative representation of uncertainty. Journal of Risk and Uncertainty, 5, 297–323. doi:10.1007/BF00122574
- Ubl, B. , Kuehner, C. , Kirsch, P. , Ruttorf, M. , Diener, C. , & Flor, H. (2015). Altered neural reward and loss processing and prediction error signalling in depression. Social Cognitive & Affective Neuroscience, 10, 1102–1112. doi:10.1093/scan/nsu158
- Vadillo, M. A. , Konstantinidis, E. , & Shanks, D. R. (2016). Under powered samples, false negatives, and unconscious learning. Psychonomics Bulletin & Reviews, 23, 87–102. doi:10.3758/s13423-015-0892-6
- Vallesi, A. , Canalaz, F. , Balestrieri, M. , & Brambilla, P. (2015). Modulating speed-accuracy strategies in major depression. Journal of Psychiatric Research, 60, 103–108. doi:10.1016/j.jpsychires.2014.09.017
- van Ravenzwaaij, D. , Donkin, C. , & Vandekerckhove, J. (2017). The EZ diffusion model provides a powerful test of simple empirical effects. Psychonomics Bulletin & Reviews, 24, 547–556. doi:10.3758/s13423-016-1081-y
- Vrieze, E. , Pizzagalli, D. A. , Demyttenaere, K. , Hompes, T. , Sienaert, P. , de Boer, P. , … Claes, S. (2012). Reduced reward learning predicts outcome in major depressive disorder. Biological Psychiatry, 73, 639–645. doi:10.1016/j.biopsych.2012.10.014
- Wacker, J. , Dillon, D. G. , & Pizzagalli, D. A. (2009). The role of the nucleus accumbens and rostral anterior cingulate cortex in anhedonia: Integration of resting EEG, fMRI, and volumetric techniques. Neuroimage, 46, 327–337. doi:10.1016/j.neuroimage.2009.01.058
- White, C. N. , Ratcliff, R. , Vasey, M. W. , & McKoon, G. (2010). Using diffusion models to understand clinical disorders. Journal of Mathematical Psychology, 54, 39–52.
- Whitmer, A. J. , Frank, M. J. , & Gotlib, I. H. (2012). Sensitivity to reward and punishment in major depressive disorder: Effects of rumination and of single versus multiple experiences. Cognition & Emotion, 26, 1475–1485. doi:10.1080/02699931.2012.682973
- Wilson, R. C. , Geana, A. , White, J. M. , Ludvig, E. A. , & Cohen, J. D. (2014). Humans use directed and random exploration to solve the explore-exploit dilemma. Journal of Experimental Psychology, General, 143, 2074–2081. doi:10.1037/a0038199
- Wu, C. C. , Samanez-Larkin, G. R. , Katovich, K. , & Knutson, B. (2014). Affective traits link to reliable neural markers of incentive anticipation. Neuroimage, 84, 279–289. doi:10.1016/j.neuroimage.2013.08.055
- Xu, J. (2015). Implications of cortical balanced excitation and inhibition, functional heterogeneity, and sparseness of neuronal activity in fMRI. Neuroscience & Biobehavioral Reviews, 57, 264–270. doi:10.1016/j.neubiorev.2015.08.018
- Yang, X. H. , Huang, J. , Lan, Y. , Zhu, C. Y. , Liu, X. Q. , Wang, Y. F. , … Chan, R. C. (2016). Diminished caudate and superior temporal gyrus responses to effort-based decision making in patients with first-episode major depressive disorder. Progress in Neuropsychopharmacology & Biological Psychiatry, 64, 52–59. doi:10.1016/j.pnpbp.2015.07.006
- Yang, X. H. , Huang, J. , Zhu, C. Y. , Wang, Y. F. , Cheung, E. F. , Chan, R. C. , & Xie, G. R. (2014). Motivational deficits in effort-based decision making in individuals with subsyndromal depression, first-episode and remitted depression patients. Psychiatry Research, 220, 874–882. doi:10.1016/j.psychres.2014.08.056
- Yonkers, K. A. , Warshaw, M. G. , Massion, A. O. , & Keller, M. B. (1996). Phenomenology and course of generalised anxiety disorder. British Journal of Psychiatry, 168, 308–313. doi:10.1192/bjp.168.3.308
- Yuille, A. L. , & Gaiger, D. (2003). Winner-take-all networks . In M. A. Arbib , (Ed.), The handbook of brain theory and neural networks (2nd ed., pp. 1228–1231). Cambridge, MA: MIT Press.
- Zhang, W. N. , Chang, S. H. , Guo, L. Y. , Zhang, K. L. , & Wang, J. (2013). The neural correlates of reward-related processing in major depressive disorder: A meta-analysis of functional magnetic resonance imaging studies. Journal of Affective Disorders, 151, 531–539. doi:10.1016/j.jad.2013.06.039
DOI: https://doi.org/10.1162/CPSY_a_00009 | Journal eISSN: 2379-6227
Language: English
Submitted on: May 20, 2016
Accepted on: Jun 6, 2017
Published on: Dec 1, 2017
Published by: MIT Press
In partnership with: Paradigm Publishing Services
Publication frequency: 1 issue per year
© 2017 Oliver J. Robinson, Henry W. Chase, published by MIT Press
This work is licensed under the Creative Commons Attribution 4.0 License.