Have a personal or library account? Click to login
Revealing Neurocomputational Mechanisms of Reinforcement Learning and Decision-Making With the hBayesDM Package Cover

Revealing Neurocomputational Mechanisms of Reinforcement Learning and Decision-Making With the hBayesDM Package

Open Access
|Oct 2017

References

  1. Agay, N., Kron, S., Carmel, Z., Mendlovic, S., & Levkovitz, Y. (2008). Ultimatum bargaining behavior of people affected by schizophrenia. Psychiatry Research, 157, 3946. doi:10.1016/j.psychres.2006.03.026
  2. Ahn, W.-Y., & Busemeyer, J. R. (2016). Challenges and promises for translating computational tools into clinical practice. Current Opinion in Behavioral Sciences, 11, 17. doi:10.1016/j.cobeha.2016.02.001
  3. Ahn, W.-Y., Busemeyer, J. R., Wagenmakers, E. J., & Stout, J. C. (2008). Comparison of decision learning models using the generalization criterion method. Cognitive Science, 32, 13761402. doi:10.1080/03640210802352992
  4. Ahn, W.-Y., Krawitz, A., Kim, W., Busemeyer, J. R., & Brown, J. W. (2011). A model-based fMRI analysis with hierarchical Bayesian parameter estimation. Journal of Neuroscience, Psychology, and Economics, 4, 95110. doi:10.1037/a0020684
  5. Ahn, W.-Y., Ramesh, D., Moeller, F. G., & Vassileva, J. (2016). Utility of machine-learning approaches to identify behavioral markers for substance use disorders: Impulsivity dimensions as predictors of present cocaine dependence. Frontiers in Psychiatry, 7, 290. doi:10.3389/fpsyt.2016.00034
  6. Ahn, W.-Y., Rass, O., Fridberg, D. J., Bishara, A. J., Forsyth, J. K., Breier, A., … O’Donnell, B. F. (2011). Temporal discounting of reward in patients with bipolar disorder and schizophrenia. Journal of Abnormal Psychology, 120, 911921. doi:10.1037/a0023333
  7. Ahn, W.-Y., & Vassileva, J. (2016). Machine-learning identifies substance-specific behavioral markers for opiate and stimulant dependence. Drug and Alcohol Dependence, 161, 247257. doi:10.1016/j.drugalcdep.2016.02.008
  8. Ahn, W.-Y., Vasilev, G., Lee, S.-H., Busemeyer, J. R., Kruschke, J. K., Bechara, A., & Vassileva, J. (2014). Decision-making in stimulant and opiate addicts in protracted abstinence: Evidence from computational modeling with pure users. Frontiers in Psychology, 5, 849. doi:10.3389/fpsyg.2014.00849
  9. Akaike, H. (1987). Factor analysis and AIC. Psychometrika, 52, 317332.
  10. Albrecht, M. A., Waltz, J. A., Cavanagh, J. F., Frank, M. J., & Gold, J. M. (2016). Reduction of Pavlovian bias in schizophrenia: Enhanced effects in clozapine-administered patients. PLoS ONE, 11, e0152781. doi:10.1371/journal.pone.0152781
  11. Bartlema, A., Lee, M., Wetzels, R., & Vanpaemel, W. (2014). A Bayesian hierarchical mixture approach to individual differences: Case studies in selective attention and representation in category learning. Journal of Mathematical Psychology, 59, 132150. doi:10.1016/j.jmp.2013.12.002
  12. Bechara, A., Damasio, A. R., Damasio, H., & Anderson, S. W. (1994). Insensitivity to future consequences following damage to human prefrontal cortex. Cognition, 50, 715. doi:10.1016/0010-0277(94)90018-3
  13. Bechara, A., Dolan, S., Denburg, N., Hindes, A., Anderson, S. W., & Nathan, P. E. (2001). Decision-making deficits, linked to a dysfunctional ventromedial prefrontal cortex, revealed in alcohol and stimulant abusers. Neuropsychologia, 39, 376389. doi:10.1016/S0028-3932(00)00136-6
  14. Bechara, A., & Martin, E. M. (2004). Impaired decision making related to working memory deficits in individuals with substance addictions. Neuropsychology, 18, 152162. doi:10.1037/0894-4105.18.1.152
  15. Behrens, T. E. J., Woolrich, M. W., Walton, M. E., & Rushworth, M. F. S. (2007). Learning the value of information in an uncertain world. Nature Neuroscience, 10, 12141221. doi:10.1038/nn1954
  16. Bickel, W. K. (2015).Discounting of delayed reward as an endophenotype. Biological Psychiatry, 77, 846847. doi:10.1016/j.biopsych.2015.03.003
  17. Bolla, K. I., Eldreth, D. A., London, E. D., Kiehl, K. A., Mouratidis, M., Contoreggi, C., … Ernst, M. (2003). Orbitofrontal cortex dysfunction in abstinent cocaine abusers performing a decision-making task. NeuroImage, 19, 10851094. doi:10.1016/S1053-8119(03)00113-7
  18. Bozdogan, H. (1987). Model selection and Akaike’s Information Criterion (AIC): The general theory and its analytical extensions. Psychometrika, 52, 345370.
  19. Busemeyer, J. R., Diederich, A. (2010). Cognitive modeling. New York, NY: Sage. doi:10.1037/e722292011-099
  20. Busemeyer, J. R., & Wang, Y.-M. (2000). Model comparisons and model selections on the basis of generalization criterion meth odology. Journal of Mathematical Psychology, 44, 119. doi:10.1006/jmps.1999.1282
  21. Camerer, C. F., & Ho, T.-H. (1999). Experienced-weighted attraction learning in normal form games. Econometrica, 67, 827874.
  22. Carpenter, B., Gelman, A., Hoffman, M., Lee, D., Goodrich, B., Betancourt, M., … Riddell, A. (2016). Stan: A probabilistic pro gramming language. Journal of Statistical Software, 76(1), 132. doi:10.18637/jss.v076.i01
  23. Cavanagh, J. F., Eisenberg, I., Guitart-Masip, M., Huys, Q., & Frank, M. J. (2013). Frontal theta overrides Pavlovian learning biases. Journal of Neuroscience, 33, 85418548.
  24. Chan, T. W. S., Ahn, W.-Y., Bates, J. E., Busemeyer, J. R., Guillaume, S., Redgrave, G. W., … Courtet, P. (2014). Differential impairments underlying decision making in anorexia nervosa and bulimia nervosa: A cognitive modeling analysis. International Journal of Eating Disorders, 47, 157167. doi:10.1002/eat.22223
  25. Chowdhury, R., Guitart-Masip, M., Lambert, C., Dolan, R. J., & Duzel, E. (2013). Structural integrity of the substantia nigra and subthalamic nucleus predicts flexibility of instrumental learning in older-age individuals, 34, 22612270. doi:10.1016/j.neurobiolaging.2013.03.030
  26. Cools, R., Clark, L., Owen, A. M., & Robbins, T. W. (2002). Defining the neural mechanisms of probabilistic reversal learning using event-related functional magnetic resonance imaging. Journal of Neuroscience, 22, 45634567.
  27. Cools, R., Lewis, S. J., Clark, L., Barker, R. A., & Robbins, T. W. (2007). L-DOPA disrupts activity in the nucleus accumbens during reversal learning in Parkinson’s disease. Neuropsychopharmacology, 32, 180189. doi:10.1038/sj.npp.1301153
  28. Csukly, G., Polgár, P., Tombor, L., Réthelyi, J., & Kéri, S. (2011). Are patients with schizophrenia rational maximizers? Evidence from an ultimatum game study. Psychiatry Research, 187, 1117. doi:10.1016/j.psychres.2010.10.005
  29. Daunizeau, J., Adam, V., & Rigoux, L. (2014). VBA: A probabilistic treatment of nonlinear models for neurobiological and behavioural data. PLoS Computational Biology, 10, e1003441. doi:10.1371/journal.pcbi.1003441
  30. Daw, N. D., O’Doherty, J. P., Dayan, P., Seymour, B., & Dolan, R. J. (2006). Cortical substrates for exploratory decisions in humans. Nature, 441, 876879. doi:10.1038/nature04766
  31. Dayan, P., & Daw, N. D. (2008). Decision theory, reinforcement learning, and the brain. Cognitive, Affective, and Behavioral Neuroscience, 8, 429453. doi:10.3758/CABN.8.4.429
  32. Dayan, P., Niv, Y., Seymour, B., & Daw, N. D. (2006). The misbehavior of value and the discipline of the will. Neural Networks, 19, 11531160. doi:10.1016/j.neunet.2006.03.002
  33. den Ouden, H. E. M., Daw, N. D., Fernandez, G., Elshout, J. A., Rijpkema, M., Hoogman, M., … Cools, R. (2013). Dissociable effects of dopamine and serotonin on reversal learning. Neuron, 80, 10901100. doi:10.1016/j.neuron.2013.08.030
  34. Ebert, J., & Prelec, D. (2007). The fragility of time: Time-insensitivity and valuation of the near and far future. Management Science, 53, 14231438.
  35. Erev, I., Ert, E., Roth, A. E., Haruvy, E., Herzog, S. M., Hau, R., … Lebiere, C. (2010). A choice prediction competition: Choices from experience and from description. Journal of Behavioral Decision Making, 23, 1547.
  36. Ersche, K. D., Roiser, J. P., Robbins, T. W., & Sahakian, B. J. (2008). Chronic cocaine but not chronic amphetamine use is associated with perseverative responding in humans. Psychopharmacology, 197, 421431. doi:10.1007/s00213-007-1051-1
  37. Fellows, L. K., & Farah, M. J. (2003). Ventromedial frontal cortex mediates affective shifting in humans: Evidence from a reversal learning paradigm. Brain, 126, 18301837.
  38. Forstmann, B. U.Wagenmakers, E.-J.2015). An introduction to model-based cognitive neuroscience. Berlin, Germany: Springer. doi:10.1007/978-1-4939-2236-9
  39. Fridberg, D. J., Ahn, W.-Y., Kim, W., Bishara, A. J., & Stout, J. C. (2010). Cognitive mechanisms underlying risky decision-making in chronic cannabis users. Journal of Mathematical Psychology, 54, 2838. doi:10.1016/j.jmp.2009.10.002
  40. Friston, K. J., Stephan, K. E., Montague, P. R., & Dolan, R. J. (2014). Computational psychiatry: The brain as a phantastic organ. Lancet: Psychiatry, 1, 148158. doi:10.1016/S2215-0366(14)70 275-5
  41. Gelman, A. (2006). Prior distributions for variance parameters in hierarchical models (comment on article by Browne and Draper). Bayesian Analysis, 1, 515534.
  42. Gelman, A., Dunson, D. B., Vehtari, A. (2013). Bayesian data analysis (3rd ed.). New York, NY: CRC Press.
  43. Gelman, A., & Rubin, D. B. (1992). Inference from iterative simulation using multiple sequences. Statistical Science, 7, 457472.
  44. Gläscher, J., Daw, N. D., Dayan, P., & Doherty, J. P. O. (2010). States versus rewards: Dissociable neural prediction error signals underlying model-based and model-free reinforcement learning. Neuron, 66, 585595. doi:10.1016/j.neuron.2010.04.016
  45. Gläscher, J., Hampton, A. N., & O’Doherty, J. P. (2009). Determining a role for ventromedial prefrontal cortex in encoding action-based value signals during reward-related decision making. Cerebral Cortex, 19, 483495. doi:10.1093/cercor/bhn098
  46. Grant, S., Contoreggi, C., & London, E. D. (2000). Drug abusers show impaired performance in a laboratory test of decision making. Neuropsychologia, 38, 11801187.
  47. Green, L., & Myerson, J. (2004). A Discounting framework for choice with delayed and probabilistic rewards. Psychological Bulletin, 130, 769792. doi:10.1037/0033-2909.130.5.769
  48. Gu, X., Wang, X., Hula, A., Wang, S., Xu, S., Lohrenz, T. M., … Montague, P. R. (2015). Necessary, yet dissociable contributions of the insular and ventromedial prefrontal cortices to norm adaptation: Computational and lesion evidence in humans. Journal of Neuroscience, 35, 467473. doi:10.1523/JNEUROSCI.2906-14.2015
  49. Guitart-Masip, M., Duzel, E., Dolan, R., & Dayan, P. (2014). Action versus valence in decision making. Trends in Cognitive Sciences, 18, 194202. doi:10.1016/j.tics.2014.01.003
  50. Guitart-Masip, M., Huys, Q. J. M., Fuentemilla, L., Dayan, P., Duzel, E., & Dolan, R. J. (2012). Go and no-go learning in reward and punishment: Interactions between affect and effect. NeuroImage, 62, 154166. doi:10.1016/j.neuroimage.2012.04.024
  51. Güth, W., Schmittberger, R., & Schwarze, B. (1982). An experimental analysis of ultimatum bargaining. Journal of Economic Behavior and Organization, 3, 367388. doi:10.1016/0167-2681(82)
  52. Hampton, A. N., Bossaerts, P., & O’Doherty, J. P. (2006). The role of the ventromedial prefrontal cortex in abstract state-based infer ence during decision making in humans. Journal of Neuroscience, 26, 83608367. doi:10.1523/JNEUROSCI.1010-06.2006
  53. Heerey, E. A., Matveeva, T. M., & Gold, J. M. (2011). Imagining the future: Degraded representations of future reward and events in schizophrenia. Journal of Abnormal Psychology, 120, 483489. doi:10.1037/a0021810
  54. Heerey, E. A., Robinson, B. M., McMahon, R. P., & Gold, J. M. (2007). Delay discounting in schizophrenia. Cognitive Neuropsychiatry, 12, 213221. doi:10.1080/13546800601005900
  55. Hertwig, R., Barron, G., Weber, E. U., & Erev, I. (2004). Decisions from experience and the effect of rare events in risky choice. Psychological Science, 15, 534539.
  56. Hinson, J. M., Jameson, T. L., & Whitney, P. (2003). Impulsive decision making and working memory. Journal of Experimental Psychology: Learning, Memory, and Cognition, 29, 298306. doi:10.1037/0278-7393.29.2.298
  57. Hoffman, M. D., & Gelman, A. (2014). The No-U-Turn Sampler: Adaptively setting path lengths in Hamiltonian Monte Carlo. Journal of Machine Learning Research, 15, 15931623.
  58. Huys, Q. J. M., Cools, R., Gölzer, M., Friedel, E., Heinz, A., Dolan, R. J., & Dayan, P. (2011). Disentangling the roles of approach, activation and valence in instrumental and Pavlovian responding. PLoS Computational Biology, 7, e1002028. doi:10.1371/journal.pcbi.1002028.t002
  59. Huys, Q. J. M., Maia, T. V., & Frank, M. J. (2016). Computational psychiatry as a bridge from neuroscience to clinical applications. Nature Neuroscience, 19, 404413. doi:10.1038/nn.4238
  60. Iglesias, S., Mathys, C., Brodersen, K. H., Kasper, L., Piccirelli, M., den Ouden, H. E. M., & Stephan, K. E. (2013). Hierarchical pre diction errors in midbrain and basal forebrain during sensory learning. Neuron, 80, 519530. doi:10.1016/j.neuron.2013.09.009
  61. Insel, T. R. (2014).The NIMH Research Domain Criteria (RDoc) Project: Precision medicine for psychiatry. American Journal of Psychiatry, 171, 395397. doi:10.1176/appi.ajp.2014.14020138
  62. Kable, J. W., & Glimcher, P. W. (2007). The neural correlates of subjective value during intertemporal choice. Nature Neuroscience, 10, 16251633. doi:10.1038/nn2007
  63. Kaelbling, L. P., Littman, M. L., & Moore, A. W. (1996). Reinforcement learning: A survey. Journal of Artificial Intelligence Research, 4, 237285.
  64. Kass, R. E., & Raftery, A. E. (1995). Bayes factors. Journal of the American Statistical Association, 90, 773795. doi:10.1080/01621459.1995.10476572
  65. Katahira, K. (2016). How hierarchical models improve point estimates of model parameters at the individual level. Journal of Mathematical Psychology, 73, 3758. doi:10.1016/j.jmp.2016.03.007
  66. Knill, D. C., & Pouget, A. (2004). The Bayesian brain: The role of uncertainty in neural coding and computation. Trends in Neurosciences, 27, 712719. doi:10.1016/j.tins.2004.10.007
  67. Koenigs, M., Young, L., Adolphs, R., Tranel, D., Cushman, F., Hauser, M., & Damasio, A. (2007). Damage to the prefrontal cortex increases utilitarian moral judgements. Nature, 446, 908911. doi:10.1038/nature05631
  68. Kruschke, J. K. (2010). What to believe: Bayesian methods for data analysis. Trends in Cognitive Sciences, 14, 293300. doi:10.1016/j.tics.2010.05.001
  69. Kruschke, J. K. (2011). Bayesian assessment of null values via parameter estimation and model comparison. Perspectives on Psychological Science, 6, 299312. doi:10.1177/1745691611406925
  70. Kruschke, J. K. (2014). Doing Bayesian data analysis: A tutorial with R, JAGS, and Stan (2nd ed.). Boston, MA: Academic Press.
  71. Lee, M. D. (2011). How cognitive modeling can benefit from hierarchical Bayesian models. Journal of Mathematical Psychology, 55, 17. doi:10.1016/j.jmp.2010.08.013
  72. Lewandowsky, S., Farrell, S. (2010). Computational modeling in cognition: Principles and practice. New York, NY: Sage.
  73. Li, J., Schiller, D., Schoenbaum, G., Phelps, E. A., & Daw, N. D. (2011). Differential roles of human striatum and amygdala in associative learning. Nature Neuroscience, 14, 12501252. doi:10.1038/nn.2904
  74. Luce, R. D. (1959). Individual choice behavior: A theoretical analysis. Mineola, NY: Dover. doi:10.1037/14396-000
  75. Lunn, D., Spiegelhalter, D., Thomas, A., & Best, N. (2009). The BUGS project: Evolution, critique and future directions. Statistics in Medicine, 28, 30493067. doi:10.1002/sim.3680
  76. Lunn, D. J., Thomas, A., Best, N., & Spiegelhalter, D. (2000). WinBUGS—a Bayesian modelling framework: Concepts, structure, and extensibility. Statistics and Computing, 10, 325337.
  77. MacKillop, J. (2013). Integrating behavioral economics and behavioral genetics: Delayed reward discounting as an endophenotype for addictive disorders. Journal of the Experimental Analysis of Behavior, 99, 1431. doi:10.1002/jeab.4
  78. Mars, R. B., Debener, S., Gladwin, T. E., Harrison, L. M., Haggard, P., Rothwell, J. C., & Bestmann, S. (2008). Trial-by-trial fluctuations in the event-related electroencephalogram reflect dynamic changes in the degree of surprise. Journal of Neuroscience, 28, 1253912545. doi:10.1523/JNEUROSCI.2925-08.2008
  79. Matzke, D., Love, J., Wiecki, T. V., Brown, S. D., Logan, G. D., & Wagenmakers, E.-J. (2013). Release the BEESTS: Bayesian estimation of ex-Gaussian stop-signal reaction time distributions. Frontiers in Psychology, 4, 918. doi:10.3389/fpsyg.2013.00918
  80. Mazur, J. E. (1987). An adjusting procedure for studying delayed reinforcement. In M. L. Commons, J. E. Mazur, J. A. Nevin, & H. Rachlin (Eds.), Quantitative analyses of behavior (Vol. 5, pp. 5573). Hillsdale, NJ: Erlbaum.
  81. Montague, P. R., Dolan, R. J., Friston, K. J., & Dayan, P. (2012). Computational psychiatry. Trends in Cognitive Sciences, 16, 7280. doi:10.1016/j.tics.2011.11.018
  82. Montague, P. R., & Lohrenz, T. (2007). To detect and correct: Norm violations and their enforcement. Neuron, 56, 1418. doi:10.1016/j.neuron.2007.09.020
  83. Murphy, F. C., Michael, A., Robbins, T. W., & Sahakian, B. J. (2003). Neuropsychological impairment in patients with major depressive disorder: The effects of feedback on task performance. Psychological Medicine, 33, 455467. doi:10.1017/S0033291702007018
  84. Myung, I.-J. (2003). Tutorial on maximum likelihood estimation. Journal of Mathematical Psychology, 47, 90100. doi:10.1016/S0022-2496(02)00028-7
  85. Myung, I.-J., & Pitt, M. A. (1997). Applying Occam’s razor in modeling cognition: A Bayesian approach. Psychonomic Bulletin & Review, 4, 7995. doi:10.3758/BF03210778
  86. O’Doherty, J., Critchley, H., Deichmann, R., & Dolan, R. J. (2003). Dissociating valence of outcome from behavioral control in human orbital and ventral prefrontal cortices. Journal of Neuroscience, 23, 79317939.
  87. O’Doherty, J., Dayan, P., Schultz, J., Deichmann, R., Friston, K., & Dolan, R. J. (2004). Dissociable roles of ventral and dorsal striatum in instrumental conditioning. Science, 304, 452454. doi:10.1126/science.1094285
  88. O’Doherty, J. P., Hampton, A., & Kim, H. (2007). Model-based fMRI and its application to reward learning and decision making. Annals of the New York Academy of Sciences, 1104, 3553. doi:10.1196/annals.1390.022
  89. Plummer, M. (2003, March). JAGS: A program for analysis of Bayes ian graphical models using Gibbs sampling. Article presented at the 3rd International Workshop on Distributed Statistical Computing (DSC 2003), Vienna, Austria.
  90. Rachlin, H., Raineri, A., & Cross, D. (1991). Subjective probability and delay. Journal of the Experimental Analysis of Behavior, 55, 233244. doi:10.1901/jeab.1991.55-233
  91. Raja Beharelle, A., Polania, R., Hare, T. A., & Ruff, C. C. (2015). Transcranial stimulation over frontopolar cortex elucidates the choice attributes and neural mechanisms used to resolve exploration–exploitation trade-offs. Journal of Neuroscience, 35, 1454414556. doi:10.1523/JNEUROSCI.2322-15.2015
  92. Rangel, A., Camerer, C. F., & Montague, P. R. (2008). A framework for studying the neurobiology of value-based decision making. Nature Reviews Neuroscience, 9, 545556. doi:10.1038/nrn2357
  93. Ratcliff, R., & Childers, R. (2015). Individual differences and fitting methods for the two-choice diffusion model of decision making. Decision, 2, 237279. doi:10.1037/dec0000030
  94. Rescorla, R. A., & Wagner, A. R. (1972). A theory of Pavlovian conditioning: Variations in the effectiveness of reinforcement and nonreinforcement. In A. H. Black & W. F. Prokasy (Eds.), Classical conditioning II: Current research and theory (pp. 6499). New York, NY: Appleton-Century-Crofts.
  95. Rolls, E. T., Hornak, J., Wade, D., & McGrath, J. (1994). Emotion-related learning in patients with social and emotional changes associated with frontal lobe damage. Journal of Neurology, Neurosurgery, and Psychiatry, 57, 15181524.
  96. Samuelson, P. A. (1937). A note on measurement of utility. Review of Economic Studies, 4, 155161. doi:10.2307/2967612
  97. Sanfey, A. G. (2003). The neural basis of economic decision-making in the Ultimatum Game. Science, 300, 17551758. doi:10.1126/science.1082976
  98. Shamosh, N. A., DeYoung, C. G., Green, A. E., Reis, D. L., Johnson, M. R., Conway, A. R. A., … Gray, J. R. (2008). Individual differences in delay discounting: Relation to intelligence, working memory, and anterior prefrontal cortex. Psychological Science, 19, 904911. doi:10.1111/j.1467-9280.2008.02175.x
  99. Shiffrin, R. M., Lee, M. D., Kim, W., & Wagenmakers, E.-J. (2008). A survey of model evaluation approaches with a tutorial on hierarchical Bayesian methods. Cognitive Science, 32, 12481284. doi:10.1080/03640210802414826
  100. Singmann, H., Brown, S., Gretton, M., Heathcote, A., Voss, A., Voss, J., & Terry, A. (2016). rtdists: Response time distributions [Software]. Retrieved from CRAN.R-project.org/package=rtdists
  101. Sokol-Hessner, P., Camerer, C. F., & Phelps, E. A. (2013). Emotion regulation reduces loss aversion and decreases amygdala responses to losses. Social Cognitive and Affective Neuroscience, 8, 341350. doi:10.1093/scan/nss002
  102. Sokol-Hessner, P., Hsu, M., Curley, N. G., Delgado, M. R., Camerer, C. F., & Phelps, E. A. (2009). Thinking like a trader selectively reduces individuals’ loss aversion. Proceedings of the National Academy of Sciences, 106, 50355040. doi:10.1073/pnas.0806761106
  103. Spiegelhalter, D. J., Best, N. G., Carlin, B. P., & van der Linde, A. (2002). Bayesian measures of model complexity and fit. Journal of the Royal Statistical Society: Series B, 64, 583639. doi:10.1111/1467-9868.00353
  104. Stan Development Team. (2016). RStan: The R interface to Stan. R package version 2.14.1. Retrieved from https://cran.r-project.org/web/packages/rstan/citation.html
  105. Steingroever, H., Wetzels, R., & Wagenmakers, E.-J. (2014). Absolute performance of reinforcement-learning models for the Iowa Gambling Task. Decision, 1, 161183. doi:10.1037/dec0000005
  106. Stephan, K. E., Bach, D. R., Fletcher, P. C., Flint, J., Frank, M. J., Friston, K. J., … Breakspear, M. (2016). Charting the land scape of priority problems in psychiatry, part 1: Classification and diagnosis. Lancet: Psychiatry, 3, 7783. doi:10.1016/S2215-0366(15)00361-2
  107. Stephan, K. E., Binder, E. B., Breakspear, M., Dayan, P., Johnstone, E. C., Meyer-Lindenberg, A., … Friston, K. J. (2016). Charting the landscape of priority problems in psychiatry, part 2: Pathogenesis and aetiology. Lancet: Psychiatry, 3, 8490. doi:10.1016/S2215-0366(15)00360-0
  108. Stephan, K. E., Iglesias, S., Heinzle, J., & Diaconescu, A. O. (2015). Translational perspectives for computational neuroimaging. Neuron, 87, 716732. doi:10.1016/j.neuron.2015.07.008
  109. Swainson, R., Rogers, R. D., Sahakian, B. J., Summers, B. A., Polkey, C. E., & Robbins, T. W. (2000). Probabilistic learning and reversal deficits in patients with Parkinson’s disease or frontal or temporal lobe lesions: Possible adverse effects of dopaminergic medication. Neuropsychologia, 38, 596612.
  110. Thaler, R. H. (1988). Anomalies: The ultimatum game. Journal of Economic Perspectives, 2, 195206. doi:10.2307/1942788
  111. Tom, S. M., Fox, C. R., Trepel, C., & Poldrack, R. A. (2007). The neural basis of loss aversion in decision-making under risk. Science, 315, 515518. doi:10.1126/science.1134239
  112. Vassileva, J., Ahn, W.-Y., Weber, K. M., Busemeyer, J. R., Stout, J. C., Gonzalez, R., & Cohen, M. H. (2013). Computational modeling reveals distinct effects of HIV and history of drug use on decision-making processes in women. PLoS ONE, 8, e68962. doi:10.1371/journal.pone.0068962
  113. Vassileva, J., Gonzalez, R., Bechara, A., & Martin, E. M. (2007). Are all drug addicts impulsive? Effects of antisociality and extent of multidrug use on cognitive and motor impulsivity. Addictive Behaviors, 32, 30713076. doi:10.1016/j.addbeh.2007.04.017
  114. Vehtari, A., Gelman, A., & Gabry, J. (2016). Practical Bayesian model evaluation using leave-one-out cross-validation andWAIC. arXiv:1507.04544
  115. Vincent, B. T. (2016). Hierarchical Bayesian estimation and hypothesis testing for delay discounting tasks. Behavior Research Methods, 48, 16081620. doi:10.3758/s13428-015-0672-2
  116. Wabersich, D., & Vandekerckhove, J. (2014). The RWiener package: An R package providing distribution functions for the Wiener diffusion model. R Journal, 6, 4956. Retrieved from https://journal.r-project.org/archive/2014-1/vandekerckhove-wabersich.pdf
  117. Wagenmakers, E.-J. (2007). A practical solution to the pervasive problems of p values. Psychonomic Bulletin & Review, 14, 779804. doi:10.3758/BF03194105
  118. Waltz, J. A., & Gold, J. M. (2007). Probabilistic reversal learning impairments in schizophrenia: Further evidence of orbitofrontal dysfunction. Schizophrenia Research, 93, 296303. doi:10.1016/j.schres.2007.03.010
  119. Wang, X.-J., & Krystal, J. H. (2014). Computational psychiatry. Neuron, 84, 638654. doi:10.1016/j.neuron.2014.10.018
  120. Wetzels, R., Vandekerckhove, J., & Tuerlinckx, F. (2010). Bayesian parameter estimation in the Expectancy Valence model of the Iowa Gambling Task. Journal of Mathematical Psychology, 54, 1427. doi:10.1016/j.jmp.2008.12.001
  121. Wiecki, T. V., Poland, J., & Frank, M. J. (2015). Model-based cognitive neuroscience approaches to computational psychiatry: Clustering and classification. Clinical Psychological Science, 3, 378399. doi:10.1177/2167702614565359
  122. Wiecki, T. V., Sofer, I., & Frank, M. J. (2013). HDDM: Hierarchical Bayesian estimation of the drift-diffusion model in Python. Frontiers in Neuroinformatics, 7, 14. doi:10.3389/fninf.2013.00014
  123. Worthy, D. A., Pang, B., & Byrne, K. A. (2013). Decomposing the roles of perseveration and expected value representation in models of the Iowa gambling task. Frontiers in Psychology, 4, 640. doi:10.3389/fpsyg.2013.00640
  124. Xiang, T., Lohrenz, T., & Montague, P. R. (2013). Computational substrates of norms and their violations during social exchange. Journal of Neuroscience, 33, 10991108. doi:10.1523/JNEUROSCI.1642-12.2013
Language: English
Submitted on: Jul 20, 2016
Accepted on: Mar 6, 2017
Published on: Oct 1, 2017
Published by: MIT Press
In partnership with: Paradigm Publishing Services
Publication frequency: 1 issue per year

© 2017 Woo-Young Ahn, Nathaniel Haines, Lei Zhang, published by MIT Press
This work is licensed under the Creative Commons Attribution 4.0 License.