References
- 1Archer, E., Park, I., & Pillow, J. (2013). Bayesian and quasi-Bayesian estimators for mutual information from discrete data. Entropy, 15, 1738–1755. DOI: 10.3390/e15051738
- 2Arimoto, S. (1972). An algorithm for computing the capacity of arbitrary discrete memoryless channels. IEEE Transactions on Information Theory, 18, 14–20. DOI: 10.1109/TIT.1972.1054753
- 3Bar-Gad, I., Morris, G., & Bergman, H. (2003). Information processing, dimensionality reduction and reinforcement learning in the basal ganglia. Progress in Neurobiology, 71, 439–473. DOI: 10.1016/j.pneurobio.2003.12.001
- 4Bates, C. J., & Jacobs, R. A. (2020). Efficient data compression in perception and perceptual memory. Psychological Review, 127, 891–917. DOI: 10.1037/rev0000197
- 5Berger, T. (1971). Rate Distortion Theory: A Mathematical Basis for Data Compression. NJ: Prentice-Hall.
- 6Blahut, R. (1972). Computation of channel capacity and rate-distortion functions. IEEE transactions on Information Theory, 18, 460–473. DOI: 10.1109/TIT.1972.1054855
- 7Brady, T., Konkle, T., & Alvarez, G. (2009). Compression in visual working memory: Using statistical regularities to form more efficient memory representations. Journal of Experimental Psychology: General, 138, 487–502. DOI: 10.1037/a0016797
- 8Collins, A. G. (2018). The tortoise and the hare: Interactions between reinforcement learning and working memory. Journal of Cognitive Neuroscience, 30, 1422–1432. DOI: 10.1162/jocn_a_01238
- 9Collins, A. G., Albrecht, M. A., Waltz, J. A., Gold, J. M., & Frank, M. J. (2017a). Interactions among working memory, reinforcement learning, and effort in value-based choice: A new paradigm and selective deficits in schizophrenia. Biological Psychiatry, 82, 431–439. DOI: 10.1016/j.biopsych.2017.05.017
- 10Collins, A. G., Brown, J. K., Gold, J. M., Waltz, J. A., & Frank, M. J. (2014). Working memory contributions to reinforcement learning impairments in schizophrenia. Journal of Neuroscience, 34, 13747–13756. DOI: 10.1523/JNEUROSCI.0989-14.2014
- 11Collins, A. G., Ciullo, B., Frank, M. J., & Badre, D. (2017b). Working memory load strengthens reward prediction errors. Journal of Neuroscience, 37, 4332–4342. DOI: 10.1523/JNEUROSCI.2700-16.2017
- 12Collins, A. G., & Frank, M. J. (2012). How much of reinforcement learning is working memory, not reinforcement learning? A behavioral, computational, and neurogenetic analysis. European Journal of Neuroscience, 35, 1024–1035. DOI: 10.1111/j.1460-9568.2011.07980.x
- 13Collins, A. G., & Frank, M. J. (2018). Within-and across-trial dynamics of human eeg reveal cooperative interplay between reinforcement learning and working memory. Proceedings of the National Academy of Sciences, 115, 2502–2507. DOI: 10.1073/pnas.1720963115
- 14Culbreth, A., Westbrook, A., & Barch, D. (2016). Negative symptoms are associated with an increased subjective cost of cognitive effort. Journal of Abnormal Psychology, 125, 528–536. DOI: 10.1037/abn0000153
- 15Culbreth, A. J., Moran, E. K., & Barch, D. M. (2018). Effort-based decision-making in schizophrenia. Current Opinion in Behavioral Sciences, 22, 1–6. DOI: 10.1016/j.cobeha.2017.12.003
- 16Dezfouli, A., & Balleine, B. W. (2012). Habits, action sequences and reinforcement learning. European Journal of Neuroscience, 35, 1036–1051. DOI: 10.1111/j.1460-9568.2012.08050.x
- 17Dowd, E. C., Frank, M. J., Collins, A., Gold, J. M., & Barch, D. M. (2016). Probabilistic reinforcement learning in patients with schizophrenia: relationships to anhedonia and avolition. Biological Psychiatry: Cognitive Neuroscience and Neuroimaging, 1, 460–473. DOI: 10.1016/j.bpsc.2016.05.005
- 18Fortgang, R., Srihari, V., & Cannon, T. (2020). Cognitive effort and amotivation in first-episode psychosis. Journal of Abnormal Psychology, 129, 422–431. DOI: 10.1037/abn0000509
- 19Fox, R., Pakman, A., & Tishby, N. (2016). Taming the noise in reinforcement learning via soft updates. In Proceedings of the Thirty-Second Conference on Uncertainty in Artificial Intelligence (pp. 202–211).
- 20Gershman, S. J. (2020). Origin of perseveration in the trade-off between reward and complexity. Cognition, 204,
104394 . DOI: 10.1016/j.cognition.2020.104394 - 21Gold, J. M., Kool, W., Botvinick, M. M., Hubzin, L., August, S., & Waltz, J. A. (2015). Cognitive effort avoidance and detection in people with schizophrenia. Cognitive, Affective, & Behavioral Neuroscience, 15, 145–154. DOI: 10.3758/s13415-014-0308-5
- 22Grau-Moya, J., Leibfried, F., & Vrancx, P. (2018). Soft q-learning with mutual-information regularization. In International Conference on Learning Representations.
- 23Haarnoja, T., Zhou, A., Abbeel, P., & Levine, S. (2018). Soft actor-critic: Off-policy maximum entropy deep reinforcement learning with a stochastic actor. Volume 80 of Proceedings of Machine Learning Research (pp. 1861–1870). Stockholm Sweden:
Stockholmsmässan . PMLR. - 24Hernaus, D., Xu, Z., Brown, E., Ruiz, R., Frank, M., Gold, J., & Waltz, J. (2018). Motivational deficits in schizophrenia relate to abnormalities in cortical learning rate signals. Cognitive, Affective, & Behavioral Neuroscience, 18, 1338–1351. DOI: 10.3758/s13415-018-0643-z
- 25Horan, W. P., Reddy, L. F., Barch, D. M., Buchanan, R. W., Dunayevich, E., Gold, J. M., Marder, S. R., Wynn, J. K., Young, J. W., & Green, M. F. (2015). Effort-based decision-making paradigms for clinical trials in schizophrenia: Part 2—external validity and correlates. Schizophrenia Bulletin, 41(5), 1055–1065. DOI: 10.1093/schbul/sbv090
- 26Hutter, M. (2002). Distribution of mutual information. In Advances in Neural Information Processing Systems (pp. 399–406).
- 27Insel, C., Reinen, J., Weber, J., Wager, T. D., Jarskog, L. F., Shohamy, D., & Smith, E. E. (2014). Antipsychotic dose modulates behavioral and neural responses to feedback during reinforcement learning in schizophrenia. Cognitive, Affective, & Behavioral Neuroscience, 14, 189–201. DOI: 10.3758/s13415-014-0261-3
- 28Joel, D., Niv, Y., & Ruppin, E. (2002). Actor–critic models of the basal ganglia: New anatomical and computational perspectives. Neural Networks, 15, 535–547. DOI: 10.1016/S0893-6080(02)00047-3
- 29Konda, V. R., & Tsitsiklis, J. N. (2000). Actor-critic algorithms. In Advances in Neural Information Processing Systems (pp. 1008–1014).
- 30Kool, W., McGuire, J., Rosen, Z., & Botvinick, M. (2010). Decision making and the avoidance of cognitive demand. Journal of Experimental Psychology: General, 139, 665–682. DOI: 10.1037/a0020198
- 31Lee, J., & Park, S. (2005). Working memory impairments in schizophrenia: A meta-analysis. Journal of Abnormal Psychology, 114, 599–611. DOI: 10.1037/0021-843X.114.4.599
- 32Ma, W. J., Husain, M., & Bays, P. M. (2014). Changing concepts of working memory. Nature Neuroscience, 17(3), 347–356. DOI: 10.1038/nn.3655
- 33Malloy, T., Sims, C. R., Klinger, T., Liu, M., Riemer, M., & Tesauro, G. (2020). Deep RL with information constrained policies: Generalization in continuous control. arXiv preprint arXiv:2010.04646.
- 34Miller, G. (1956). The magical number seven, plus or minus two: some limits on our capacity for processing information. Psychological Review, 63, 81–97. DOI: 10.1037/h0043158
- 35Parush, N., Tishby, N., & Bergman, H. (2011). Dopaminergic balance between reward maximization and policy complexity. Frontiers in Systems Neuroscience, 5. DOI: 10.3389/fnsys.2011.00022
- 36Patzelt, E. H., Kool, W., Millner, A. J., & Gershman, S. J. (2019). The transdiagnostic structure of mental effort avoidance. Scientific Reports, 9, 1–10. DOI: 10.1038/s41598-018-37802-1
- 37Reddy, L. F., Horan, W. P., Barch, D. M., Buchanan, R. W., Dunayevich, E., Gold, J. M., Lyons, N., Marder, S. R., Treadway, M. T., Wynn, J. K., et al. (2015). Effort-based decisionmaking paradigms for clinical trials in schizophrenia: part 1—psychometric characteristics of 5 paradigms. Schizophrenia Bulletin, 41, 1045–1054. DOI: 10.1093/schbul/sbv089
- 38Rigoux, L., Stephan, K. E., Friston, K. J., & Daunizeau, J. (2014). Bayesian model selection for group studies—revisited. NeuroImage, 84, 971–985. DOI: 10.1016/j.neuroimage.2013.08.065
- 39Robbins, H., & Monro, S. (1951). A stochastic approximation method. The Annals of Mathematical Statistics, (pp. 400–407). DOI: 10.1214/aoms/1177729586
- 40Shannon, C. E. (1948). A mathematical theory of communication. Bell System Technical Journal, 27, 379–423. DOI: 10.1002/j.1538-7305.1948.tb01338.x
- 41Sims, C., Jacobs, R., & Knill, D. (2012). An ideal observer analysis of visual working memory. Psychological Review, 119, 807–830. DOI: 10.1037/a0029856
- 42Sims, C. R. (2016). Rate-distortion theory and human perception. Cognition, 152, 181–198. DOI: 10.1016/j.cognition.2016.03.020
- 43Still, S., & Precup, D. (2012). An information-theoretic approach to curiosity-driven reinforcement learning. Theory in Biosciences, 131, 139–148. DOI: 10.1007/s12064-011-0142-z
- 44Sutton, R. S., & Barto, A. G. (2018). Reinforcement Learning: An Introduction. MIT Press.
- 45Tishby, N., & Polani, D. (2011).
Information theory of decisions and actions . In Perception-action cycle (pp. 601–636). Springer. DOI: 10.1007/978-1-4419-1452-1_19 - 46Tomov, M. S., Yagati, S., Kumar, A., Yang, W., & Gershman, S. J. (2020). Discovery of hierarchical representations for efficient planning. PLoS Computational Biology, 16,
e1007594 . DOI: 10.1371/journal.pcbi.1007594 - 47Weickert, T. W., Goldberg, T. E., Egan, M. F., Apud, J. A., Meeter, M., Myers, C. E., Gluck, M. A., & Weinberger, D. R. (2010). Relative risk of probabilistic category learning deficits in patients with schizophrenia and their siblings. Biological Psychiatry, 67, 948–955. DOI: 10.1016/j.biopsych.2009.12.027
- 48Wolf, D. H., Satterthwaite, T. D., Kantrowitz, J. J., Katchmar, N., Vandekar, L., Elliott, M. A., & Ruparel, K. (2014). Amotivation in schizophrenia: integrated assessment with behavioral, clinical, and imaging measures. Schizophrenia Bulletin, 40, 1328–1337. DOI: 10.1093/schbul/sbu026