References
- 1Agrawal, M., Mattar, M. G., Cohen, J. D., & Daw, N. D. (2022). The temporal dynamics of opportunity costs: A normative account of cognitive fatigue and boredom. Psychological Review, 129(3), 564–585. 10.1037/rev0000309
- 2Alméras, C., Chambon, V., & Wyart, V. (2021). Competing cognitive pressures on human exploration in the absence of trade-off with exploitation. Preprint. 10.31234/osf.io/9qpuz
- 3Antinori, A., Carter, O. L., & Smillie, L. D. (2017). Seeing it both ways: Openness to experience and binocular rivalry suppression. Journal of Research in Personality, 68, 15–22. 10.1016/j.jrp.2017.03.005
- 4Ariely, D. (2000). Controlling the Information Flow: Effects on Consumers’ Decision Making and Preferences. Journal of Consumer Research, 27(2), 233–248. 10.1086/314322
- 5Attneave, F. (1955). Symmetry, information, and memory for patterns. The American journal of psychology, 68(2), 209–222. 10.2307/1418892
- 6Averbeck, B. B. (2015). Theory of Choice in Bandit, Information Sampling and Foraging Tasks. PLOS Computational Biology, 11(3),
e1004164 . 10.1371/journal.pcbi.1004164 - 7Beck, J. M., Ma, W. J., Pitkow, X., Latham, P. E., & Pouget, A. (2012). Not Noisy, Just Wrong: The Role of Suboptimal Inference in Behavioral Variability. Neuron, 74(1), 30–39. 10.1016/j.neuron.2012.03.016
- 8Berger-Tal, O., Nathan, J., Meron, E., & Saltz, D. (2014). The Exploration-Exploitation Dilemma: A Multidisciplinary Framework. PLOS ONE, 9(4),
e95693 . 10.1371/journal.pone.0095693 - 9Blanchard, T. C., Hayden, B. Y., & Bromberg-Martin, E. S. (2015). Orbitofrontal Cortex Uses Distinct Codes for Different Choice Attributes in Decisions Motivated by Curiosity. Neuron, 85(3), 602–614. 10.1016/j.neuron.2014.12.050
- 10Bromberg-Martin, E. S., & Hikosaka, O. (2009). Midbrain Dopamine Neurons Signal Preference for Advance Information about Upcoming Rewards. Neuron, 63(1), 119–126. 10.1016/j.neuron.2009.06.009
- 11Bromberg-Martin, E. S., & Monosov, I. E. (2020). Neural circuitry of information seeking. Current Opinion in Behavioral Sciences, 35, 62–70. 10.1016/j.cobeha.2020.07.006
- 12Bussche, E. V. den, Alves, M., Murray, Y. P. J., & Hughes, G. (2020). The effect of cognitive effort on the sense of agency. PLOS ONE, 15(8),
e0236809 . 10.1371/journal.pone.0236809 - 13Carstensen, L. L., Isaacowitz, D. M., & Charles, S. T. (1999). Taking time seriously: A theory of socioemotional selectivity. American Psychologist, 54(3), 165–181. 10.1037/0003-066X.54.3.165
- 14Cheadle, S., Wyart, V., Tsetsos, K., Myers, N., de Gardelle, V., Herce Castañón, S., & Summerfield, C. (2014). Adaptive Gain Control during Human Perceptual Choice. Neuron, 81(6), 1429–1441. 10.1016/j.neuron.2014.01.020
- 15Chronopoulos, M., De Reyck, B., & Siddiqui, A. (2011). Optimal investment under operational flexibility, risk aversion, and uncertainty. European Journal of Operational Research, 213(1), 221–237. 10.1016/j.ejor.2011.03.007
- 16Cogliati Dezza, I., Yu, A. J., Cleeremans, A., & Alexander, W. (2017). Learning the value of information and reward over time when solving exploration-exploitation problems. Scientific Reports, 7(1), 16919. 10.1038/s41598-017-17237-w
- 17Cohen, J. D., McClure, S. M., & Yu, A. J. (2007). Should I stay or should I go? How the human brain manages the trade-off between exploitation and exploration. Philosophical Transactions of the Royal Society B: Biological Sciences, 362(1481), 933–942. 10.1098/rstb.2007.2098
- 18Dajani, D. R., & Uddin, L. Q. (2015). Demystifying cognitive flexibility: Implications for clinical and developmental neuroscience. Trends in Neurosciences, 38(9), 571–578. 10.1016/j.tins.2015.07.003
- 19Danckert, J. (2019).
Boredom: Managing the Delicate Balance Between Exploration and Exploitation . En J. Ros Velasco (Ed.), Boredom Is in Your Mind: A Shared Psychological-Philosophical Approach (pp. 37–53). Springer International Publishing. 10.1007/978-3-030-26395-9_3 - 20Daw, N. D., O’Doherty, J. P., Dayan, P., Seymour, B., & Dolan, R. J. (2006). Cortical substrates for exploratory decisions in humans. Nature, 441(7095), 876–879. 10.1038/nature04766
- 21De Lange, F. P., Jensen, O., & Dehaene, S. (2010). Accumulation of Evidence during Sequential Decision Making: The Importance of Top–Down Factors. The Journal of Neuroscience, 30(2), 731–738. 10.1523/JNEUROSCI.4080-09.2010
- 22Dhawale, A. K., Smith, M. A., & Ölveczky, B. P. (2017). The Role of Variability in Motor Learning. Annual Review of Neuroscience, 40(1), 479–498. 10.1146/annurev-neuro-072116-031548
- 23Dow, J., & Werlang, S. R. da C. (1992). Uncertainty Aversion, Risk Aversion, and the Optimal Choice of Portfolio. Econometrica, 60(1), 197–204. 10.2307/2951685
- 24Drugowitsch, J., Moreno-Bote, R., Churchland, A. K., Shadlen, M. N., & Pouget, A. (2012). The Cost of Accumulating Evidence in Perceptual Decision Making. Journal of Neuroscience, 32(11), 3612–3628. 10.1523/JNEUROSCI.4010-11.2012
- 25Evans, D. A., Stempel, A. V., Vale, R., & Branco, T. (2019). Cognitive Control of Escape Behaviour. Trends in Cognitive Sciences, 23(4), 334–348. 10.1016/J.TICS.2019.01.012
- 26Faisal, A. A., Selen, L. P. J., & Wolpert, D. M. (2008). Noise in the nervous system. Nature Reviews Neuroscience, 9(4), Article 4. 10.1038/nrn2258
- 27Freeman, S., Eddy, S. L., McDonough, M., Smith, M. K., Okoroafor, N., Jordt, H., & Wenderoth, M. P. (2014). Active learning increases student performance in science, engineering, and mathematics. Proceedings of the National Academy of Sciences, 111(23), 8410–8415. 10.1073/pnas.1319030111
- 28Garrett, D. D., Samanez-Larkin, G. R., MacDonald, S. W. S., Lindenberger, U., McIntosh, A. R., & Grady, C. L. (2013). Moment-to-moment brain signal variability: A next frontier in human brain mapping? Neuroscience & Biobehavioral Reviews, 37(4), 610–624. 10.1016/j.neubiorev.2013.02.015
- 29Geana, A., & Daw, N. (2016). Boredom, Information-Seeking and Exploration. Proceedings of the 38th Annual Meeting of the Cognitive Science Society, 1751–1756.
- 30Gershman, S. J. (2019). Uncertainty and exploration. Decision, 6(3), 277–286. 10.1037/dec0000101
- 31Gigante, G., Mattia, M., Braun, J., & Del Giudice, P. (2009). Bistable Perception Modeled as Competing Stochastic Integrations at Two Levels. PLOS Computational Biology, 5(7). 10.1371/journal.pcbi.1000430
- 32Glaze, C. M., Kable, J. W., & Gold, J. I. (2015). Normative evidence accumulation in unpredictable environments. eLife, 4,
e08825 . 10.7554/eLife.08825 - 33Gottlieb, J. (2012). Attention, Learning, and the Value of Information. Neuron, 76(2), 281–295. 10.1016/j.neuron.2012.09.034
- 34Green, L., & Myerson, J. (2004). A Discounting Framework for Choice With Delayed and Probabilistic Rewards. Psychological Bulletin, 130(5), 769–792. 10.1037/0033-2909.130.5.769
- 35Green, R. F. (1984). Stopping Rules for Optimal Foragers. The American Naturalist, 123(1), 30–43. 10.1086/284184
- 36Hayden, B. Y., Pearson, J. M., & Platt, M. L. (2009).
Fictive reward signals in the anterior cingulate cortex . Science (New York, N.Y.), 324(5929), 948–950. 10.1126/science.1168488 - 37Hon, N., & Yeo, N. (2021). Having a sense of agency can improve memory. Psychonomic Bulletin & Review, 28(3), 946–952. 10.3758/s13423-020-01849-x
- 38Hunt, L. T., Daw, N. D., Kaanders, P., MacIver, M. A., Mugan, U., Procyk, E., Redish, A. D., Russo, E., Scholl, J., Stachenfeld, K., Wilson, C. R. E., & Kolling, N. (2021). Formalizing planning and information search in naturalistic decision-making. Nature Neuroscience, 24(8), 1051–1064. 10.1038/s41593-021-00866-w
- 39Inzlicht, M., Shenhav, A., & Olivola, C. Y. (2018). The Effort Paradox: Effort Is Both Costly and Valued. Trends in Cognitive Sciences, 22(4), 337–349. 10.1016/j.tics.2018.01.007
- 40Kelly, C. A., & Sharot, T. (2021). Individual differences in information-seeking. Nature Communications, 12(1), 7062. 10.1038/s41467-021-27046-5
- 41Kloosterman, N. A., De Gee, J. W., Bergner, M. W., Lindenberger, U., Garrett, D. D., & Fahrenfort, J. J. (2019). Humans strategically shift decision bias by flexibly adjusting sensory evidence accumulation. eLife, 8. 10.7554/eLife.37321
- 42Kool, W., McGuire, J. T., Rosen, Z. B., & Botvinick, M. M. (2010). Decision Making and the Avoidance of Cognitive Demand. Journal of experimental psychology. General, 139(4), 665–682. 10.1037/a0020198
- 43Kurzban, R. (2016). The sense of effort. Current Opinion in Psychology, 7, 67–70. 10.1016/j.copsyc.2015.08.003
- 44Kurzban, R., Duckworth, A., Kable, J. W., & Myers, J. (2013). An opportunity cost model of subjective effort and task performance. Behavioral and Brain Sciences, 36(6), 661–679. 10.1017/S0140525X12003196
- 45Lee, J. K., Rouault, M., & Wyart, V. (2023). Adaptive tuning of human learning and choice variability to unexpected uncertainty. Science Advances, 9(13),
eadd0501 . 10.1126/sciadv.add0501 - 46Loewenstein, G., & Molnar, A. (2018). The renaissance of belief-based utility in economics. Nature Human Behaviour, 2(3), Article 3. 10.1038/s41562-018-0301-z
- 47Markant, D. B., & Gureckis, T. M. (2014). Is it better to select or to receive? Learning via active and passive hypothesis testing. Journal of Experimental Psychology: General, 143(1), 94–122. 10.1037/a0032108
- 48Markant, D. B., Ruggeri, A., Gureckis, T. M., & Xu, F. (2016). Enhanced Memory as a Common Effect of Active Learning. Mind, Brain, and Education, 10(3), 142–152. 10.1111/mbe.12117
- 49Markowitz, J. E., Gillis, W. F., Jay, M., Wood, J., Harris, R. W., Cieszkowski, R., Scott, R., Brann, D., Koveal, D., Kula, T., Weinreb, C., Osman, M., Pinto, S., Uchiida, N., Linderman, S., Sabatini, B., & Datta S. (2023). Spontaneous behaviour is structured by reinforcement without explicit reward. Nature, 614(7946). 10.1038/s41586-022-05611-2
- 50Milyavskaya, M., Galla, B. M., Inzlicht, M., & Duckworth, A. L. (2021). More Effort, Less Fatigue: The Role of Interest in Increasing Effort and Reducing Mental Fatigue. Frontiers in Psychology, 12.
https://www.frontiersin.org/articles/10.3389/fpsyg.2021.755858 - 51Mooneyham, B. W., & Schooler, J. W. (2013). The costs and benefits of mind-wandering: A review. Canadian Journal of Experimental Psychology / Revue canadienne de psychologie expérimentale, 67(1), 11–18. 10.1037/a0031569
- 52Moreno-Bote, R., Ramírez-Ruiz, J., Drugowitsch, J., & Hayden, B. Y. (2020). Heuristics and optimal solutions to the breadth:depth dilemma. Proceedings of the National Academy of Sciences of the United States of America, 117(33), 19799–19808. 10.1073/PNAS.2004929117
- 53Ölveczky, B. P., Andalman, A. S., & Fee, M. S. (2005). Vocal Experimentation in the Juvenile Songbird Requires a Basal Ganglia Circuit. PLOS Biology, 3(5),
e153 . 10.1371/journal.pbio.0030153 - 54Ölveczky, B. P., Otchy, T. M., Goldberg, J. H., Aronov, D., & Fee, M. S. (2011). Changes in the neural control of a complex motor sequence during learning. Journal of Neurophysiology, 106(1), 386–397. 10.1152/jn.00018.2011
- 55Petitet, P., Attaallah, B., Manohar, S. G., & Husain, M. (2021). The computational cost of active information sampling before decision-making under uncertainty. Nature Human Behaviour, 5(7), 935–946. 10.1038/s41562-021-01116-6
- 56Pisupati, S., Chartarifsky-Lynn, L., Khanal, A., & Churchland, A.K. (2021). Lapses in perceptual decisions reflect exploration. eLife, 10. 10.7554/eLife.55490
- 57Rahnev, D., & Denison, R. N. (2018). Suboptimality in perceptual decision making. Behavioral and Brain Sciences, 41,
e223 . 10.1017/S0140525X18000936 - 58Ramírez-Ruiz, J., Grytskyy, D., Mastrogiuseppe, C., Habib, Y., & Moreno-Bote, R. (2024). Complex behavior from intrinsic motivation to occupy future action-state path space. Nature Communications, 15(1), 6368. 10.1038/s41467-024-49711-1
- 59Renart, A., & Machens, C. K. (2014). Variability in neural activity and behavior. Current Opinion in Neurobiology, 25, 211–220. 10.1016/j.conb.2014.02.013
- 60Rényi, A. (1961). On measures of entropy and information. Mathematical Institute Hungarian Academy Of Sciences.
- 61Risko, E. F., Anderson, N. C., Lanthier, S., & Kingstone, A. (2012). Curious eyes: Individual differences in personality predict eye movement behavior in scene-viewing. Cognition, 122(1), 86–90. 10.1016/j.cognition.2011.08.014
- 62Sang, K., Todd, P. M., Goldstone, R. L., & Hills, T. T. (2020). Simple Threshold Rules Solve Explore/Exploit Trade-offs in a Resource Accumulation Search Task. Cognitive Science, 44(2),
e12817 . 10.1111/cogs.12817 - 63Schulz, E., Bhui, R., Love, B. C., Brier, B., Todd, M. T., & Gershman, S. J. (2019). Structured, uncertainty-driven exploration in real-world consumer choice. Proceedings of the National Academy of Sciences of the United States of America, 116(28), 13903–13908. 10.1073/pnas.1821028116
- 64Schwartenbeck, P., FitzGerald, T. H. B., Mathys, C., Dolan, R., Kronbichler, M., & Friston, K. (2015). Evidence for surprise minimization over value maximization in choice behavior. Scientific Reports, 5(1), Article 1. 10.1038/srep16575
- 65Seymour, B., & McClure, S. M. (2008). Anchors, scales and the relative coding of value in the brain. Current Opinion in Neurobiology, 18(2), 173–178. 10.1016/j.conb.2008.07.010
- 66Sharot, T., & Sunstein, C. R. (2020). How people decide what they want to know. Nature Human Behaviour, 4(1), 14–19. 10.1038/s41562-019-0793-1
- 67Smith, P. L., & Ratcliff, R. (2004). Psychology and neurobiology of simple decisions. Trends in Neurosciences, 27(3), 161–168. 10.1016/j.tins.2004.01.006
- 68Stephens, D. W., & Krebs, J. R. (1986). Foraging Theory. 10.1515/9780691206790
- 69Sternad, D. (2018). It’s not (only) the mean that matters: Variability, noise and exploration in skill learning. Current Opinion in Behavioral Sciences, 20, 183–195. 10.1016/j.cobeha.2018.01.004
- 70Tulloch, A. I. T., Maloney, R. F., Joseph, L. N., Bennett, J. R., Di Fonzo, M. M. I., Probert, W. J. M., O’Connor, S. M., Densem, J. P., & Possingham, H. P. (2015). Effect of risk aversion on prioritizing conservation projects. Conservation Biology, 29(2), 513–524. 10.1111/cobi.12386
- 71Uddin, L. Q. (2021). Cognitive and behavioural flexibility: Neural mechanisms and clinical considerations. Nature Reviews Neuroscience, 22(3), Article 3. 10.1038/s41583-021-00428-w
- 72van der Wel, R. P. R. D., Sebanz, N., & Knoblich, G. (2012). The sense of agency during skill learning in individuals and dyads. Consciousness and Cognition, 21(3), 1267–1279. 10.1016/j.concog.2012.04.001
- 73Vidal, A., Soto-Faraco, S., & Moreno Bote, R. (2022). Balance between breadth and depth in human many-alternative decisions. eLife, 11. 10.7554/ELIFE.76985
- 74von Neumann, J., & Morgenstern, O. (1944). Theory of Games and Economic Behavior. Princeton University Press.
- 75Voss, J. L., Gonsalves, B. D., Federmeier, K. D., Tranel, D., & Cohen, N. J. (2011). Hippocampal brain-network coordination during volitional exploratory behavior enhances learning. Nature Neuroscience, 14(1), Article 1. 10.1038/nn.2693
- 76Waschke, L., Kloosterman, N. A., Obleser, J., & Garrett, D. D. (2021). Behavior needs neural variability. Neuron, 109(5), 751–766. 10.1016/j.neuron.2021.01.023
- 77Weiss, A., Chambon, V., Lee, J. K., Drugowitsch, J., & Wyart, V. (2021). Interacting with volatile environments stabilizes hidden-state inference and its brain signatures. Nature Communications, 12(1), Article 1. 10.1038/s41467-021-22396-6
- 78Wilson, R. C., Bonawitz, E., Costa, V. D., & Ebitz, R. B. (2021). Balancing exploration and exploitation with information and randomization. Current Opinion in Behavioral Sciences, 38, 49–56. 10.1016/j.cobeha.2020.10.001
- 79Wilson, R. C., Geana, A., White, J. M., Ludvig, E. A., & Cohen, J. D. (2014). Humans use directed and random exploration to solve the explore-exploit dilemma. Journal of Experimental Psychology: General, 143(6), 2074–2081. 10.1037/A0038199
- 80Wu, C. M., Schulz, E., Speekenbrink, M., Nelson, J. D., & Meder, B. (2018). Generalization guides human exploration in vast decision spaces. Nature Human Behaviour, 2(12), Article 12. 10.1038/s41562-018-0467-4
- 81Wu, R., Ferguson, A. M., & Inzlicht, M. (2023). Do humans prefer cognitive effort over doing nothing? Journal of Experimental Psychology: General, 152(4), 1069–1079. 10.1037/xge0001320
- 82Wyart, V., de Gardelle, V., Scholl, J., & Summerfield, C. (2012). Rhythmic Fluctuations in Evidence Accumulation during Decision Making in the Human Brain. Neuron, 76(4), 847–858. 10.1016/j.neuron.2012.09.015
- 83Wyart, V., & Koechlin, E. (2016). Choice variability and suboptimality in uncertain environments. Current Opinion in Behavioral Sciences, 11, 109–115. 10.1016/J.COBEHA.2016.07.003
- 84Zelick, P. R. (2007). Issues in the Psychology of Motivation. Nova Publishers. 10.1016/S1351-4180(06)72048-7