Have a personal or library account? Click to login
From a Measure of Confidence to a Measure of the Level of Knowledge Cover

From a Measure of Confidence to a Measure of the Level of Knowledge

By: Daniel Defays  
Open Access
|May 2025

References

  1. Attali, Y., Budescu, D. V., & Arieli-Attali, M. (2020). An item response approach to calibration of confidence judgments. Decision, 7(1), 119. 10.1037/dec0000111
  2. Brier, G. W. (1950). Verification of forecasts expressed in terms of probability. Monthly Weather Review, 78(1), 13. 10.1175/1520-0493(1950)078<;0001:VOFEIT>2.0.CO;2
  3. Budescu, D. V., & Johnson, T. R. (2011). A model-based approach for the analysis of the calibration of probability judgments. Judgment and Decision Making, 6(8), 857869. 10.1017/S1930297500004277
  4. Budescu, D. V., Wallsten, T. S., & Au, W. T. (1997). On the importance of random error in the study of probability judgment: Part II. Applying the stochastic judgment model to detect systematic trends. Journal of Behavioral Decision Making, 10(3), 173188. 10.1002/(SICI)1099-0771(199709)10:3<;173::AID-BDM261>3.0.CO;2-6
  5. Deng, S., McCarthy, D. E., Piper, M. E., Baker, T. B., & Bolt, D. M. (2018). Extreme response style and the measurement of intra-individual variability in affect. Multivariate Behavioral Research, 53(2), 199218. 10.1080/00273171.2017.1413636
  6. Erev, I., Wallsten, T. S., & Budescu, D. V. (1994). Simultaneous over- and underconfidence: The role of error in judgment processes. Psychological Review, 101, 519527. 10.1037/0033-295X.101.3.519
  7. Fajfar, P. F., & Gurman, N. (2009). When underconfident behavior is norm: Some experimental evidences from the calibration analysis. 10.2139/ssrn.1459722
  8. Ferrando, P., Anguiano-Carrasco, C., & Demestre, J. (2013). Combining IRT and SEM: A hybrid model for fitting responses and response certainties. Structural Equation Modeling, 20, 208225. 10.1080/10705511.2013.769388
  9. Fleming, S. M., & Lau, H. C. (2014). How to measure metacognition. Frontiers in Human Neuroscience, 8, 443. 10.3389/fnhum.2014.00443
  10. Gigerenzer, G. (2004). The irrationality paradox. Behavioral and Brain Sciences, 27, 336338. 10.1017/S0140525X04310083
  11. Gigerenzer, G., Hoffrage, H., & Kleinbölting, H. (1991). Probabilistic mental models: A brunswickian theory of confidence. Psychological Review, 98, 506528. 10.1037/0033-295X.98.4.506
  12. Koriat, A. (1993). How do we know that we know? The accessibility model of the feeling of knowing. Psychological Review, 100, 609639. 10.1037/0033-295X.100.4.609
  13. Koriat, A. (2011). Subjective confidence in perceptual judgments: A test of the self-consistency model. Journal of Experimental Psychology: General, 140(1), 117139. 10.1037/a0022171
  14. Koriat, A. (2013). Confidence in personal preferences. Journal of Behavioral Decision Making, 26, 247259. 10.1002/bdm.1758
  15. Koriat, A. (2018). When reality is out of focus: Can people tell whether their beliefs and judgments are correct or wrong? Journal of Experimental Psychology: General, 147(5), 613631. 10.1037/xge0000397
  16. Koriat, A., & Adiv, S. (2015). The self-consistency theory of subjective confidence. Oxford Handbooks Online. 10.1093/oxfordhb/9780199336746.013.18
  17. Leclercq, D. (1982). Confidence marking, its use in testing. Evaluation in Education, An International Review Series, 6, 191221. 10.1016/0145-9228(82)90001-2
  18. Leclercq, D. (2016). J’en suis aussi sûr que vous, mais pas avec le même pourcentage de chances, que ce soit hors contexte ou en contexte. Evaluer. Journal International de Recherche sur l’Evaluation et la Formation, 2(1), 89125. http://hdl.handle.net/2268/202730
  19. Leclercq, D. (2017). Une mata-analyse des degrés de certitude exprimés en mots. Evaluer. Journal International de Recherche sur l’Evaluation et la Formation, 2(3), 69105. http://hdl.handle.net/2268/210317
  20. Leclercq, D. (2022). Precision or granularity of confidence degrees. 8th International Biannual Conference of SIG 16 ‘Metacognition’ of EARLI (European Association for Research on Learning & Instruction). https://orbi.uliege.be/handle/2268/292756
  21. Leclercq, D. (2025). Dataset on measure of confidence and measure of level of knowledge on animal cries and capital cities [Data set], 10.58119/ULG/PWNZBM, ULiège Open Data Repository, V1
  22. Lichtenstein, S., Fisschoff, B., & Phillips, L. (1982). Calibration of probabilities: The state of the art to 1980. In D. Kahneman, P. Slovic & A. Tversky (Eds.), Judgment under Uncertainty. Heuristics and Biases (pp. 306334), Cambridge University Press, Cambridge, UK. 10.1017/CBO9780511809477.023
  23. Lord, F. M., & Novick, M. R. (1968). Statistical Theories of Mental Test Scores. Addison-Wesley, Menlo Park.
  24. Masson, M. E., & Rotello, C. M. (2009). Sources of bias in the Goodman–Kruskal gamma coefficient measure of association: Implications for studies of metacognitive processes. Journal of Experimental Psychology, 35(2), 509527. 10.1037/a0014876
  25. Mattes, B., & Pieschl, S. (2022). An alignment of standards enhances metacognitive judgment accuracy in explanatory knowledge tasks with internet search. Proceedings of the Annual Meeting of the Cognitive Science Society, 44(44). https://escholarship.org/uc/item/2vt834qk
  26. Merkle, E. C., & Van Zandt, T. (2006). An application of the Poisson race model to confidence calibration. Journal of Experimental Psychology: General, 135(3), 391408. 10.1037/0096-3445.135.3.391
  27. Miller, D. J., Spengler, E. S., & Spengler, P. M. (2015). A meta-analysis of confidence and judgment accuracy in clinical decision making. Journal of Counseling Psychology, 62(4), 553567. 10.1037/cou0000105
  28. Prosperi, O. (2015). Le réalisme avec degrés de certitude. Mesure et Evaluation en Education, 38, 121140. 10.7202/1036553ar
  29. Rutherford, T. (2017). The measurement of calibration in real contexts. Learning and Instruction, 47, 3342. 10.1016/j.learninstruc.2016.10.006
  30. Schraw, G., Kuch, F., & Gutierrez, A. P. (2013). Measure for measure: Calibrating ten commonly used calibration scores. Learning and Instruction, 24, 4857. 10.1016/j.learninstruc.2012.08.007
  31. Tullis, J., & Goldstone, R. (2020). Why does peer instruction benefit student learning? Cognitive Research: Principles and Implications, 5(1). 10.1186/s41235-020-00218-5
  32. Winman, A., Juslin, P., & Björkman, M. (1998). The confidence – hindsight mirror effect in judgment: An accuracy-assessment model for the knew-it-all-along phenomenon. Journal of Experimental Psychology: Learning, Memory, and Cognition, 24(2), 415431. 10.1037/0278-7393.24.2.415
DOI: https://doi.org/10.5334/pb.1332 | Journal eISSN: 0033-2879
Language: English
Submitted on: Jun 20, 2024
|
Accepted on: Apr 18, 2025
|
Published on: May 22, 2025
Published by: Ubiquity Press
In partnership with: Paradigm Publishing Services
Publication frequency: 1 issue per year

© 2025 Daniel Defays, published by Ubiquity Press
This work is licensed under the Creative Commons Attribution 4.0 License.