Have a personal or library account? Click to login
We need more replication research – A case for test-retest reliability Cover

We need more replication research – A case for test-retest reliability

Open Access
|Apr 2017

References

  1. Crandall CS Sherman JW On the scientific superiority of conceptual replications for scientific progress J Exp Soc Psychol 2016 66 93 99 10.1016/j.jesp.2015.10.002
  2. Cumming G Replication and p intervals: p values predict the future only vaguely, but confidence intervals do much better Perspect Psychol Sci 2008 3 286 300 10.1111/j.1745-6924.2008.00079.x
  3. Earp BD Trafimow D Replication, falsification and the crisis of confidence in social psychology Front Psychol 2015 6 1 11 10.3389/fpsyg.2015.00621
  4. Huffmeier J Mazei J Schultze T Reconceptualizing replication as a sequence of different studies:a replication typology J Exp Soc Psychol 2016 66 81 92 10.1016/j.jesp.2015.09.009
  5. Ioannidis JP Why most published research findings are false PLOS Med 2005 2 e124 10.1371/journal.pmed.0020124
  6. Klein SB What can recent replication failures tell us about theoretical commitments of psychology? Theory Psychol 2014 24 326 338 10.1177/0959354314529616
  7. Nosek BA Aarts A Anderson JE PSYCHOLOGY. Estimating the reproducibility of psychological science Science 2015
  8. Pashler H Harris CR Is the replicability crisis overblown? Three arguments examined Perspect Psychol Sci 2012 7 531 536 10.1177/1745691612463401
  9. Schmidt S Shall we really do it again? The powerful concept of replication is neglected in social sciences Rev Gen Psychol 2009 13 90 100 10.1037/a0015108
  10. Leppink J O’Sullivan P Winston K On variation and uncertainty Perspect Med Educ 2016 5 231 234 10.1007/s40037-016-0281-5
  11. Leppink J Pérez-Fuster P What is science without replication? Perspect Med Educ 2016
  12. Picho K Maggio L Artino AR Science: the slow march of accumulating evidence Perspect Med Educ 2016
  13. Cronbach LJ Coefficient alpha and the internal structure of tests Psychometrika 1951 16 297 334 10.1007/BF02310555
  14. Revelle W Zinbarg RE Coefficients alpha, beta, omega, and the glb: comments on Sijtsma Psychometrika 2009 74 145 154 10.1007/s11336-008-9102-z
  15. Crocker L Algina J Introduction to classical & modern test theory 2006 London Thomson
  16. Cohen J A coefficient of agreement for nominal scales Educ Psychol Meas 1960 20 37 46 10.1177/001316446002000104
  17. Fleiss JL Cohen J The equivalence of weighted kappa and the intraclass correlation coefficient as measures of reliability Educ Psychol Meas 1973 33 613 619 10.1177/001316447303300309
  18. Beckman TJ Ghosh AK Cook DA Erwin PJ Mandrekar JN How reliable are assessments of clinical teaching? A review of the published instruments J Gen Intern Med 2004 19 971 977 10.1111/j.1525-1497.2004.40066.x
  19. Bland JM Altman DG Statistics notes: Cronbach’s alpha BMJ 1997 314 572 10.1136/bmj.314.7080.572
  20. Sullivan GM A primer on the validity of assessment instruments J Grad Med Educ 2011 3 119 120 10.4300/JGME-D-11-00075.1
  21. Tavakol M Dennick R Making sense of Cronbach’s alpha Int J Med Educ 2011 2 53 55 10.5116/ijme.4dfb.8dfd
  22. Kline RB Principle and practice of structural equation modeling 2010 3 London The Guilford Press
  23. Tacq J Multivariate analysis techniques in social science research:from problem to analysis 1997 London SAGE
  24. Field A Discovering statistics using IBM SPSS statistics 2013 4 London SAGE
  25. Dunn TJ Baguley T Brunsden V From alpha to omega: a practical solution to the pervasive problem of internal consistency estimation Br J Psychol 2014 105 399 412 10.1111/bjop.12046
  26. Peters GJY The alpha and the omega of scale reliability and validity Eur Health Psychol 2014 16 56 69
  27. Sijtsma K On the use, the misuse, and the very limited usefulness of Cronbach’s alpha Psychometrika 2009 74 107 120 10.1007/s11336-008-9101-0
  28. Paas F Training strategies for attaining transfer of problem-solving skill in statistics: a cognitive-load approach J Educ Psychol 1992 84 429 434 10.1037/0022-0663.84.4.429
  29. Ayres P Using subjective measures to detect variations of intrinsic load within problems Learn Instr 2006 16 389 400 10.1016/j.learninstruc.2006.09.001
  30. Paas F Tuovinen J Tabbers H Van Gerven PWM Cognitive load measurement as a means to advance cognitive load theory Educ Psychol 2003 38 63 71 10.1207/S15326985EP3801_8
  31. Crutzen R Time is a jailer: what do alpha and its alternatives tell us about reliability? Eur Health Psychol 2014 16 70 74
  32. Leppink J Van Merriënboer JJG The beast of aggregating cognitive load measures in technology-based learning Educ Technol Soc 2015 18 230 245
  33. Graham JM Congeneric and (essentially) tau-equivalent estimates of score reliability: what they are and how to use them Educ Psychol Meas 2006 66 930 944 10.1177/0013164406288165
  34. Koriat A Nussinson R Ackerman R Judgments of learning depend on how learners interpret study effort J Exp Psychol 2014 40 1624 1637
  35. Van Loon MH De Bruin ABH Van Gog T Van Merriënboer JJG The effect of delayed JOLs and sentence generation on children’s monitoring accuracy and regulation of idiom study Metacogn Learn 2013 8 173 191 10.1007/s11409-013-9100-0
  36. Sibbald M De Bruin ABH Feasibility of self-reflection as a tool to balance clinical reasoning strategies Adv Health Sci Educ 2012 17 419 429 10.1007/s10459-011-9320-5
  37. Leppink J Van den Heuvel A The evolution of cognitive load theory and its application to medical education Perspect Med Educ 2015 4 119 127 10.1007/s40037-015-0192-x
  38. Leppink J Paas F Van der Vleuten CPM Van Gog T Van Merriënboer JJG Development of an instrument for measuring different types of cognitive load Behav Res Methods 2013 45 1058 1072 10.3758/s13428-013-0334-1
  39. Naismith LM Cheung JJH Ringsted C Cavalcanti RB Limitations of subjective cognitive load measures in simulation-based procedural training Med Educ 2015 49 805 814 10.1111/medu.12732
  40. Young JQ Irby DM Barilla-LaBarca ML Ten Cate O O’Sullivan PS Measuring cognitive load:mixed results from a handover simulation for medical students Perspect Med Educ 2016 5 24 32 10.1007/s40037-015-0240-6
  41. Chmielewski M Watson D What is being assessed and why it matters: the impact of transient error on trait research J Pers Soc Psychol 2009 97 186 202 10.1037/a0015618
  42. Green SB A coefficient alpha for test-retest data Psychol Meth 2003 8 88 101 10.1037/1082-989X.8.1.88
  43. Salerno DF Franzblau A Armstrong TJ Werner RA Becker MA Test-retest reliability of the upper extremity questionnaire among keyboard operators Am J Ind Med 2001 40 655 666 10.1002/ajim.10024
  44. Hedeker D Gibbons RD Longitudinal data analysis 2006 New York Wiley
  45. Molenberghs G Verbeke G Models for discrete longitudinal data 2006 New York Springer
  46. Verbeke G Molenberghs G Linear mixed models for longitudinal data 2000 New York Springer
  47. Kramer MS Feinstein AR Clinical biostatistics LIV. The biostatistics of concordance Clin Pharmacol Ther 1981 29 111 123 10.1038/clpt.1981.18
Language: English
Published on: Apr 7, 2017
In partnership with: Paradigm Publishing Services
Publication frequency: 1 issue per year

© 2017 Jimmie Leppink, Patricia Pérez-Fuster, published by Bohn Stafleu van Loghum
This work is licensed under the Creative Commons Attribution 4.0 License.