Have a personal or library account? Click to login
Are Faster Participants Always Faster? Assessing Reliability of Participants’ Mean Response Speed in Picture Naming Cover

Are Faster Participants Always Faster? Assessing Reliability of Participants’ Mean Response Speed in Picture Naming

Journal of Cognition
Volume 7 (2024): Issue 1
By: Pamela Fuhrmeister,  Shereen Elbuy and  Audrey Bürki  
Open Access
|Jan 2024

References

  1. 1Alario, F.-X., Ferrand, L., Laganaro, M., New, B., Frauenfelder, U. H., & Segui, J. (2004). Predictors of picture naming speed. Behavior Research Methods, Instruments, & Computers, 36(1), 140–155. DOI: 10.3758/BF03195559
    Back to article
  2. 2Baayen, R. H., & Milin, P. (2010). Analyzing reaction times. International Journal of Psychological Research, 3(2), 12–28. DOI: 10.21500/20112084.807
    Back to article
  3. 3Bartolozzi, F., Jongman, S. R., & Meyer, A. S. (2021). Concurrent speech planning does not eliminate repetition priming from spoken words: Evidence from linguistic dual-tasking. Journal of Experimental Psychology: Learning, Memory, and Cognition, 47(3), 466. DOI: 10.1037/xlm0000944
    Back to article
  4. 4Boersma, P., & Weenink, D. (2021). Praat: Doing phonetics by computer [computer program]. Version 6.1. 44.
    Back to article
  5. 5Borgmann, K. W., Risko, E. F., Stolz, J. A., & Besner, D. (2007). Simon says: Reliability and the role of working memory and attentional control in the simon task. Psychonomic Bulletin & Review, 14(2), 313–319. DOI: 10.3758/BF03194070
    Back to article
  6. 6Borragan, M., Martin, C. D., De Bruin, A., & Duñabeitia, J. A. (2018). Exploring different types of inhibition during bilingual language production. Frontiers in Psychology, 9, 2256. DOI: 10.3389/fpsyg.2018.02256
    Back to article
  7. 7Boudiaf, N., Laboissière, R., Cousin, É., Fournet, N., Krainik, A., & Baciu, M. (2018). Behavioral evidence for a differential modulation of semantic processing and lexical production by aging: A full linear mixed-effects modeling approach. Aging, Neuropsychology, and Cognition, 25(1), 1–22. DOI: 10.1080/13825585.2016.1257100
    Back to article
  8. 8Bürki, A. (2017). Electrophysiological characterization of facilitation and interference in the picture-word interference paradigm. Psychophysiology, 54(9), 1370–1392. DOI: 10.1111/psyp.12885
    Back to article
  9. 9Bürkner, P.-C. (2017). brms: An R package for Bayesian multilevel models using Stan. Journal of Statistical Software, 80(1), 1–28. DOI: 10.18637/jss.v080.i01
    Back to article
  10. 10Chen, G., Pine, D. S., Brotman, M. A., Smith, A. R., Cox, R. W., & Haller, S. P. (2021). Trial and error: A hierarchical modeling approach to test-retest reliability. NeuroImage, 245, 118647. DOI: 10.1016/j.neuroimage.2021.118647
    Back to article
  11. 11Congdon, E., Mumford, J. A., Cohen, J. R., Galvan, A., Canli, T., & Poldrack, R. A. (2012). Measurement and reliability of response inhibition. Frontiers in Psychology, 3, 37. DOI: 10.3389/fpsyg.2012.00037
    Back to article
  12. 12Conway, A. R., Kane, M. J., Bunting, M. F., Hambrick, D. Z., Wilhelm, O., & Engle, R. W. (2005). Working memory span tasks: A methodological review and user’s guide. Psychonomic Bulletin & Review, 12(5), 769–786. DOI: 10.3758/BF03196772
    Back to article
  13. 13Costa, A., Strijkers, K., Martin, C., & Thierry, G. (2009). The time course of word retrieval revealed by event-related brain potentials during overt speech. Proceedings of the National Academy of Sciences, 106(50), 21442–21446. DOI: 10.1073/pnas.0908921106
    Back to article
  14. 14Cuetos, F., Ellis, A. W., & Alvarez, B. (1999). Naming times for the snodgrass and vanderwart pictures in spanish. Behavior Research Methods, Instruments, & Computers, 31(4), 650–658. DOI: 10.3758/BF03200741
    Back to article
  15. 15Damian, M. F., & Dumay, N. (2007). Time pressure and phonological advance planning in spoken production. Journal of Memory and Language, 57(2), 195–209. DOI: 10.1016/j.jml.2006.11.001
    Back to article
  16. 16Donders, J. (1997). A short form of the WISC–III for clinical use. Psychological Assessment, 9(1), 15. DOI: 10.1037/1040-3590.9.1.15
    Back to article
  17. 17Duñabeitia, J. A., Crepaldi, D., Meyer, A. S., New, B., Pliatsikas, C., Smolka, E., & Brysbaert, M. (2018). MultiPic: A standardized set of 750 drawings with norms for six european languages. Quarterly Journal of Experimental Psychology, 71(4), 808–816. DOI: 10.1080/17470218.2017.1310261
    Back to article
  18. 18Ellis, A. W., & Morrison, C. M. (1998). Real age-of-acquisition effects in lexical retrieval. Journal of Experimental Psychology: Learning, Memory, and Cognition, 24(2), 515. DOI: 10.1371/journal.pone.0258908
    Back to article
  19. 19Fairs, A., & Strijkers, K. (2021). Can we use the internet to study speech production? Yes we can! Evidence contrasting online versus laboratory naming latencies and errors. PLoS One, 16(10). DOI: 10.1371/journal.pone.0258908
    Back to article
  20. 20Fargier, R., & Laganaro, M. (2019). Interference in speaking while hearing and vice versa. Scientific Reports, 9(1), 1–13. DOI: 10.1038/s41598-019-41752-7
    Back to article
  21. 21Fieder, N., Wartenburger, I., & Rahman, R. A. (2019). A close call: Interference from semantic neighbourhood density and similarity in language production. Memory & Cognition, 47(1), 145–168. DOI: 10.3758/s13421-018-0856-y
    Back to article
  22. 22Fuhrmeister, P., Madec, S., Lorenz, A., Elbuy, S., & Bürki, A. (2022). Behavioral and EEG evidence for inter-individual variability in late encoding stages of word production. Language, Cognition, and Neuroscience. DOI: 10.1177/1747021817747266
    Back to article
  23. 23Gauvin, H. S., Jonen, M. K., Choi, J., McMahon, K., & Zubicaray, G. I. de. (2018). No lexical competition without priming: Evidence from the picture–word interference paradigm. Quarterly Journal of Experimental Psychology, 71(12), 2562–2570. DOI: 10.1177/1747021817747266
    Back to article
  24. 24Gelman, A., & Carlin, J. (2014). Beyond power calculations: Assessing type s (sign) and type m (magnitude) errors. Perspectives on Psychological Science, 9(6), 641–651. DOI: 10.1177/1745691614551642
    Back to article
  25. 25Gerhand, S., & Barry, C. (1999). Age-of-acquisition and frequency effects in speeded word naming. Cognition, 73(2), B27–B36. DOI: 10.1016/S0010-0277(99)00052-9
    Back to article
  26. 26Gordon, J. K., & Cheimariou, S. (2013). Semantic interference in a randomized naming task: Effects of age, order, and category. Cognitive Neuropsychology, 30(7–8), 476–494. DOI: 10.1080/02643294.2013.877437
    Back to article
  27. 27Haines, N., Kvam, P. D., Irving, L. H., Smith, C., Beauchaine, T. P., Pitt, M. A., &… Turner, B. (2020). Learning from the reliability paradox: How theoretically informed generative models can advance the social, behavioral, and brain sciences.
    Back to article
  28. 28Hedge, C., Powell, G., & Sumner, P. (2018). The reliability paradox: Why robust cognitive tasks do not produce reliable individual differences. Behavior Research Methods, 50(3), 1166–1186. DOI: 10.3758/s13428-017-0935-1
    Back to article
  29. 29Heitz, R. P. (2014). The speed-accuracy tradeoff: History, physiology, methodology, and behavior. Frontiers in Neuroscience, 8, 150. DOI: 10.3389/fnins.2014.00150
    Back to article
  30. 30Hintz, F., Jongman, S. R., Dijkhuis, M., Hoff, V. van’t, McQueen, J. M., & Meyer, A. S. (2020). Shared lexical access processes in speaking and listening? An individual differences study. Journal of Experimental Psychology: Learning, Memory, and Cognition, 46(6), 1048. DOI: 10.1037/xlm0000768
    Back to article
  31. 31Jhangiani, R. S., Chiang, I., & Price, P. C. (2015). Research methods in psychology-2nd canadian edition. BC Campus.
    Back to article
  32. 32Jongman, S. R. (2017). Sustained attention ability affects simple picture naming. Collabra: Psychology, 3(1). DOI: 10.1525/collabra.84
    Back to article
  33. 33Jongman, S. R., Meyer, A. S., & Roelofs, A. (2015). The role of sustained attention in the production of conjoined noun phrases: An individual differences study. PloS One, 10(9), e0137557. DOI: 10.1371/journal.pone.0137557
    Back to article
  34. 34Jongman, S. R., Roelofs, A., & Meyer, A. S. (2015). Sustained attention in language production: An individual differences investigation. Quarterly Journal of Experimental Psychology, 68(4), 710–730. DOI: 10.1080/17470218.2014.964736
    Back to article
  35. 35Kello, C. T., Plaut, D. C., & MacWhinney, B. (2000). The task dependence of staged versus cascaded processing: An empirical and computational study of stroop interference in speech perception. Journal of Experimental Psychology: General, 129(3), 340. DOI: 10.1037/0096-3445.129.3.340
    Back to article
  36. 36Klaus, J., & Schriefers, H. (2018). An investigation of the role of working memory capacity and naming speed in phonological advance planning in language production. The Mental Lexicon, 13(2), 159–185. DOI: 10.1075/ml.17020.kla
    Back to article
  37. 37Klein, K., & Fiss, W. H. (1999). The reliability and stability of the turner and engle working memory task. Behavior Research Methods, Instruments, & Computers, 31(3), 429–432. DOI: 10.3758/BF03200722
    Back to article
  38. 38Kruschke, J. K. (2018). Rejecting or accepting parameter values in bayesian estimation. Advances in Methods and Practices in Psychological Science, 1(2), 270–280. DOI: 10.1177/2515245918771304
    Back to article
  39. 39Kuperman, V., Stadthagen-Gonzalez, H., & Brysbaert, M. (2012). Age-of-acquisition ratings for 30,000 english words. Behavior Research Methods, 44(4), 978–990. DOI: 10.3758/s13428-012-0210-4
    Back to article
  40. 40Laganaro, M., Valente, A., & Perret, C. (2012). Time course of word production in fast and slow speakers: A high density ERP topographic study. NeuroImage, 59(4), 3881–3888. DOI: 10.1016/j.neuroimage.2011.10.082
    Back to article
  41. 41Landis, J. R., & Koch, G. G. (1977). The measurement of observer agreement for categorical data. Biometrics, 159–174. DOI: 10.2307/2529310
    Back to article
  42. 42Lorenz, A., Zwitserlood, P., Regel, S., & Abdel Rahman, R. (2019). Age-related effects in compound production: Evidence from a double-object picture naming task. Quarterly Journal of Experimental Psychology, 72(7), 1667–1681. DOI: 10.1177/1747021818806491
    Back to article
  43. 43Ly, A., Verhagen, J., & Wagenmakers, E.-J. (2016). Harold jeffreys’s default bayes factor hypothesis tests: Explanation, extension, and application in psychology. Journal of Mathematical Psychology, 72, 19–32. DOI: 10.1016/j.jmp.2015.06.004
    Back to article
  44. 44Meyer, A. S. (1996). Lexical access in phrase and sentence production: Results from picture–word interference experiments. Journal of Memory and Language, 35(4), 477–496. DOI: 10.1006/jmla.1996.0026
    Back to article
  45. 45Morey, R. D., & Rouder, J. N. (2018). BayesFactor: Computation of bayes factors for common designs (R package version 0.9.12-4.2). https://CRAN.R-project.org/package=BayesFactor
    Back to article
  46. 46Nunnally, J. C. (1994). Psychometric theory 3E. Tata McGraw-hill education.
    Back to article
  47. 47Parsons, S., Kruijt, A.-W., & Fox, E. (2019). Psychological science needs a standard practice of reporting the reliability of cognitive-behavioral measurements. Advances in Methods and Practices in Psychological Science, 2(4), 378–395. DOI: 10.1177/2515245919879695
    Back to article
  48. 48Piai, V., & Roelofs, A. (2013). Working memory capacity and dual-task interference in picture naming. Acta Psychologica, 142(3), 332–342. DOI: 10.1016/j.actpsy.2013.01.006
    Back to article
  49. 49Pinet, S., Ziegler, J. C., & Alario, F.-X. (2016). Typing is writing: Linguistic properties modulate typing execution. Psychonomic Bulletin & Review, 23(6), 1898–1906. DOI: 10.3758/s13423-016-1044-3
    Back to article
  50. 50R Core Team. (2021). R: A language and environment for statistical computing. Vienna, Austria: R Foundation for Statistical Computing. Retrieved from https://www.R-project.org/
    Back to article
  51. 51Rabovsky, M., Schad, D. J., & Rahman, R. A. (2016). Language production is facilitated by semantic richness but inhibited by semantic density: Evidence from picture naming. Cognition, 146, 240–244. DOI: 10.3758/BF03196853
    Back to article
  52. 52Roelofs, A. (2004). Seriality of phonological encoding in naming objects and reading their names. Memory & Cognition, 32(2), 212–222. DOI: 10.3758/BF03196853
    Back to article
  53. 53Rouder, J. N., & Haaf, J. M. (2019). A psychometrics of individual differences in experimental tasks. Psychonomic Bulletin & Review, 26(2), 452–467. DOI: 10.3758/s13423-018-1558-y
    Back to article
  54. 54Schad, D. J., Betancourt, M., & Vasishth, S. (2021). Toward a principled bayesian workflow in cognitive science. Psychological Methods, 26(1), 103. DOI: 10.1037/met0000275
    Back to article
  55. 55Shao, Z., Meyer, A. S., & Roelofs, A. (2013). Selective and nonselective inhibition of competitors in picture naming. Memory & Cognition, 41(8), 1200–1211. DOI: 10.3758/s13421-013-0332-7
    Back to article
  56. 56Shao, Z., Roelofs, A., Acheson, D. J., & Meyer, A. S. (2014). Electrophysiological evidence that inhibition supports lexical selection in picture naming. Brain Research, 1586, 130–142. DOI: 10.1016/j.brainres.2014.07.009
    Back to article
  57. 57Shao, Z., Roelofs, A., & Meyer, A. S. (2012). Sources of individual differences in the speed of naming objects and actions: The contribution of executive control. Quarterly Journal of Experimental Psychology, 65(10), 1927–1944. DOI: 10.1080/17470218.2012.670252
    Back to article
  58. 58Sikora, K., Roelofs, A., Hermans, D., & Knoors, H. (2016). Executive control in spoken noun-phrase production: Contributions of updating, inhibiting, and shifting. Quarterly Journal of Experimental Psychology, 69(9), 1719–1740. DOI: 10.1080/17470218.2015.1093007
    Back to article
  59. 59Snodgrass, J. G., & Yuditsky, T. (1996). Naming times for the snodgrass and vanderwart pictures. Behavior Research Methods, Instruments, & Computers, 28(4), 516–536. DOI: 10.3758/BF03200540
    Back to article
  60. 60Starreveld, P. A., & La Heij, W. (1999). Word substitution errors in a speeded picture-word task. The American Journal of Psychology, 112(4), 521. DOI: 10.2307/1423649
    Back to article
  61. 61Urbina, S. (2014). Essentials of psychological testing. John Wiley & Sons.
    Back to article
  62. 62Valente, A., Bürki, A., & Laganaro, M. (2014). ERP correlates of word production predictors in picture naming: A trial by trial multiple regression analysis from stimulus onset to response. Frontiers in Neuroscience, 8, 390. DOI: 10.3389/fnins.2014.00390
    Back to article
  63. 63Van Heuven, W. J., Mandera, P., Keuleers, E., & Brysbaert, M. (2014). SUBTLEX-UK: A new and improved word frequency database for british english. Quarterly Journal of Experimental Psychology, 67(6), 1176–1190. DOI: 10.1080/17470218.2013.850521
    Back to article
  64. 64Viswesvaran, C., & Ones, D. S. (2000). Measurement error in “big five factors” personality assessment: Reliability generalization across studies and measures. Educational and Psychological Measurement, 60(2), 224–235. DOI: 10.1177/00131640021970475
    Back to article
  65. 65Waters, G. S., & Caplan, D. (2003). The reliability and stability of verbal working memory measures. Behavior Research Methods, Instruments, & Computers, 35(4), 550–564. DOI: 10.3758/BF03195534
    Back to article
  66. 66Zehr, J., & Schwarz, F. (2018). Penncontroller for internet based experiments (ibex).
    Back to article
  67. 67Zu Wolfsthurn, S. von G., Robles, L. P., & Schiller, N. O. (2021). Cross-linguistic interference in late language learners: An ERP study. Brain and Language, 221, 104993. DOI: 10.1016/j.bandl.2021.104993
    Back to article
DOI: https://doi.org/10.5334/joc.337 | Journal eISSN: 2514-4820
Journal RSS Feed
Language: English
Submitted on: Aug 26, 2021
Accepted on: Dec 6, 2023
Published on: Jan 11, 2024
Published by: Ubiquity Press
In partnership with: Paradigm Publishing Services
Publication frequency: 1 issue per year
Keywords:
language production,
picture naming,
reliability,
individual differences

© 2024 Pamela Fuhrmeister, Shereen Elbuy, Audrey Bürki, published by Ubiquity Press
This work is licensed under the Creative Commons Attribution 4.0 License.

Previous articleVolume 7 (2024): Issue 1Next article