Have a personal or library account? Click to login
About Face: Seeing the Talker Improves Spoken Word Recognition but Increases Listening Effort Cover

About Face: Seeing the Talker Improves Spoken Word Recognition but Increases Listening Effort

Journal of Cognition
Volume 2 (2019): Issue 1
By: Violet A. Brown and  Julia F. Strand  
Open Access
|Nov 2019

References

  1. 1Alhanbali, S., Dawes, P., Millman, R. E., & Munro, K. J. (2019). Measures of listening effort are multidimensional. Ear and Hearing. DOI: 10.1097/AUD.0000000000000697
    Back to article
  2. 2Alsius, A., Navarra, J., Campbell, R., & Soto-Faraco, S. (2005). Audiovisual integration of speech falters under high attention demands. Current Biology: CB, 15(9), 839–843. DOI: 10.1016/j.cub.2005.03.046
    Back to article
  3. 3Arnold, P., & Hill, F. (2001). Bisensory augmentation: A speechreading advantage when speech is clearly audible and intact. British Journal of Psychology, 92, 339–355. DOI: 10.1348/000712601162220
    Back to article
  4. 4Balota, D. A., Yap, M. J., Cortese, M. J., Hutchison, K. A., Kessler, B., Loftis, B., Treiman, R., et al. (2007). The English Lexicon Project. Behavior Research Methods, 39(3), 445–459. DOI: 10.3758/BF03193014
    Back to article
  5. 5Barr, D. J., Levy, R., Scheepers, C., & Tily, H. J. (2013). Random effects structure for confirmatory hypothesis testing: Keep it maximal. Journal of Memory and Language, 68(3). DOI: 10.1016/j.jml.2012.11.001
    Back to article
  6. 6Bates, D., Maechler, M., Bolker, B., Walker, S., Christensen, R., Singmann, H., Green, P., et al. (2014). Package “lme4” (Version 1.1-15). Retrieved from R foundation for statistical computing, Vienna, 12 website: https://github.com/lme4/lme4/
    Back to article
  7. 7Bourland-Hicks, C., & Tharpe, A. M. (2002). Listening effort and fatigue in school-age children with and without hearing loss. Journal of Speech, Language, and Hearing Research: JSLHR, 45(3), 573–584. DOI: 10.1044/1092-4388(2002/046)
    Back to article
  8. 8Brown, V. A., & Strand, J. F. (2018). Noise increases listening effort in normal-hearing young adults, regardless of working memory capacity. Language, Cognition and Neuroscience, 34, 628–640. DOI: 10.1080/23273798.2018.1562084
    Back to article
  9. 9Danan, M. (1992). Reversed subtitling and dual coding theory: New directions for foreign language instruction. Language Learning, 42(4), 497–527. DOI: 10.1111/j.1467-1770.1992.tb01042.x
    Back to article
  10. 10Desjardins, J. L., & Doherty, K. A. (2013). Age-related changes in listening effort for various types of masker noises. Ear and Hearing, 34(3), 261–272. DOI: 10.1097/AUD.0b013e31826d0ba4
    Back to article
  11. 11Downs, D. W. (1982). Effects of hearing aid use on speech discrimination and listening effort. The Journal of Speech and Hearing Disorders, 47(2), 189–193. DOI: 10.1044/jshd.4702.189
    Back to article
  12. 12Erber, N. P. (1969). Interaction of audition and vision in the recognition of oral speech stimuli. Journal of Speech and Hearing Research, 12(2), 423–425. DOI: 10.1044/jshr.1202.423
    Back to article
  13. 13Fraser, S., Gagné, J.-P., Alepins, M., & Dubois, P. (2010). Evaluating the effort expended to understand speech in noise using a dual-task paradigm: The effects of providing visual speech cues. Journal of Speech, Language, and Hearing Research: JSLHR, 53(1), 18–33. DOI: 10.1044/1092-4388(2009/08-0140)
    Back to article
  14. 14Frtusova, J. B., Winneke, A. H., & Phillips, N. A. (2013). ERP evidence that auditory-visual speech facilitates working memory in younger and older adults. Psychology and Aging, 28(2), 481–494. DOI: 10.1037/a0031243
    Back to article
  15. 15Gagné, J.-P., Besser, J., & Lemke, U. (2017). Behavioral assessment of listening effort using a dual-task paradigm: A review. Trends in Hearing, 21, 2331216516687287. DOI: 10.1177/2331216516687287
    Back to article
  16. 16Gosselin, P. A., & Gagné, J.-P. (2011a). Older adults expend more listening effort than young adults recognizing audiovisual speech in noise. International Journal of Audiology, 50(11), 786–792. DOI: 10.3109/14992027.2011.599870
    Back to article
  17. 17Gosselin, P. A., & Gagné, J.-P. (2011b). Older adults expend more listening effort than young adults recognizing speech in noise. Journal of Speech Language and Hearing Research, 54(3), 944–958. DOI: 10.1044/1092-4388(2010/10-0069)
    Back to article
  18. 18Grant, K. W., & Walden, B. E. (1996). Evaluating the articulation index for auditory-visual consonant recognition. The Journal of the Acoustical Society of America, 100(4), 2415–2424. DOI: 10.1121/1.417950
    Back to article
  19. 19Grant, K. W., Walden, B. E., & Seitz, P. F. (1998). Auditory-visual speech recognition by hearing-impaired subjects: Consonant recognition, sentence recognition, and auditory-visual integration. The Journal of the Acoustical Society of America, 103(5 Pt 1), 2677–2690. DOI: 10.1121/1.422788
    Back to article
  20. 20Helfer, K. S., & Freyman, R. L. (2005). The role of visual speech cues in reducing energetic and informational masking. The Journal of the Acoustical Society of America, 117(2), 842–849. DOI: 10.1121/1.1836832
    Back to article
  21. 21Johnson, J., Xu, J., Cox, R., & Pendergraft, P. (2015). A comparison of two methods for measuring listening effort as part of an audiologic test battery. American Journal of Audiology, 24(3), 419–431. DOI: 10.1044/2015_AJA-14-0058
    Back to article
  22. 22Kahneman, D. (1973). Attention and effort. Englewood Cliffs, NJ: Prentice-Hall.
    Back to article
  23. 23Keidser, G., Best, V., Freeston, K., & Boyce, A. (2015). Cognitive spare capacity: evaluation data and its association with comprehension of dynamic conversations. Frontiers in Psychology, 6, 597. DOI: 10.3389/fpsyg.2015.00597
    Back to article
  24. 24Koelewijn, T., Zekveld, A. A., Festen, J. M., Rönnberg, J., & Kramer, S. E. (2012). Processing load induced by informational masking is related to linguistic abilities. International Journal of Otolaryngology, 2012, 865731. DOI: 10.1155/2012/865731
    Back to article
  25. 25Kuchinsky, S. E., Ahlstrom, J. B., Vaden, K. I., Jr., Cute, S. L., Humes, L. E., Dubno, J. R., & Eckert, M. A. (2013). Pupil size varies with word listening and response selection difficulty in older adults with hearing loss. Psychophysiology, 50(1), 23–34. DOI: 10.1111/j.1469-8986.2012.01477.x
    Back to article
  26. 26Kuznetsova, A., Brockhoff, P., & Christensen, R. (2017). lmerTest Package: Tests in Linear Mixed Effects Models. Journal of Statistical Software, Articles, 82(13), 1–26. DOI: 10.18637/jss.v082.i13
    Back to article
  27. 27Leys, C., Ley, C., Klein, O., Bernard, P., & Licata, L. (2013). Detecting outliers: Do not use standard deviation around the mean, use absolute deviation around the median. Journal of Experimental Social Psychology, 49(4), 764–766. DOI: 10.1016/j.jesp.2013.03.013
    Back to article
  28. 28Mastroberardino, S., Santangelo, V., Botta, F., Marucci, F. S., & Olivetti Belardinelli, M. (2008). How the bimodal format of presentation affects working memory: An overview. Cognitive Processing, 9(1), 69–76. DOI: 10.1007/s10339-007-0195-6
    Back to article
  29. 29Mattys, S. L., Davis, M. H., Bradlow, A. R., & Scott, S. K. (2012). Speech recognition in adverse conditions: A review. Language and Cognitive Processes, 27(7–8), 953–978. DOI: 10.1080/01690965.2012.705006
    Back to article
  30. 30McCoy, S. L., Tun, P. A., Cox, L. C., Colangelo, M., Stewart, R. A., & Wingfield, A. (2005). Hearing loss and perceptual effort: Downstream effects on older adults’ memory for speech. The Quarterly Journal of Experimental Psychology. A, Human Experimental Psychology, 58(1), 22–33. DOI: 10.1080/02724980443000151
    Back to article
  31. 31McGarrigle, R., Dawes, P., Stewart, A. J., Kuchinsky, S. E., & Munro, K. J. (2016). Pupillometry reveals changes in physiological arousal during a sustained listening task. Psychophysiology, 54(2), 193–203. DOI: 10.1111/psyp.12772
    Back to article
  32. 32McGarrigle, R., Munro, K. J., Dawes, P., Stewart, A. J., Moore, D. R., Barry, J. G., & Amitay, S. (2014). Listening effort and fatigue: What exactly are we measuring? A British Society of Audiology Cognition in Hearing Special Interest Group “white paper.” International Journal of Audiology, 53(7), 433–445. DOI: 10.3109/14992027.2014.890296
    Back to article
  33. 33Mishra, S., Lunner, T., Stenfelt, S., Rönnberg, J., & Rudner, M. (2013a). Seeing the talker’s face supports executive processing of speech in steady state noise. Frontiers in Systems Neuroscience, 7, 96. DOI: 10.3389/fnsys.2013.00096
    Back to article
  34. 34Mishra, S., Lunner, T., Stenfelt, S., Rönnberg, J., & Rudner, M. (2013b). Visual information can hinder working memory processing of speech. Journal of Speech, Language, and Hearing Research, 56, 1120–1132. DOI: 10.1044/1092-4388(2012/12-0033)
    Back to article
  35. 35Ng, E. H. N., Rudner, M., Lunner, T., Pedersen, M. S., & Rönnberg, J. (2013). Effects of noise and working memory capacity on memory processing of speech for hearing-aid users. International Journal of Audiology, 52(7), 433–441. DOI: 10.3109/14992027.2013.776181
    Back to article
  36. 36Pashler, H. (1994). Dual-task interference in simple tasks: Data and theory. Psychological Bulletin, 116(2), 220–244. DOI: 10.1037/0033-2909.116.2.220
    Back to article
  37. 37Pichora-Fuller, M. K., Kramer, S. E., Eckert, M. A., Edwards, B., Hornsby, B. W. Y., Humes, L. E., Wingfield, A., et al. (2016). Hearing impairment and cognitive energy: The Framework for Understanding Effortful Listening (FUEL). Ear and Hearing, 37(Suppl 1), 5S–27S. DOI: 10.1097/AUD.0000000000000312
    Back to article
  38. 38Pichora-Fuller, M. K., Schneider, B. A., & Daneman, M. (1995). How young and old adults listen to and remember speech in noise. The Journal of the Acoustical Society of America, 97(1), 593–608. DOI: 10.1121/1.412282
    Back to article
  39. 39Picou, E. M., & Ricketts, T. A. (2014). The effect of changing the secondary task in dual-task paradigms for measuring listening effort. Ear and Hearing, 35(6), 611–622. DOI: 10.1097/AUD.0000000000000055
    Back to article
  40. 40Picou, E. M., Ricketts, T. A., & Hornsby, B. W. Y. (2011). Visual cues and listening effort: individual variability. Journal of Speech, Language, and Hearing Research: JSLHR, 54(5), 1416–1430. DOI: 10.1044/1092-4388(2011/10-0154)
    Back to article
  41. 41Picou, E. M., Ricketts, T. A., & Hornsby, B. W. Y. (2013). How hearing aids, background noise, and visual cues influence objective listening effort. Ear and Hearing, 34(5), e52–e64. DOI: 10.1097/AUD.0b013e31827f0431
    Back to article
  42. 42Rabbitt, P. M. (1968). Channel-capacity, intelligibility and immediate memory. The Quarterly Journal of Experimental Psychology, 20(3), 241–248. DOI: 10.1080/14640746808400158
    Back to article
  43. 43Rönnberg, J., Lunner, T., Zekveld, A., Sörqvist, P., Danielsson, H., Lyxell, B., Rudner, M., et al. (2013). The Ease of Language Understanding (ELU) model: Theoretical, empirical, and clinical advances. Frontiers in Systems Neuroscience, 7, 31. DOI: 10.3389/fnsys.2013.00031
    Back to article
  44. 44Rudner, M., Mishra, S., Stenfelt, S., Lunner, T., & Rönnberg, J. (2016). Seeing the Talker’s Face Improves Free Recall of Speech for Young Adults With Normal Hearing but Not Older Adults With Hearing Loss. Journal of Speech, Language, and Hearing Research: JSLHR, 59(3), 590–599. DOI: 10.1044/2015_JSLHR-H-15-0014
    Back to article
  45. 45Sarampalis, A., Kalluri, S., Edwards, B., & Hafter, E. (2009). Objective measures of listening effort: Effects of background noise and noise reduction. Journal of Speech, Language, and Hearing Research: JSLHR, 52(5), 1230–1240. DOI: 10.1044/1092-4388(2009/08-0111)
    Back to article
  46. 46Seeman, S., & Sims, R. (2015). Comparison of psychophysiological and dual-task measures of listening effort. Journal of Speech, Language, and Hearing Research: JSLHR, 58(6), 1781–1792. DOI: 10.1044/2015_JSLHR-H-14-0180
    Back to article
  47. 47Shoop, C., & Binnie, C. A. (1979). The effects of age upon the visual perception of speech. Scandinavian Audiology, 8(1), 3–8. DOI: 10.3109/01050397909076295
    Back to article
  48. 48Sommers, M. S., & Phelps, D. (2016). Listening effort in younger and older adults: A comparison of auditory-only and auditory-visual presentations. Ear and Hearing, 37(Suppl 1), 62S–8S. DOI: 10.1097/AUD.0000000000000322
    Back to article
  49. 49Sommers, M. S., Tye-Murray, N., & Spehar, B. (2005). Auditory-visual speech perception and auditory-visual enhancement in normal-hearing younger and older adults. Ear and Hearing, 26(3), 263–275. DOI: 10.1097/00003446-200506000-00003
    Back to article
  50. 50Strand, J. F., Brown, V. A., Merchant, M. B., Brown, H. E., & Smith, J. (2018). Measuring listening effort: Convergent validity, sensitivity, and links with cognitive and personality measures. Journal of Speech, Language, and Hearing Research: JSLHR, 61, 1463–1486. DOI: 10.1044/2018_JSLHR-H-17-0257
    Back to article
  51. 51Sumby, W. H., & Pollack, I. (1954). Visual contributions to speech intelligibility in noise. The Journal of the Acoustical Society of America, 26(2), 212–215. DOI: 10.1121/1.1907309
    Back to article
  52. 52Thompson, V. A., & Paivio, A. (1994). Memory for pictures and sounds: Independence of auditory and visual codes. Canadian Journal of Experimental Psychology = Revue Canadienne de Psychologie Experimentale, 48(3), 380–398. DOI: 10.1037/1196-1961.48.3.380
    Back to article
  53. 53Tye-Murray, N., Sommers, M. S., & Spehar, B. (2007). Auditory and visual lexical neighborhoods in audiovisual speech perception. Trends in Amplification, 11(4), 233–241. DOI: 10.1177/1084713807307409
    Back to article
  54. 54Tye-Murray, N., Sommers, M. S., Spehar, B., Myerson, J., Hale, S., & Rose, N. S. (2008). Auditory-visual discourse comprehension by older and young adults in favorable and unfavorable conditions. International Journal of Audiology, 47(sup2), S31–S37. DOI: 10.1080/14992020802301662
    Back to article
  55. 55Tye-Murray, N., Spehar, B., Myerson, J., Hale, S., & Sommers, M. S. (2016). Lipreading and audiovisual speech recognition across the adult lifespan: Implications for audiovisual integration. Psychology and Aging, 31(4), 380–389. DOI: 10.1037/pag0000094
    Back to article
  56. 56Van Engen, K. J., Phelps, J. E. B., Smiljanic, R., & Chandrasekaran, B. (2014). Enhancing speech intelligibility: Interactions among context, modality, speech style, and masker. Journal of Speech, Language, and Hearing Research: JSLHR, 57(5), 1908–1918. DOI: 10.1044/JSLHR-H-13-0076
    Back to article
  57. 57Wagner, A. E., Toffanin, P., & Başkent, D. (2016). The timing and effort of lexical access in natural and degraded speech. Frontiers in Psychology, 7, 398. DOI: 10.3389/fpsyg.2016.00398
    Back to article
  58. 58Winn, M. (2018). Praat script for creating speech-shaped noise [software] version 12. Retrieved July 9, 2018, from http://www.mattwinn.com/praat.html
    Back to article
DOI: https://doi.org/10.5334/joc.89 | Journal eISSN: 2514-4820
Journal RSS Feed
Language: English
Submitted on: Jul 16, 2019
Accepted on: Nov 4, 2019
Published on: Nov 22, 2019
Published by: Ubiquity Press
In partnership with: Paradigm Publishing Services
Publication frequency: 1 issue per year
Keywords:
Auditory word processing,
Multi-sensory perception,
Attention

© 2019 Violet A. Brown, Julia F. Strand, published by Ubiquity Press
This work is licensed under the Creative Commons Attribution 4.0 License.

Previous articleVolume 2 (2019): Issue 1Next article