Have a personal or library account? Click to login
Testing a Planned Missing Design to Reduce Respondent Burden in Web and SMS Administrations of the CAHPS Clinician and Group Survey (CG-CAHPS) Cover

Testing a Planned Missing Design to Reduce Respondent Burden in Web and SMS Administrations of the CAHPS Clinician and Group Survey (CG-CAHPS)

Journal of Official Statistics
Volume 38 (2022): Issue 4 (December 2022)
By: Philip S. Brenner,  J. Lee Hargraves and  Carol Cosenza  
Open Access
|Dec 2022

References

  1. Ahmad, F., A.K. Jhajj, D.E. Stewart, M. Burghardt, and A.S. Bierman. 2014. “Single Item Measures of Self-Rated Mental Health: A Scoping Review.” BMC Health Services Research 14: 398. DOI: https://doi.org/10.1186/1472-6963-14-398.417716525231576
    Search in Google ScholarBack to article
  2. AHRQ, Agency for Healthcare Research and Quality. n.d. “How to Report Results of the CAHPS Clinician & Group Survey.” Available at: https://www.ahrq.gov/sites/default/-files/wysiwyg/cahps/surveys-guidance/cg/cgkit/HowtoReportResultsofCGCAHPS080610FINAL.pdf (accessed September 2021).
    Search in Google ScholarBack to article
  3. Alemi, F., and H.M. Jasper. 2014. “An Alternative to Satisfaction Surveys: Let the Patients Talk.” Quality Management in Health Care 23(1): 10–19. DOI: https://doi.org/10.1097/QMH.0000000000000014.24368718
    Search in Google ScholarBack to article
  4. Antoun, C., and A. Cernat. 2020. “Factors Affecting Completion Times: A Comparative Analysis of Smartphone and PC Web Surveys.” Social Science Computer Review 38(4): 477–489. DOI: https://doi.org/10.1177/0894439318823703.
    Search in Google ScholarBack to article
  5. Ayers, R., S. Brothers, and W. Jansen. 2014. “Guidelines on Mobile Device Forensics.” National Institute of Standards and Technology, Special Publication 800-101r1. Washington, DC: US Department of Commerce. DOI: http://dx.doi.org/10.6028/NIST.Sp.800-101r1.10.6028/NIST.SP.800-101r1
    Search in Google ScholarBack to article
  6. Blumberg, S.J., and J.V. Luke. 2020. “Wireless substitution: Early release of estimates from the National Health Interview Survey, July-December 2019.” National Center for Health Statistics. Available at: https://www.cdc.gov/nchs/nhis.htm. (accessed September 2021).10.15620/cdc:108678
    Search in Google ScholarBack to article
  7. Bosnjak, M., W. Neibarth, M.P. Couper, W. Bandilla, and L. Kaczmirek. 2008. “Prenotification in Web-Based Access Panel Surveys.” Social Science Computer Review 26(2): 213–223. DOI: https://doi.org/10.1177/0894439307305895.
    Search in Google ScholarBack to article
  8. Bradburn, N. 1978. “Respondent Burden.” In Proceedings of the Survey Research Methods Section of the American Statistical Association: 35–40. Alexandria, VA: American Statistical Association. Available at: http://www.asasrms.org/Proceedings/y1978f.html (accessed September 2021).
    Search in Google ScholarBack to article
  9. Brenner, P.S., and J.D. DeLamater. 2013. “Paradata Correlates of Data Quality in an SMS Time Use Study–Evidence from a Validation Study.” Electronic International Journal of Time Use Research 10(1): 38–54. DOI: dx.doi.org/10.13085/eIJTUR.10.1.38-54.10.13085/eIJTUR.10.1.38-54
    Search in Google ScholarBack to article
  10. Conrad, FG., M.P. Couper, R. Tourangeau, and C. Zhang. 2017. “Reducing Speeding in Web Surveys by Providing Immediate Feedback.” Survey Research Methods 11(1): 45–61. DOI: https://doi.org/10.18148/srm/2017.v11i1.6304.686351531745400
    Search in Google ScholarBack to article
  11. Crawford, S.D., M.P. Couper, and M.J. Lamias. 2001. “Web Surveys: Perception of Burden.” Social Science Computer Review 19(2): 146–162. DOI: https://doi.org/10.1177/089443930101900202.
    Search in Google ScholarBack to article
  12. De Leeuw, E.D. 2018. “Mixed-Mode: Past, Present, and Future.” Survey Research Methods 12(2): 75–89. DOI: doi: 10.18148/srm/2018.v12i2.7402.
    Open DOISearch in Google ScholarBack to article
  13. DeSalvo, K.B., V.S. Fan, M.B. McDonell, and S.D. Fihn. 2004. “Predicting Mortality and Healthcare Utilization with a Single Question.” Health Services Research 40(4): 1234–1246. DOI: https://doi.org/10.1111/j.1475-6773.2005.00404.x.136119016033502
    Search in Google ScholarBack to article
  14. Dillman, D.A., J.D. Smyth, and L.M. Christian. 2014. Internet, Phone, Mail, and Mixed-Mode Surveys: The Tailored Design Method. New York: Wiley.
    Search in Google ScholarBack to article
  15. Dyer, N., J.S. Sorra, S.A. Smith, P. Cleary, and R. Hays. 2012. “Psychometric Properties of the Consumer Assessment of Healthcare Providers and Systems (CAHPS) Clinician and Group Adult Visit Survey.” Medical Care 50: S28. DOI: https://doi.org/10.1097/MLR.0b013e31826cbc0d.348067123064274
    Search in Google ScholarBack to article
  16. Elliott M.N., M.K. Beckett, W.G. Lehrman, P. Cleary, C.W. Cohea, L.A. Giordano, E.H. Goldstein, and C.L. Damberg. 2016. “Understanding The Role Played By Medicare’s Patient Experience Points System In Hospital Reimbursement.” Health Affairs 35(9): 1673–1680. DOI: https://doi.org/10.1377/hlthaff.2015.0691.27605650
    Search in Google ScholarBack to article
  17. Elliott, M.N., A.M. Zaslavsky, E. Goldstein, W. Lehrman, K. Hambarsoomians, M.K. Beckett, and L. Giordano. 2009. Effects of survey mode, patient mix, and nonresponse on CAHPSw hospital survey scores. Health Services Research, 44 (2p1): 501–518. DOI: https://doi.org/10.1111/j.1475-6773.2008.00914.x.267705119317857
    Search in Google ScholarBack to article
  18. Enders, C.K. 2010. Applied Missing Data Analysis. New York: Guilford.
    Search in Google ScholarBack to article
  19. Galesic, M., and M. Bosnjak. 2009. “Effects of Questionnaire Length on Participation and Indicators of Response Quality in a Web Survey.” Public Opinion Quarterly 73(2): 349–360. DOI: https://doi.org/10.1093/poq/nfp031.
    Search in Google ScholarBack to article
  20. Garrido, T., B. Raymond, and B. Wheatley. 2016. “Lessons From More than a Decade In patient Portals.” Health Affairs Blog. DOI: https://doi.org/10.1377/hblog20160407.054362.
    Search in Google ScholarBack to article
  21. Graham, J.W., B.J. Taylor, A.E. Olchowski, and P.E. Cumsille. 2006. “Planned Missing Data Designs in Psychological Research.” Psychological Methods 11(4): 323–343. DOI: https://doi.org/10.1037/1082-989X.11.4.323.17154750
    Search in Google ScholarBack to article
  22. Hillygus, D.S., N. Jackson, and M. Young. 2014. “Professional respondents in non-probability online panels.” In Online panel research: A data quality perspective, edited by M. Callegaro, R. Baker, J. Bethlehem, A.S. Góritz, J.A. Krosnick and P.J. Lavrakas: 219–237. New York: Wiley.10.1002/9781118763520.ch10
    Search in Google ScholarBack to article
  23. Holt, J.M. 2019. “Patient Experience in Primary Care: A Systematic Review of CGCAHPS Surveys.” Journal of Patient Experience 6(2): 93–102. DOI: https://doi.org/10.1177/2374373518793143.655894431218253
    Search in Google ScholarBack to article
  24. Johnson, D.R., V. Roth, and R. Young. 2013. “Planned Missing Data Designs in Health Surveys.” In Proceedings of the Tenth Conference on Health Survey Research Methods, November 2013: 43–53. Hyattsville, MD: Department of Health and Human Services, Centers for Disease Control and Prevention, National Center for Health Statistics. Available at: https://www.cdc.gov/nchs/data/hsrmc/hsrmc_10th_proceedings_2011.pdf (accessed February 2020).
    Search in Google ScholarBack to article
  25. Keller, S., A.J. O’Malley, R.D. Hays, R.A. Matthew, A.M. Zaslavsky, K.A. Hepner, and P.D. Cleary. 2005. “Methods Used to Streamline the CAHPSw Hospital Survey.” Health Services Research, 40: 2057–2077. DOI: https://doi.org/10.1111/j.1475-6773.2005.00478.x.136124816316438
    Search in Google ScholarBack to article
  26. Kim, Y., J. Dykema, J. Stevenson, P. Black, and D.P. Moberg. 2019. “Straightlining: Overview of Measurement, Comparison of Indicators, and Effects in Mail–Web Mixed-Mode Surveys.” Social Science Computer Review 37(2): 214–233. DOI: https://doi.org/10.1177/0894439317752406.
    Search in Google ScholarBack to article
  27. Lau, C.Q., H. Sanders, and A. Lombaard. 2019. “Questionnaire Design in Short Message Service (SMS) Surveys.” Field Methods 31(3): 214–229. DOI: https://doi.org/10.1177/1525822X19862506.
    Search in Google ScholarBack to article
  28. Lee, S.S.S., X. Xin, W.P. Lee, E.J. Sim, B. Tan, M.P.G. Bien, A.S.T. Lau, and J. Thumboo. 2013. “The Feasibility of Using SMS as a Health Survey Tool: An Exploratory Study in Patients with Rheumatoid Arthritis.” International Journal of Medical Informatics 82(5): 427–434. DOI: https://doi.org/10.1016/j.ijmedinf.2012.12.003.23332386
    Search in Google ScholarBack to article
  29. Mavletova, A. and M.P. Couper. 2014. “Mobile Web Survey Design: Scrolling versus Paging, SMS versus E-mail Invitations.” Journal of Survey Statistics and Methodology 2: 498–518. DOI: https://doi.org/10.1093/jssam/smu015.
    Search in Google ScholarBack to article
  30. Mavletova, A., and M.P. Couper. 2015. “A Meta-Analysis of Breakoff Rates in Mobile Web Surveys.” In Mobile Research Methods: Opportunities and Challenges of Mobile Research Methodologies, edited by D. Toninelli, R. Pinter, and P. de Pedraza: 81–98. London: Ubiquity Press.
    Search in Google ScholarBack to article
  31. Mavletova, A., M.P. Couper, and D. Lebedev. 2018. “Grid and Item-by-Item Formats in PC and Mobile Web Surveys.” Social Science Computer Review 36(6):647-668. DOI: https://doi.org/10.1177/0894439317735307.
    Search in Google ScholarBack to article
  32. Morgan, S.L., and E.S. Taylor Poppe. 2015. “A Design and a Model For Investigating the Heterogeneity of Context Effects in Public Opinion Surveys.” Sociological Methodology 45(1): 184–222. DOI: https://doi.org/10.1177/0081175015570095.
    Search in Google ScholarBack to article
  33. Ng, J.H., E. Henry, T. Oberlander, P. Shi, S.H. Scholle. 2016. “Shortening a Patient Experiences Survey for Medical Homes.” Healthcare 4:1. DOI: https://doi.org/10.3390/healthcare4010001.493453527417589
    Search in Google ScholarBack to article
  34. Paddison, C.A.M., G.A. Abel, M.O. Roland, M.N. Elliott, G. Lyratzopoulos, and J.L. Campbell. 2013. “Drivers of Overall Satisfaction with Primary Care: Evidence from the English General Practice Patient Survey.” Health Expectations 18: 1081-1092. DOI: https://doi.org/10.1111/hex.12081.506081223721257
    Search in Google ScholarBack to article
  35. Pew Research Center. 2019. Mobile Technology and Home Broadband. Available at: https://www.pewresearch.org/internet/wp-content/uploads/sites/9/2019/06/PI_2019.06.13_Mobile-Technology-and-Home-Broadband_FINAL2.pdf. (accessed September 2021).
    Search in Google ScholarBack to article
  36. Peytchev, A. 2009. “Survey Breakoff.” Public Opinion Quarterly 73(1): 74–97. DOI: https://doi.org/10.1093/poq/nfp014.
    Search in Google ScholarBack to article
  37. Peytchev, A. and E. Peytcheva. 2017. “Reduction of Measurement Error due to Survey Length: Evaluation of the Split Questionnaire Design Approach.” Survey Research Methods 11(4): 361–368. DOI: https://doi.org/10.18148/srm/2017.v11i4.7145.
    Search in Google ScholarBack to article
  38. Revilla, M. and C. Ochoa. 2017. “Ideal and Maximum Length for a Web Survey.” International Journal of Market Research 59(5): 557–565. DOI: https://doi.org/10.2501/IJMR-2017-039.
    Search in Google ScholarBack to article
  39. Rhemtulla, M., V. Savalei, and T.D. Little. 2016. “On the Asymptotic Relative Efficiency of Planned Missingness Designs.” Psychometrika 81(1): 60–89. DOI: https://doi.org/10.1007/s11336-014-9422-0.25217133
    Search in Google ScholarBack to article
  40. Schober, M.F., F.G. Conrad, C. Antoun, P. Ehlen, S. Fail, A.L. Hupp, M. Johnston, L. Vickers, H.Y. Yan, and C. Zhang. 2015. “Precision and Disclosure in Text and Voice Interviews on Smartphones.” PLoSONE 10. DOI: https://doi.org/10.1371/journal.pone.0128337.446518426060991
    Search in Google ScholarBack to article
  41. Schonlau, M. and M. Couper. 2017. “Options for Conducting Web Surveys.” Statistical Science 32(2): 279–292. DOI: https://doi.org/10.1214/16-STS597.
    Search in Google ScholarBack to article
  42. Sharp, L.M., and J. Frankel. 1983. “Respondent Burden: A Test of Some Common Assumptions.” Public Opinion Quarterly 47: 36–53. DOI: https://doi.org/10.1086/268765.
    Search in Google ScholarBack to article
  43. Spranca, M., D.E. Kanouse, M. Elliott, P.F. Short, D.O. Farley, and R.D. Hays. 2000. “Do Consumer Reports of Health Plan Quality Affect Health Plan Selection?” Health Services Review 35(5): 933–947. Available at: https://www.ncbi.nlm.nih.gov/pmc/articles/PMC1089177. (accessed September 2021).
    Search in Google ScholarBack to article
  44. Steinbrecher, M, J. Roßmann, and J.E. Blumenstiel. 2015. “Why Do Respondents Break Off Web Surveys and Does it Matter? Results from Four Follow-up Surveys.” International Journal of Public Opinion Research 27(2): 289–302. DOI: https://doi.org/10.1093/ijpor/edu025.
    Search in Google ScholarBack to article
  45. Stucky, B.D., R.D. Hays, M.O. Edelen, J. Gurvey, and J.A. Brown. 2016. “Possibilities for Shortening the CAHPS Clinician and Group Survey.” Medical Care 54(1): 32–37. DOI: https://doi.org/10.1097/MLR.0000000000000452.26536332
    Search in Google ScholarBack to article
  46. Swain, M.S. 2015. “The Effects of a Planned Missingness Design on Examinee Motivation and Psychometric Quality.” PhD Dissertation, James Madison University. Available at: https://www.proquest.com/docview/1685390254 (accessed September 2021).
    Search in Google ScholarBack to article
  47. Toepoel, V., and P. Lugtig. 2018. “Modularization in an Era of Mobile Web: Investigating the Effects of Cutting a Survey Into Smaller Pieces on Data Quality.” Social Science Computer Review: 1–15. DOI: https://doi.org/10.1177/0894439318784882.
    Search in Google ScholarBack to article
  48. Toepoel, V., M. Das, and A. van Soest. 2008. “Effects of Design in Web Surveys: Comparing Trained and Fresh Respondents.” Public Opinion Quarterly 72(5): 985–1007. DOI: https://doi.org/10.1093/poq/nfn060.
    Search in Google ScholarBack to article
  49. Villar, A., M. Callegaro, and Y. Yang. 2013. “Where am I? A meta-analysis of experiments on the effects of progress indicators for web surveys.” Social Science Computer Review 31(6): 744–762. DOI: https://doi.org/10.1177/0894439313497468.
    Search in Google ScholarBack to article
  50. West, B.T., D. Ghimire, and W.G. Axinn. 2015. “Evaluating a Modular Design Approach to Collecting Survey Data Using Text Messages.” Survey Research Methods 9:111–123. DOI: https://doi.org/10.18148/srm/2015.v9i2.6135.
    Search in Google ScholarBack to article
  51. Yan, T., S. Fricker, and S. Tsai. 2020. “Response Burden: What Is It and What Predicts It?” In Advances in Questionnaire Design, Development, Evaluation, and Testing, edited by P.C. Beatty, D. Collins, L. Kaye, J.L. Padilla, G. B. Willis, and A. Wilmot: 193–212. New York: Wiley.10.1002/9781119263685.ch8
    Search in Google ScholarBack to article
  52. Zhang, C., and F.G. Conrad. 2018. “Intervening to Reduce Satisficing Behaviors in Web Surveys: Evidence From Two Experiments on How it Works.” Social Science Computer Review 36(1): 57–81. DOI: https://doi.org/10.1177/0894439316683923
    Search in Google ScholarBack to article
DOI: https://doi.org/10.2478/jos-2022-0042 | Journal eISSN: 2001-7367
Journal RSS Feed
Language: English
Page range: 963 - 986
Submitted on: May 1, 2020
|
Accepted on: Dec 1, 2021
|
Published on: Dec 3, 2022
Published by: Sciendo
In partnership with: Paradigm Publishing Services
Publication frequency: 4 issues per year
Keywords:
Online data collection,
text messaging,
health
Related subjects:
Mathematics,
Probability and statistics

© 2022 Philip S. Brenner, J. Lee Hargraves, Carol Cosenza, published by Sciendo
This work is licensed under the Creative Commons Attribution-NonCommercial-NoDerivatives 3.0 License.

Previous articleVolume 38 (2022): Issue 4 (December 2022)Next article