Have a personal or library account? Click to login
Estimating Components of Mean Squared Error to Evaluate the Benefits of Mixing Data Collection Modes Cover

Estimating Components of Mean Squared Error to Evaluate the Benefits of Mixing Data Collection Modes

Open Access
|Jun 2017

References

  1. Biemer, P.P. 1988. “Measuring Data Quality.” In Telephone Survey Methodology, edited by R.M. Groves, P.P. Biemer, L. Lyberg, J.T. Massey, W. II, Nicholls, and J. Waksberg, 341–375. New York: Wiley.
  2. Biemer, P.P. 2010. “Total Survey Error: Design, Implementation, and Evaluation.” Public Opinion Quarterly 74: 817–848. Doi: http://dx.doi.org/10.1093/poq/nfq058.10.1093/poq/nfq058
  3. Biemer, P.P. and L.E. Lyberg. 2003. Introduction to Survey Quality. Hoboken, NJ: Wiley.10.1002/0471458740
  4. Blumberg, S.J. and J.V. Luke. 2013. “Wireless Substitution: Early Release of Estimates from the National Health Interview Survey, July–December 2012.” National Center for Health Statistics. December. Available at: http://www.cdc.gov/nchs/data/nhis/earlyrelease/wireless201306.pdf (accessed March 2017).
  5. Brick, J.M. and D. Williams. 2013. “Explaining Rising Nonresponse Rates in Cross-Sectional Surveys.” The ANNALS of the American Academy of Political and Social Science 645: 36–59. Doi: http://dx.doi.org/10.1177/0002716212456834.10.1177/0002716212456834
  6. Carley-Baxter, L.R., A. Peytchev, and M.C. Black. 2010. “Comparison of Cell Phone and Landline Surveys: A Design Perspective.” Field Methods 22(1): 3–15. Doi: http://dx.doi.org/10.1177/1525822X09360310.10.1177/1525822X09360310
  7. Chang, L. and J.A. Krosnick. 2009. “National surveys via RDD telephone interviewing versus the internet. Comparing sample representativeness and response quality.” Public Opinion Quarterly 73: 641–678. Doi: http://dx.doi.org/10.1093/poq/nfp075.10.1093/poq/nfp075
  8. Couper, M.P. 2011. “The Future of Modes of Data Collection.” Public Opinion Quarterly 75(5): 889–908. Doi: http://dx.doi.org/10.1093/poq/nfr046.10.1093/poq/nfr046
  9. De Leeuw, E. 2005. “To Mix or Not to Mix Data Collection Modes in Surveys.” Journal of Official Statistics 21: 233–255.
  10. De Leeuw, E.D. and N. Berzelak. 2017. “Survey mode or survey modes?” In The Sage Handbook of Survey Methodology, edited by C. Wolf, D. Joye, T.W. Smith, and Y.-C. Fu, 142–156. London: Sage Publications.10.4135/9781473957893.n11
  11. De Leeuw, E. and W. de Heer. 2002. “Trends in Household Survey Nonresponse: A Longitudinal and International Comparison.” In Survey nonresponse, edited by R. Groves, D. Dillman, J. Eltinge, and R.J.A. Little, 41–54. New York: Wiley.
  12. De Leeuw, E.D., J.J. Hox, and D.A. Dillman. 2008. “Mixed mode surveys: When and why.” In International Handbook of Survey Methodology, edited by E.D. de Leeuw, J.J. Hox, and D.A. Dillman, 299–316. New York/London: Erlbaum/Taylor & Francis.
  13. Dillman, D.A., G. Phelps, R. Tortora, K. Swift, J. Kohrell, J. Berck, and B.L. Messer. 2009. “Response Rate and Measurement Differences in Mixed-Mode Surveys using Mail, Telephone, Interactive Voice Response (IVR) and the Internet.” Social Science Research 38: 1–18. Doi: http://dx.doi.org/10.1016/j.ssresearch.2008.03.007.10.1016/j.ssresearch.2008.03.007
  14. Dillman, D.A., J.D. Smyth, and L.M. Christian. 2014. Internet, Phone, Mail, and Mixedmode Surveys: the Tailored Design Method (4th Edition). Hoboken: Wiley.10.1002/9781394260645
  15. Eva, G., G. Loosveldt, P. Lynn, P. Martin, M. Revilla, W. Saris, and J. Vannieuwenhuyze. 2010. Assessing the Cost-Effectiveness of Different Modes for ESS Data Collection. London: City University.
  16. Fowler, F.J., P.M. Gallagher, V.L. Stringfellow, A.M. Zaslavsky, J.W. Thompson, and P.D. Cleary. 2002. “Using Telephone Interviews to Reduce Nonresponse Bias to Mail Surveys of Health Plan Members.” Medical Care 40: 190–200.10.1097/00005650-200203000-0000311880792
  17. Gordoni, G., P. Schmidt, and Y. Gordoni. 2012. “Measurement invariance across face-to-face and telephone modes: the case of minority-status collectivistic oriented groups.” International Journal of Public Opinion Research 24(2): 185–207. Doi: http://dx.doi.org/10.1093/ijpor/edq054.10.1093/ijpor/edq054
  18. Greene, J., H. Speizer, and W. Wiitala. 2008. “Telephone and Web: Mixed-Mode Challenge.” Health Services Research 43: 230–248. Doi: http://dx.doi.org/10.1111/j.1475-6773.2007.00747.x.10.1111/j.1475-6773.2007.00747.x232313918211527
  19. Groves, R.M. 1989. Survey Errors and Survey Costs. New York: Wiley.10.1002/0471725277
  20. Groves, R.M. 2006. “Nonresponse Rates and Nonresponse Bias in Household Surveys.” Public Opinion Quarterly 70: 646–675.10.1093/poq/nfl033
  21. Groves, R.M. 2011. “Three Eras of Survey Research.” Public Opinion Quarterly 75(5): 861–971. Doi: http://dx.doi.org/10.1093/poq/nfl033.10.1093/poq/nfl033
  22. Groves, R.M., F.J. Fowler Jr., M.P. Couper, J.M. Lepkowski, E. Singer, and R. Tourangeau. 2009. Survey Methodology (Wiley Series in Survey Methods), 2nd Ed. Hoboken, NJ: John Wiley & Sons.
  23. Groves, R.M. and L.J. Magilavy. 1984. “An Experimental Measurement of Total Survey Error.” In Proceedings of the Section on Survey Research Methods: American Statistical Association, 698–703. Alexandria, VA: American Statistical Association. Available at: http://ww2.amstat.org/sections/srms/Proceedings/ (accessed March 2017).
  24. Heerwegh, D. and G. Loosveldt. 2011. “Assessing mode effects in a national crime victimization survey using structural equation models: social desirability bias and acquiescence.” Journal of Official Statistics 27: 49–63.
  25. Hochstim, J.R. 1967. “A Critical Comparison of Three Strategies of Collecting Data from Households.” Journal of the American Statistical Association 62: 976–989. Doi: http://dx.doi.org/10.2307/2283686.10.2307/2283686
  26. Holbrook, A., M. Green, and J. Krosnick. 2003. “Telephone Versus Face-to-face Interviewing of National Probability Samples with Long Questionnaires.” Public Opinion Quarterly 67: 79–125. Doi: http://dx.doi.org/10.1086/346010.10.1086/346010
  27. Hox, J., E.D. de Leeuw, and T. Klausch. 2017. “Mixed mode research: Issues in design and analysis.” In Total Survey Error in Practice: Improving Quality in the Era of Big Data, edited by P.P. Biemer, E.D. de Leeuw, S. Eckman, B. Edwards, F. Kreuter, L.E. Lyberg, C. Tucker, and B.T. West, 511–530. Hoboken, NJ: John Wiley and Sons, Inc.10.1002/9781119041702.ch23
  28. Hox, J.J., E.D. de Leeuw, and E.A.O. Zijlmans. 2015. “Measurement equivalence in mixed mode surveys.” Frontiers in Psychology: Quantitative Psychology and Measurement. Doi: http://dx.doi.org/10.3389/fpsyg.2015.00087.10.3389/fpsyg.2015.00087431828225699002
  29. Kappelhof, J.W.S. 2013. “The Effect of Different Survey Designs on Nonresponse in Surveys of Non-Western Minorities in The Netherlands.” Survey Research Methods 8(2): 81–98. Doi: http://dx.doi.org/10.18148/srm/2014.v8i2.5784.
  30. Klausch, L.T., J.J. Hox, and B. Schouten. 2013. “Measurement effects of survey mode on the equivalence of attitudinal rating scale questions.” Sociological Methods and Research 42: 227–263. Doi: http://dx.doi.org/10.1177/0049124113500480.10.1177/0049124113500480
  31. Klausch, T., J.J. Hox, and B. Schouten. 2015a. “Selection Error in Single- and Mixed Mode Surveys of the Dutch General Population.” Journal of the Royal Statistical Society Series A 178: 945–961. Doi: http://dx.doi.org/10.1111/rssa.12102.10.1111/rssa.12102
  32. Klausch, T., B. Schouten, and J.J. Hox. 2014. “The Use of Within-subject Experiments for Estimating Measurement Effects in Mixed-mode Surveys.” Statistics Netherlands Discussion Paper, 2015/06. Available at: https://www.cbs.nl/en-gb/background/2014/11/the-use-of-within-subject-experiments-for-estimating-measurement-effects-in-mixed-mode-surveys (accessed March 2017).
  33. Klausch, T., B. Schouten, and J.J. Hox. 2015b. “Evaluating Bias of Sequential Mixedmode Designs Against Benchmark Surveys.” Sociological Methods and Research 1–34. Doi: http://dx.doi.org/10.1177/0049124115585362.10.1177/0049124115585362
  34. Kreuter, F., S. Presser, and R. Tourangeau. 2008. “Social Desirability Bias in CATI, IVR, and Web Surveys: The Effects of Mode and Question Sensitivity.” Public Opinion Quarterly 72: 847–865. Doi: http://dx.doi.org/10.1093/poq/nfn063.10.1093/poq/nfn063
  35. Kreuter, F., G. Müller, and M. Trappmann. 2010. “Nonresponse and Measurement Error in Employment Research: Making Use of Administrative Data.” Public Opinion Quarterly 74: 880–906. Doi: http://dx.doi.org/10.1093/poq/nfq0.10.1093/poq/nfq060
  36. Link, M.W. and A.H. Mokdad. 2006. “Can Web and Mail Survey Modes Improve Participation in an RDD-Based National Health Surveillance?” Journal of Official Statistics 22: 293–312.
  37. Lipps, O., N. Pekari, and C. Roberts. 2015. “Undercoverage and Nonresponse in a List-Sampled Telephone Election Study.” Survey Research Methods 9(2): 71–82. Doi: http://dx.doi.org/10.18148/srm/2015.v9i2.6139.
  38. Lynn, P. 2013. “Alternative Sequential Mixed-Mode Designs: Effects on Attrition Rates, Attrition Bias, and Costs.” Journal of Survey Statistics and Methodology 1: 183–205. Doi: http://dx.doi.org/10.1093/jssam/smt015.10.1093/jssam/smt015
  39. Massey, D.S. and R. Tourangeau. 2013. “Where Do We Go from Here? Nonresponse and Social Measurement.” The Annals of the American Academy of Political and Social Science 645(1): 222–236. Doi: http://dx.doi.org/10.1177/0002716212464191.10.1177/0002716212464191425747725484368
  40. Millar, M.M. and D.A. Dillman. 2011. “Improving Response to Web and Mixed-Mode Surveys.” Public Opinion Quarterly 75(2): 249–269. Doi: http://dx.doi.org/10.1093/poq/nfr003.10.1093/poq/nfr003
  41. Olson, K. 2006. “Survey Participation, Nonresponse Bias, Measurement Error Bias, and Total Bias.” Public Opinion Quarterly 70(5): 737–758. Doi: http://dx.doi.org/10.1093/poq/nfl038.10.1093/poq/nfl038
  42. Olson, K., J.D. Smyth, and H. Wood. 2012. “Does Providing Sample Members with Their Preferred Survey Mode Really Increase Participation Rates?” Public Opinion Quarterly 76(4): 611–635. Doi: http://dx.doi.org/10.1093/poq/nfs024.10.1093/poq/nfs024
  43. Peytchev, A., R.K. Baxter, and L.R. Carley-Baxter. 2009. “Not All Survey Effort is Equal. Reduction of Nonresponse Bias and Nonresponse Error.” Public Opinion Quarterly 73(4): 785–806. Doi: http://dx.doi.org/10.1093/poq/nfp037.10.1093/poq/nfp037
  44. Peytcheva, E. and R.M. Groves. 2009. “Using Variation in Response Rates of Demographic Subgroups as Evidence of Nonresponse Bias in Survey Estimates.” Journal of Official Statistics 25(2): 193–201.
  45. Rao, J.N.K. and A.J. Scott. 1987. “On simple adjustments to chi-square tests with sample survey data.” The Annals of Statistics 15(1): 385–397. Doi: http://dx.doi.org/10.1214/aos/1176348654.10.1214/aos/1176348654
  46. Roberts, C., D. Joye, M. Ernst Stähli, and R. Sanchez Tome. 2016. Mixing modes of data collection in Swiss social surveys: Methodological Report of the LIVES-FORS Mixed Mode Experiment. LIVES Working Paper Series, 2016/48. Doi: http://dx.doi.org/10.12682/lives.2296-1658.2016.48.
  47. Sakshaug, J.W., T. Yan, and R. Tourangeau. 2010. “Nonresponse Error, Measurement Error, and Mode of Data Collection: Tradeoffs in a Multi-mode Survey of Sensitive and Non-sensitive Items.” Public Opinion Quarterly 74(5): 907–933. Doi: http://dx.doi.org/10.1093/poq/nfq057.10.1093/poq/nfq057
  48. Schouten, B., J. van den Brakel, B. Buelens, J. van der Laan, and T. Klausch. 2013. “Disentangling Mode-Specific Selection and Measurement Bias in Social Surveys.” Social Science Research 42(6): 1555–1570. Doi: http://dx.doi.org/10.1016/j.ssresearch.2013.07.005.10.1016/j.ssresearch.2013.07.00524090851
  49. Siemiatycki, J. 1979. “A Comparison of Mail, Telephone, and Home Interview Strategies for Household Health Surveys.” American Journal of Public Health 69: 238–245.10.2105/AJPH.69.3.238
  50. Smith, T.W. 2011. “Refining the Total Survey Error Perspective.” International Journal of Public Opinion Research 23(4): 464–484. Doi: http://dx.doi.org/10.1093/ijpor/edq052.10.1093/ijpor/edq052
  51. Suzer-Gurtekin, Z., S. Heeringa, and R. Vaillant. 2012. “Investigating the Bias of Alternative Statistical Inference Methods in Sequential Mixed-mode Surveys.” Proceedings of the JSM, Section on Survey Research Methods 4711-2. Available at: https://www.niss.org/sites/default/files/VII%201%20Suzer-Gurtekin_itsew2013.pdf (accessed April 2016).
  52. Tourangeau, R. 2017. “Mixing modes: Tradeoffs among coverage, nonresponse, and measurement error.” In Total Survey Error in Practice: Improving Quality in the Era of Big Data, edited by P.P. Biemer, E.D. de Leeuw, S. Eckman, B. Edwards, F. Kreuter, L.E. Lyberg, C. Tucker, and B.T. West. 115–132. Hoboken, NJ: John Wiley and Sons, Inc.10.1002/9781119041702.ch6
  53. Tourangeau, R., R.M. Groves, and C.D. Redline. 2010. “Sensitive topics and reluctant respondents: Demonstrating a link between nonresponse bias and measurement error.” Public Opinion Quarterly 74(3): 413–432. Doi: http://dx.doi.org/10.1093/poq/nfq004.10.1093/poq/nfq004
  54. Vannieuwenhuyze, J.T.A., G. Loosveldt, and G. Molenberghs. 2010. “A Method for Evaluating Mode Effects in Mixed-mode Surveys.” Public Opinion Quarterly 74: 1027–1045. Doi: http://dx.doi.org/10.1093/poq/nfq059.10.1093/poq/nfq059
  55. Vannieuwenhuyze, J.T.A. 2014. “On the Relative Advantage of Mixed-Mode versus Single-Mode Surveys.” Survey Research Methods 8(1): 31–42. Doi: http://dx.doi.org/10.18148/srm/2014.v8i1.5500#sthash.xSmtK1fH.dpuf.
  56. Vannieuwenhuyze, J.T.A. and G. Loosveldt. 2012. “Evaluating Relative Mode Effects in Mixed-mode Surveys: Three Methods to Disentangle Selection and Measurement Effects.” Sociological Methods and Research 42: 82–104. Doi: http://dx.doi.org/10.1177/0049124112464868.10.1177/0049124112464868
  57. Wagner, J., J. Arrieta, H. Guyer, and M.B. Ofstedal. 2014. “Does Sequence Matter in Multimode Surveys: Results from an Experiment.” Field Methods 26(2): 141–155. Doi: http://dx.doi.org/10.1177/1525822X13491863.10.1177/1525822X13491863399248024764767
Language: English
Page range: 303 - 334
Submitted on: Feb 1, 2016
|
Accepted on: Apr 1, 2017
|
Published on: Jun 12, 2017
Published by: Sciendo
In partnership with: Paradigm Publishing Services
Publication frequency: 4 issues per year

© 2017 Caroline Roberts, Caroline Vandenplas, published by Sciendo
This work is licensed under the Creative Commons Attribution-NonCommercial-NoDerivatives 3.0 License.