Assessing Nonresponse in a Longitudinal Establishment Survey Using Regression Trees

Morgan Earp; Daniell Toth; Polly Phipps; Charlotte Oslund

doi:10.2478/jos-2018-0021

Abstract

This article introduces and discusses a method for conducting an analysis of nonresponse for a longitudinal establishment survey using regression trees. The methodology consists of three parts: analysis during the frame refinement and enrollment phases, common in longitudinal surveys; analysis of the effect of time on response rates during data collection; and analysis of the potential for nonresponse bias. For all three analyses, regression tree models are used to identify establishment characteristics and subgroups of establishments that represent vulnerabilities during the data collection process. This information could be used to direct additional resources to collecting data from identified establishments in order to improve the response rate.

References

Andridge, R.R. and R.J.A. Little. 2011. “Proxy Pattern-Mixture Analysis for Survey Nonresponse.” Journal of Official Statistics 27: 153–180.
Search in Google Scholar Back to article
Andridge, R.R. and K.J. Thompson. 2015. “Assessing Nonresponse Bias in a Business Survey: Proxy Pattern-Mixture Analysis for Skewed Data.” Annals of Applied Statistics 9(4): 2237–2265. Doi: http://dx.doi.org/10.1214/15-AOAS878.10.1214/15-AOAS878
Open DOI Search in Google Scholar Back to article
Atrostic, B.K., N. Bates, and A. Silberstein. 2001. “Nonresponse in US Government Household Surveys: Consistent Measures, Recent Trends, and New Insights.” Journal of Official Statistics 17: 209–226.
Search in Google Scholar Back to article
Bureau of Labor Statistics. 2015. Handbook of Methods. “Job Openings and Labor Turnover Survey” Chapter 18. Available at: https://www.bls.gov/opub/hom/pdf/homch18.pdf (accessed October 2017).
Search in Google Scholar Back to article
Davis, W.R. and N. Pihama. 2009. “Survey Response as Organisational Behaviour: an Analysis of the Annual Enterprise Survey, 2003–2007.” New Zealand Association of Economists Conference. New Zealand: New Zealand Association of Economists. Available at: http://ro.uow.edu.au/eispapers/826/ (accessed October 2017).
Search in Google Scholar Back to article
De Heer, W. and E. De Leeuw. 2002. “Trends in Household Survey Nonresponse: a Longitudinal and International Comparison.” In Survey Nonresponse, edited by R.M. Groves, D.A. Dillman, J.L. Eltinge, and R.J.A. Little, 41–54. New York: John Wiley and Sons.
Search in Google Scholar Back to article
Dillman, D.A. 2000. Mail and Internet Surveys: The Tailored Design Method (second edition). New York: John Wiley & Sons.
Search in Google Scholar Back to article
Dillman, D.A., J.D. Smyth, and L.M. Christian. 2009. Internet, Phone, Mail, and Mixed-Mode Surveys: the Tailored Design Method (third edition). Hoboken, NJ: John Wiley & Sons.
Search in Google Scholar Back to article
Freedman, D.S., A. Thornton, and D. Camburn. 1980. “Maintaining Response Rates in Longitudinal Studies.” Sociological Methods & Research 9: 87–98.10.1177/004912418000900104
Search in Google Scholar Back to article
Groves, R.M. 2006. “Nonresponse Rates and Nonresponse Bias in Household Surveys.” Public Opinion Quarterly 70: 646–675.10.1093/poq/nfl033
Search in Google Scholar Back to article
Groves, R.M., D.A. Dillman, J.L. Eltinge, and R.J.A. Little. 2002. Survey Nonresponse. New York: John Wiley and Sons.
Search in Google Scholar Back to article
Heberlein, T.A. and R. Baumgartner. 1978. “Factors Affecting Response Rates to Mailed Questionnaires: A Quantitative Analysis of the Published Literature.” American Sociological Review 43: 447–462.10.2307/2094771
Search in Google Scholar Back to article
Holbrook, A., J.A. Krosnick, and A. Pfent. 2007. “The Causes and Consequences of Response Rates in Surveys by the News Media and Government Contractor Survey Research Firms.” In Advances in Telephone Survey Methodology, edited by J.M. Lepkowski, C. Tucker, J.M. Brick, E. De Leeuw, L. Japec, P.J. Lavrakas, M.W. Link and R.L. Sangster, 499–528. Hoboken, NJ: Wiley.10.1002/9780470173404.ch23
Search in Google Scholar Back to article
Hothorn, T., K. Hornik, and A. Zeileis. 2006. “Unbiased Recursive Partitioning: a Conditional Inference Framework.” Journal of Computational and Graphical Statistics 15: 651–674.10.1198/106186006X133933
Search in Google Scholar Back to article
Janik, F. and S. Kohaut. 2012. “Why Don’t They Answer? Unit Nonresponse in the IAB Establishment Panel.” Quality & Quantity 46: 917–934.10.1007/s11135-011-9436-y
Search in Google Scholar Back to article
Kreuter, F., K. Olson, J. Wagner, T. Yan, T.M. Ezzati-Rice, C. Casas-Cordero, M. Lemay, A. Peytchev, R.M. Groves, and T.E. Raghunathan. 2010. “Using Proxy Measures and Other Correlates of Survey Outcomes to Adjust for Non-Response: Examples from Multiple Surveys.” Journal of the Royal Statistical Society: Series A (Statistics in Society) 173: 389–407.10.1111/j.1467-985X.2009.00621.x
Search in Google Scholar Back to article
Lepkowski, J.M. and M.P. Couper. 2002. “Nonresponse in the Second Wave of Longitudinal Household Surveys.” In Survey Nonresponse, edited by R.M. Groves, D.A. Dillman, J.L. Eltinge, and R.J.A. Little, 259–272. New York: John Wiley and Sons.
Search in Google Scholar Back to article
Little, R.J.A. and S. Vartivarian. 2005. “Does Weighting for Nonresponse Increase the Variance of Survey Means?” Survey Methodology 31: 161–168.
Search in Google Scholar Back to article
Lohr, S., V. Hsu, and J. Montaquila. 2015. “Using Classification and Regression Trees to Model Survey Nonresponse.” In Joint Statistical Meetings, Proceedings of the Survey Research Methods Section: American Statistical Association. 2071–2085. Alexandria, VA: American Statistical Association. Available at: http://ww2.amstat.org/sections/srms/Proceedings/y2015f.html (accessed October 2017).
Search in Google Scholar Back to article
Morgan, J.N. and J.A. Sonquist. 1963. “Problems in the Analysis of Survey Data, and a Proposal.” Journal of the American Statistical Association 58: 415–434.10.1080/01621459.1963.10500855
Search in Google Scholar Back to article
Office of Management and Budget. 2006. Statistical Directive No. 2, Standards and Guidelines for Statistical Surveys 2006. Washington DC: OMB. Available at: https://obamawhitehouse.archives.gov/sites/default/files/omb/inforeg/statpolicy/standards_stat_surveys.pdf (accessed October 2017).
Search in Google Scholar Back to article
Paxson, M.C., D.A. Dillman, and J. Tarnai. 1995. “Improving Response to Business Mail Surveys.” In Business Survey Methods, edited by B.G. Cox, D.A. Binder, B.N. Chinnappa, A. Christianson, M.J. Colledge, and P.S. Kott, 303–316. New York: John Wiley and Sons.10.1002/9781118150504.ch17
Search in Google Scholar Back to article
Phipps, P. and D. Toth. 2012. “Analyzing Establishment Nonresponse Using an Interpretable Regression Tree Model with Linked Administrative Data.” The Annals of Applied Statistics 6: 772–794.10.1214/11-AOAS521
Search in Google Scholar Back to article
Särndal, C. 2011. “The 2010 Morris Hansen Lecture. Dealing with Survey Nonresponse in Data Collection, in Estimation.” Journal of Official Statistics 27(1): 1–21.
Search in Google Scholar Back to article
Särndal, C. and P. Lundquist. 2014. “Accuracy in Estimation with Nonresponse: A Function of Degree of Imbalance and Degree of Explanation.” Journal of Survey Statistics and Methodology 2(4): 361–387. Doi: https://doi.org/10.1093/jssam/smu014.10.1093/jssam/smu014
Open DOI Search in Google Scholar Back to article
Schouten, B., F. Cobben, and J. Bethlehem. 2009. “Indicators for the Representativeness of Survey Response.” Survey Methodology 35(1): 101–113.
Search in Google Scholar Back to article
Schouten, B., N. Shlomo, and C. Skinner. 2011. “Indicators for Monitoring and Improving Representativeness of Response.” Journal of Official Statistics 27(2): 231–253.
Search in Google Scholar Back to article
Seiler, C. 2010. Dynamic Modelling of Nonresponse in Business Surveys. No. 93. Ifo Working Paper. Available at: https://econpapers.repec.org/paper/cesifowps/_5f93.htm (accessed October 2017).
Search in Google Scholar Back to article
Singer, E. 2002. “The Use of Incentives to Reduce Nonresponse in Household Surveys.” In Survey Nonresponse, edited by R.M. Groves, D.A. Dillman, J.L. Eltinge, and R.J.A. Little, 163–177. New York: John Wiley and Sons.
Search in Google Scholar Back to article
Smaill, K. 2012. “Trajectory Modelling of Longitudinal Non-Response in Business Surveys.” Statistical Journal of the International Association of Official Statistics 28: 137–144.
Search in Google Scholar Back to article
Su, X., M. Wang, and J. Fan. 2004. “Maximum Likelihood Regression Trees.” Journal of Computational and Graphical Statistics 13: 586–598.10.1198/106186004X2165
Search in Google Scholar Back to article
Tomaskovic-Devey, D., J. Leiter, and S. Thompson. 1994. “Organizational Survey Nonresponse.” Administrative Science Quarterly 39: 439–457.10.2307/2393298
Search in Google Scholar Back to article
Toth, D. 2017. rpms: Recursive Partitioning for Modeling Survey Data. R package version 0.2.0. Available at: https://CRAN.R-project.org/package=rpms (accessed October 2017).10.32614/CRAN.package.rpms
Search in Google Scholar Back to article
Wagner, J. 2010. “The Fraction of Missing Information as a Tool for Monitoring the Quality of Survey Data.” Public Opinion Quarterly 74: 223–243. Doi: http://dx.doi.org/10.1093/poq/nfq007.10.1093/poq/nfq007
Open DOI Search in Google Scholar Back to article
Wagner, J. 2012. “A Comparison of Alternative Indicators for the Risk of Nonresponse Bias.” Public Opinion Quarterly 76: 555–575. Doi: http://dx.doi.org/10.1093/poq/nfs032.10.1093/poq/nfs032627697530538342
Open DOI Search in Google Scholar Back to article
Watson, N. and M. Wooden. 2009. “Identifying Factors Affecting Longitudinal Survey Response.” In Methodology of Longitudinal Surveys, edited by Peter Lynn, 151–181. Hoboken, NJ: John Wiley and Sons.10.1002/9780470743874.ch10
Search in Google Scholar Back to article

Assessing Nonresponse in a Longitudinal Establishment Survey Using Regression Trees

Abstract

Paradigm

My account