Setting M-Estimation Parameters for Detection and Treatment of Influential Values

Mary H. Mulry; Stephen Kaputa; Katherine J. Thompson

doi:10.2478/jos-2018-0022

Abstract

Recent research on the use of M-estimation methodology for detecting and treating verified influential values in economic surveys found that initial parameter settings affect effectiveness. In this article, we explore the basic question of how to develop initial settings for the M-estimation parameters. The economic populations that we studied are highly skewed and are consequently highly stratified. While we investigated settings for several parameters, the most challenging problem was to develop an “automatic” data-driven method for setting the initial value of the tuning constant φ, the parameter with the greatest influence on performance of the algorithm. Of all the methods that we considered, we found that methods defined in terms of the accuracy of published estimates can be implemented on a large scale and yielded the best performance. We illustrate the methodology with an empirical analysis of 36 consecutive months of data from 19 industries in the Monthly Wholesale Trade Survey.

References

Beaumont, J.-F. 2004. “Robust Estimation of a Finite Population Total in the Presence of Influential Units.” Report for the Office for National Statistics. Newport, U.K.: Office for National Statistics.
Search in Google Scholar Back to article
Beaumont, J.-F. and A. Alavi. 2004. “Robust Generalized Regression Estimation.” Survey Methodology 30(2): 195–208. Available at: http://www.statcan.gc.ca/pub/12-001-x/2004002/article/7752-eng.pdf (accessed August 2017).
Search in Google Scholar Back to article
Beaumont, J.-F., D. Haziza, and A. Ruiz-Gazen. 2013. “A Unified Approach to Robust Estimation in Finite Population Sampling.” Biometrika 100: 555–569. Doi: https://doi.org/10.1093/biomet/ast010.10.1093/biomet/ast010
Open DOI Search in Google Scholar Back to article
Binder, D.A., C. Babyak, M. Brodeur, M. Hidiroglou, and W. Jocelyn. 2000. “Variance Estimation for Two-Phase Stratified Sampling.” The Canadian Journal of Statistics 28(4): 751–764. Doi: https://doi.org/10.2307/3315914.10.2307/3315914
Open DOI Search in Google Scholar Back to article
Clark, R.G. 1995. “Winsorization Methods in Sample Surveys.” Masters Thesis. Department of Statistics. Canberra: Australia National University. Available at: http://hdl.handle.net/10440/1031 (accessed August 2017).
Search in Google Scholar Back to article
Clark, R.G., P. Kokic, and P.A. Smith. 2017 “A Comparison of two Robust Estimation Methods for Business Surveys.” International Statistical Review 85: 270–289. Doi: https://doi.org/10.1111/insr.12177.10.1111/insr.12177
Open DOI Search in Google Scholar Back to article
Duchesne, P. 1999. “Robust Calibration Estimators.” Survey Methodology 25(1): 43–56. Available at: http://hbanaszak.mjr.uw.edu.pl/TempTxt/Duchesne_1999_RobustCalibrationEstimators.PDF (accessed August 2017).10.1023/A:1005963508085
Search in Google Scholar Back to article
Gwet, J.-P. and L.-P. Rivest. 1992. “Outlier Resistant Alternatives to Ratio Estimator.” Journal of the American Statistical Association 87: 1174–1182. Available at: https://www.mat.ulaval.ca/fileadmin/mat/documents/lrivest/Publications/26-GwetRivest1992.pdf (accessed August 2017).10.1080/01621459.1992.10476275
Search in Google Scholar Back to article
Hampel, F.R., E.M. Ronchetti, P.J. Rousseeuw, and S.A. Werner. 1986. Robust Statistics. An Approach Based on Influence Functions. New York, NY: John Wiley & Sons.
Search in Google Scholar Back to article
Huang, E.T. 1984. “An Imputation Study for the Monthly Retail Trade Survey.” In JSM Proceedings, Survey Research Methods Section, American Statistical Association, Philadelphia, Pennsylvania, August 13–16, 1984. Alexandria, VA: American Statistical Association. 610–615. Available at: http://ww2.amstat.org/sections/srms/Proceedings/papers/1984_117.pdf (accessed August 2017).
Search in Google Scholar Back to article
Huang, E.T. 1986. “Report on the Imputation Research for the Monthly Retail Trade Survey.” Statistical Research Report Series No. CENSUS/SRD/RR-86-09. U.S. Census Bureau. Washington, DC. Available at: https://www.census.gov/srd/papers/pdf/rr86-09.pdf (accessed August 2017).
Search in Google Scholar Back to article
Huber, P.J. 1964. “Robust Estimation of a Location Parameter.” Annals of Mathematical Statistics 35: 73–101. Doi: https://doi.org/10.1214/aoms/1177703732.10.1214/aoms/1177703732
Open DOI Search in Google Scholar Back to article
Hulliger, B. 1995. “Outlier Robust Horvitz-Thompson Estimators.” Survey Methodology 21(1): 79–87. Available at: http://www.statcan.gc.ca/pub/12-001-x/1995001/article/14407-eng.pdf (accessed August 2017).
Search in Google Scholar Back to article
Hulliger, B. 1999. “Simple and Robust Estimators for Sampling.” In JSM Proceedings, Survey Research Methods Section, American Statistical Association, Baltimore, MD, August 8–12. Alexandria, VA: American Statistical Association. 54–63. Available at: http://ww2.amstat.org/sections/srms/Proceedings/papers/1999_009.pdf (accessed August 2017).
Search in Google Scholar Back to article
Martinoz, C.F., D. Haziza, and J.-F. Beaumont. 2015. “A Method of Determining the Winsorization Threshold, with an Application to Domain Estimation.” Survey Methodology 41(1): 57–77.
Search in Google Scholar Back to article
Mulry, M.H. and R. Feldpausch. 2007. “Investigation of Treatment of Influential Values.” Proceedings of the Third International Conference on Establishment Surveys, Montreal, Quebec, Canada, June 18–21, 2007. Alexandria, VA: American Statistical Association. 1173–1179. Available at: http://ww2.amstat.org/meetings/ices/2007/proceedings/ICES2007-000229.PDF (accessed April 2018).
Search in Google Scholar Back to article
Mulry, M.H., S. Kaputa, and K.J. Thompson. 2016. “A Cautionary Note on Clark Winsorization.” Survey Methodology 42(2): 297–305. Available at: http://www.statcan.gc.ca/pub/12-001-x/2016002/article/14676-eng.pdf (accessed August 2017).
Search in Google Scholar Back to article
Mulry, M.H., B. Oliver, and S. Kaputa. 2014. “Detecting and Treating Verified Influential Values in a Monthly Retail Trade Survey.” Journal of Official Statistics 30(4): 721–747. Doi: http://dx.doi.org/10.2478/JOS-2014-0045.10.2478/JOS-2014-0045
Open DOI Search in Google Scholar Back to article
Mulry, M.H., B. Oliver, S. Kaputa, and K.J. Thompson. 2013. “Setting M-Estimation Parameters for Detection and Treatment of Influential Values.” In JSM Proceedings, Survey Research Methods Section, American Statistical Association, Montréal, Quebéc, Canada, August 3–8, 2013. Alexandria, VA: American Statistical Association. 1424–1438. Available at: http://ww2.amstat.org/sections/srms/Proceedings/y2013/Files/308309_80627.pdf (accessed August 2017).
Search in Google Scholar Back to article
Mulry, M.H., B. Oliver, and S. Kaputa. 2012. “Several Scenarios for Influential Values in Business Surveys and Methods for Their Treatment.” In JSM Proceedings, Survey Research Methods Section, American Statistical Association, San Diego, CA, July 28–August 2, 2012. Alexandria, VA: American Statistical Association. 4015–4029. Available at: http://ww2.amstat.org/sections/srms/Proceedings/y2012/Files/304652_73493.pdf (accessed August 2017).
Search in Google Scholar Back to article
Särndal, C.-E., B. Swensson, and J. Wretman. 1992. Model Assisted Survey Sampling. Springer-Verlag. New York, NY.10.1007/978-1-4612-4378-6
Search in Google Scholar Back to article
SAS/STAT(R) 9.22 User’s Guide. Web. 06 Apr. 2016.
Search in Google Scholar Back to article
Thompson, K.J. and R.S. Sigman. 1996. Evaluation of Statistical Methods for Developing Ratio Edit Module Parameters. Technical report #ESM-9610. Washington, DC: U.S. Bureau of the Census. Available upon request.
Search in Google Scholar Back to article
Wang, Y.-G., X. Lin, M. Zhu, and Z. Bai. 2007. “Robust Estimation Using the Huber Function with a Data-Dependent Tuning Constant.” Journal of Computational and Graphical Statistics 16(2): 468–481.Doi: http://dx.doi.org/10.1198/106186007X180156.10.1198/106186007X180156
Open DOI Search in Google Scholar Back to article
White, H. 1980. “A Heteroskedasticity-Consistent Covariance Matrix Estimator and a Direct Test for Heteroskedasticity.” Econometrica 48(4): 817–838. Doi: https://doi.org/10.2307/1912934.10.2307/1912934
Open DOI Search in Google Scholar Back to article

Setting M-Estimation Parameters for Detection and Treatment of Influential Values

Abstract

Paradigm

My account