Dynamic Question Ordering in Online Surveys
References
- Achtyes, E.D., S. Halstead, L. Smart, T. Moore, E. Frank, D.J. Kupfer, and R. Gibbons. 2015. “Validation of Computerized Adaptive Testing in an Outpatient Nonacademic Setting: The VOCATIONS Trial.”
Psychiatric Services 66(10): 1091–1096. Doi:10.1176/appi.ps.201400390">http://dx.doi.org/10.1176/appi.ps.201400390 . - Almirall, D., S.N. Compton, M. Gunlicks-Stoessel, N. Duan, and S.A. Murphy. 2012. “Designing a Pilot Sequential Multiple Assignment Randomized Trial for Developing an Adaptive Treatment Strategy.”
Statistics in Medicine 31(17): 1887–1902. Doi:10.1002/sim.4512">http://dx.doi.org/10.1002/sim.4512 . - Angelovska, J. and P.M. Mavrikiou. 2013.
Can Creative Web Survey Questionnaire Design Improve the Response Quality? University of Amsterdam, AIAS Working Paper, 131. Available at:http://archive.uva-aias.net/uploadedfiles/publications/AIASWP131-1.pdf (accessed April 14, 2017). - Barge, S. and H. Gehlbach. 2012. “Using the Theory of Satisficing to Evaluate the Quality of Survey Data.”
Research in Higher Education 53(2): 182–200. Doi:10.1007/s11162-011-9251-2">http://dx.doi.org/10.1007/s11162-011-9251-2 . - Benedetto, G., M. Stinson, and J.M. Abowd. 2013.
The Creation and Use of the SIPP Synthetic Beta. U.S. Census Bureau . Available at:http://www.census.gov/content/dam/Census/programs-surveys/sipp/methodology/SSBdescribenontechnical.pdf (accessed April 14, 2017). - Bouamrane, M.-M., A. Rector, and M. Hurrell. 2008.
Gathering Precise Patient Medical History with an Ontology-Driven Adaptive Questionnaire . In Proceedings of the 21st IEEE International Symposium on Computer-Based Medical Systems, 539–541, 17–19 June 2008, Jyväskylä, Finland. Doi:10.1109/CBMS.2008.24">http://dx.doi.org/10.1109/CBMS.2008.24 . - Bradburn, N. 1978. “Respondent Burden.” In Proceedings of the American Statistical Association, Survey Research Methods Section, 35–40. Available at:
http://ww2.amstat.org/sections/SRMS/Proceedings/papers/1978_007.pdf (accessed April 14, 2017). - Bureau of Justice Statistics. 2014.
National Crime and Victimization Survey: Technical Documentation . Available at:http://www.bjs.gov/content/pub/pdf/ncvstd13.pdf (accessed April 14, 2017). - Clark, S.L. 2014.
American Community Survey Item Nonresponse Rates: Mail versus internet . American Community Survey Research and Evaluation Program (March). Available at:https://www.census.gov/content/dam/Census/library/working-papers/2014/acs/2014Clark01.pdf (accessed April 14, 2017). - Cohn, D.A., Z. Ghahramani, and M.I. Jordan. 1996. “Active Learning with Statistical Models.”
Journal of Artificial Intelligence Research 4: 129–145. - Collins, L.M., S.A. Murphy, V.N. Nair, N. Vijay, and V.J. Strecher. 2005. “A Strategy for Optimizing and Evaluating Behavioral Interventions.”
Annals of Behavioral Medicine 30(1): 65–73. Doi:10.1207/s15324796abm3001_8">http://dx.doi.org/10.1207/s15324796abm3001_8 . - Collins, L.M., S.A. Murphy, and V. Strecher. 2007. “The Multiphase Optimization Strategy (MOST) and the Sequential Multiple Assignment Randomized Trial (SMART): New Methods for More Potent eHealth Interventions.”
American Journal of Preventive Medicine 32(5): S112–S118. Doi:10.1016/j.amepre.2007.01.022">http://dx.doi.org/10.1016/j.amepre.2007.01.022 . - Cook, C., F. Heath, and R.L. Thompson. 2000. “A Meta-Analysis of Response Rates in Web- or Internet-Based Surveys.”
Educational and Psychological Measurement 60(6): 821–836. Doi:10.1177/00131640021970934">http://dx.doi.org/10.1177/00131640021970934 . - Couper, M.P., G.L. Alexander, N. Zhang, R.J.A. Little, N. Maddy, M.A. Nowak, J.B. McClure, J.J. Calvi, S.J. Rolnick, M.A. Stopponi, and C.C. Johnson. 2010. “Engagement and Retention: Measuring Breadth and Depth of Participant Use of an Online Intervention.”
Journal of Medical Internet Research 12(4): e52. Doi:10.2196/jmir.1430">http://dx.doi.org/10.2196/jmir.1430 . - Cover, T.M. and P.E. Hart. 1967. “Nearest Neighbor Pattern Classification.”
IEEE Transactions on Information Theory 13(1): 21–27. Doi:10.1109/tit.1967.1053964">http://dx.doi.org/10.1109/tit.1967.1053964 . - Cover, T.M. and J.A. Thomas. 2012.
Elements of Information Theory . John Wiley & Sons. Doi:10.1002/047174882x">http://dx.doi.org/10.1002/047174882x . - Dietz, T., G.T. Gardner, J. Gilligan, P.C. Stern, and M.P. Vandenbergh. 2009. “Household Actions Can Provide a Behavioral Wedge to Rapidly Reduce U.S. Carbon Emissions.” In Proceedings of the National Academy of Sciences, November 3, 2009. 106: 18452–18456. National Academy of Sciences. Doi:
10.1073/pnas.0908738106">http://dx.doi.org/10.1073/pnas.0908738106 . - Early, K., S. Fienberg, and J. Mankoff. 2016. “Test-Time Feature Ordering with FOCUS: Interactive Predictions with Minimal User Burden.” In Proceedings of the 2016 ACM International Joint Conference on Pervasive and Ubiquitous Computing, September 12–16, Heidelberg, Germany, 992–1003. Doi:
10.1145/2971648.2971748">http://dx.doi.org/10.1145/2971648.2971748 . - Ferrucci, D., A. Levas, S. Bagchi, D. Gondek, and E.T. Mueller. 2013. “Watson: Beyond Jeopardy!”
Artificial Intelligence 199: 93–105. Doi:10.1016/j.artint.2012.06.009">http://dx.doi.org/10.1016/j.artint.2012.06.009 . - Fliege, H., J. Becker, O.B. Walter, J.B. Bjorner, B.F. Klapp, and M. Rose. 2005. “Development of a Computer-Adaptive Test for Depression (D-CAT).”
Quality of Life Research 14(10): 2277–2291. Doi:10.1007/s11136-005-6651-9">http://dx.doi.org/10.1007/s11136-005-6651-9 . - Fricker, S., T. Yan, and S. Tsai. 2014. “Response Burden: What Predicts it and who is Burdened Out.” In Proceedings of the American Association for Public Opinion Research, May 15–18, 2014, Anaheim, California. 4568–4577. Available at:
http://ww2.amstat.org/sections/SRMS/Proceedings/y2014/Files/400298500838.pdf (accessed April 14, 2017). - Fuller, W.A. 2009.
Measurement Error Models . New York: Wiley. Doi:10.1002/9780470316665">http://dx.doi.org/10.1002/9780470316665 . - Galesic, M. 2006. “Dropouts on the Eeb: Effects of Interest and Burden Experienced During an Online Survey.”
Journal of Official Statistics 22(2): 313–328. - Galesic, M. and M. Bosnjak. 2009. “Effects of Questionnaire Length on Participation and Indicators of Response Quality in a Web Survey.”
Public Opinion Quarterly 73(2): 349–360. Doi:10.1093/poq/nfp031">http://dx.doi.org/10.1093/poq/nfp031 . - Gardner, W., K. Shear, K.J. Kelleher, K.A. Pajer, O. Mammen, D. Buysse, and E. Frank. 2004. “Computerized Adaptive Measurement of Depression: A Simulation Study.”
BMC Psychiatry 4(1): 13–23. Doi:10.1186/1471-244x-4-13">http://dx.doi.org/10.1186/1471-244x-4-13 . - Gibbons, R.D., D.J. Weiss, E. Frank, and D.J. Kupfer. 2016. “Computerized Adaptive Diagnosis and Testing of Mental Health Disorders.”
Annual Review of Clinical Psychology 12(1): 83–104. Doi:10.1146/annurev-clinpsy-021815-093634">http://dx.doi.org/10.1146/annurev-clinpsy-021815-093634 . - Gibbons, R.D., D.J. Weiss, D.J. Kupfer, E. Frank, A. Fagiolini, V.J. Grochocinski, and J.C. Immekus. 2008. “Using Computerized Adaptive Testing to Reduce the Burden of Mental Health Assessment.”
Psychiatric Services 59(4): 361–368. Doi:10.1176/ps.2008.59.4.361">http://dx.doi.org/10.1176/ps.2008.59.4.361 . - Gibbons, R.D., D.J. Weiss, P.A. Pilkonis, E. Frank, T. Moore, J.B. Kim, and D.J. Kupfer. 2012. “Development of a Computerized Adaptive Test for Depression.”
Archives of General Psychiatry 69(11): 1104–1112. Doi:10.1001/archgenpsychiatry.2012.14">http://dx.doi.org/10.1001/archgenpsychiatry.2012.14 . - Griffin, D. and D. Nelson. 2014. “Reducing Respondent Burden in the ACS’s Computer Assisted Personal Visit Interviewing Operation - Phase 1 Results.” Available at:
https://www.census.gov/content/dam/Census/library/working-papers/2014/acs/2014_Griffin_02.pdf (accessed April 14, 2017). - Groves, R.M. and S.G. Heeringa. 2006. “Responsive Design for Household Surveys: Tools for Actively Controlling Survey Errors and Costs.”
Journal of the Royal Statistical Society: Series A (Statistics in Society) 169(3): 439–457. Doi:10.1111/j.1467-985x.2006.00423.x">http://dx.doi.org/10.1111/j.1467-985x.2006.00423.x . - Harrell, F.E. 2001.
Regression Modeling Strategies . New York: Springer. Doi:10.1007/978-1-4757-3462-1">http://dx.doi.org/10.1007/978-1-4757-3462-1 . - He, H., H. III Daume, and J. Eisner. 2012. “Cost-Sensitive Dynamic Feature Selection.” In Inferning 2012: ICML workshop on interaction between inference and learning. Edinburgh, Scotland. Available at:
https://www.cs.jhu.edu/~jason/papers/he+al.icmlw12.pdf (accessed April 14, 2017). - Horwitz, R., J. Tancreto, M.F. Zelenak, and M. Davis. 2012. “Data Quality Assessment of the American Community Survey Internet Response Data.” Available at:
https://www.census.gov/content/dam/Census/library/working-papers/2012/acs/2012.Horwitz.02.pdf (accessed April 14, 2017). - Jabine, T.B., M.L. Straf, J.M. Tanur, and R. Tourangeau. 1984.
Cognitive Aspects of Survey Methodology: Building a Bridge Between Disciplines . Washington, DC: National Academies Press. Doi:10.2307/2289187">http://dx.doi.org/10.2307/2289187 . - Kaczmirek, L. 2008. “Human-Survey Interaction: Usability and Nonresponse in Online Surveys” (Doctoral dissertation, Universität Mannheim). Available at:
http://ub-madoc.bib.uni-mannheim.de/2150/1/kaczmirek2008.pdf (accessed April 14, 2017). - Kamakura, W.A. and S.K. Balasubramanian. 1989. “Tailored Interviewing: An Application of Item Response Theory for Personality Measurement.”
Journal of Personality Assessment 53(3): 502–519. Doi:10.1207/s15327752jpa5303.8">http://dx.doi.org/10.1207/s15327752jpa5303.8 . - Kapelner, A. and D. Chandler. 2010. “Preventing Satisficing in Online Surveys.” In Proceedings of CrowdConf. San Francsico, California. Available at:
http://s3.amazonaws.com/academia.edu.documents/30740949/kapcha.pdf?AWSAccessKeyId=AKIAIWOWYYGZ2Y53UL3A&Expires=1499626168&Signature=0zH5TV1LDugz3h9FlSRJy5lRp38%3D&response-content-disposition=inline%3B%20filename%3DPreventing_Satisficing_in_Online_Surveys.pdf (accessed July 9, 2017). - Karayev, S., T. Baumgartner, M. Fritz, and T. Darrell. 2012. “Timely Object Recognition.” In
Advances in Neural Information Processing Systems 25: 890–898. Lake Tahoe, Nevada. Available at:https://papers.nips.cc/paper/4712-timely-object-recognition.pdf (accessed April 14, 2017). - Kilbourne, A.M., D. Almirall, D. Eisenberg, J. Waxmonsky, D.E. Goodrich, J.C. Fortney, J.E. Kirchner, L.I. Solberg, D. Main, M.S. Bauer, J. Kyle, S.A. Murphy, K.M. Nord, and M.R. Thomas. 2014. “Protocol: Adaptive Implementation of Effective Programs Trial (ADEPT): Cluster Randomized SMART Trial Comparing a Standard Versus Enhanced Implementation Strategy to Improve Outcomes of a Mood Disorders Program.”
Implementation Science 9(132): 1–14. Doi:10.1186/s13012-014-0132-x">http://dx.doi.org/10.1186/s13012-014-0132-x . - Laurie, H., and L. Scott. 1999. “Strategies for Reducing Nonresponse in a Longitudinal Panel Survey.”
Journal of Official Statistics 15(2): 269–282. - Lavori, P.W., R. Dawson, and A.J. Rush. 2000. “Flexible Treatment Strategies in Chronic Disease: Clinical and Research Implications.”
Biological Psychiatry 48(6): 605–614. Doi:10.1016/s0006-3223(00)00946-x">http://dx.doi.org/10.1016/s0006-3223(00)00946-x . - Lord, F.M. 1980.
Applications of Item Response Theory to Practical Testing Problems . Hillsdale, NJ: Lawrence Erlbaum Associates. Doi:10.4324/9780203056615">http://dx.doi.org/10.4324/9780203056615 . - Manfreda, K.L. and V. Vehovar. 2002. “Survey Design Features Influencing Response Rates in Web Surveys.” In The International Conference on Improving Surveys. Copenhagen, Denmark. Available at:
http://www.websm.org/uploadi/editor/Lozar_Vehovar_2001_Survey_design.pdf (accessed April 14, 2017). - Marcus, B., M. Bosnjak, S. Lindner, S. Pilischenko, and A. Schütz. 2007. “Compensating for Low Topic Interest and Long Surveys a Field Experiment on Nonresponse in Web Surveys.”
Social Science Computer Review 25(3): 372–383. Doi:10.1177/0894439307297606">http://dx.doi.org/10.1177/0894439307297606 . - McFarland, S.G. 1981. “Effects of Question Order on Survey Responses.”
Public Opinion Quarterly 45(2): 208–215. Doi:10.1086/268651">http://dx.doi.org/10.1086/268651 . - Montgomery, J.M., and J. Cutler. 2013. “Computerized Adaptive Testing for Public Opinion Surveys.”
Political Analysis 21(2): 172–192. Doi:10.1093/pan/mps060">http://dx.doi.org/10.1093/pan/mps060 . - Murphy, S.A. 2005. “An Experimental Design for the Development of Adaptive Treatment Strategies.”
Statistics in Medicine 24: 1455–1481. Doi:10.1002/sim.2022">http://dx.doi.org/10.1002/sim.2022 . - National Center for Health Statistics. 2014. “Survey Description: National Health Interview Survey.” Available at:
http://ftp.cdc.gov/pub/Health_Statistics/NCHs/Dataset_Documentation/NHIS/2014/srvydesc.pdf (accessed April 14, 2017). - Pandey, D., M. Agrawal, and J.S. Pandey. 2011. “Carbon Footprint: Current Methods of Estimation.”
Environmental Monitoring and Assessment 178(1–4): 135–160. Doi:10.1007/s10661-010-1678-y">http://dx.doi.org/10.1007/s10661-010-1678-y . - Peytchev, A. 2009. “Survey Breakoff.”
Public Opinion Quarterly 73(1): 74–97. Doi:10.1093/poq/nfp014">http://dx.doi.org/10.1093/poq/nfp014 . - Pitkow, J.E. and M.M. Recker. 1995. “Using the Web as a Survey Tool: Results from the Second WWW User Survey.”
Computer Networks and ISDN Systems 27(6): 809–822. - Porter, S.R. 2004. “Raising Response Rates: What Works?”
New Directions for Institutional Research 2004(121): 5–21. Doi:10.1002/ir.97">http://dx.doi.org/10.1002/ir.97 . - Rubin, D.B. 2004.
Multiple Imputation for Nonresponse in Surveys (Vol. 81). John Wiley & Sons. Doi:10.1002/9780470316696">http://dx.doi.org/10.1002/9780470316696 . - Rush, A.J., M. Fava, S.R. Wisniewski, P.W. Lavori, M.H. Trivedi, H.A. Sackeim, M.E. Thase, A.A. Nierenberg, F.M. Quitkin, T.M. Kashner, and D.J. Kupfher. 2004. “Sequenced Treatment Alternatives to Relieve Depression (STAR* D): Rationale and Design.”
Contemporary Clinical Trials 25(1): 119–142. Doi:10.1016/S0197-2456(03)00112-0">http://dx.doi.org/10.1016/S0197-2456(03)00112-0 . - Schouten, B., M. Calinescu, and A. Luiten. 2013. “Optimizing Quality of Response Through Adaptive Survey Designs.”
Survey Methodology 39(1): 29–58. - Shih, T.-H., and X. Fan. 2008. “Comparing Response Rates from Web and Mail Surveys: A Meta-Analysis.”
Field Methods 20(3): 249–271. Doi:10.1177/1525822x08317085">http://dx.doi.org/10.1177/1525822x08317085 . - Singh, J., R.D. Howell, and G.K. Rhoads. 1990. “Adaptive Designs for Likert-Type Data: An Approach for Implementing Marketing Surveys.”
Journal of Marketing Research 27(3): 304–321. Doi:10.2307/3172588">http://dx.doi.org/10.2307/3172588 . - Stroup, T.S., J.P. McEvoy, M.S. Swartz, M.J. Byerly, I.D. Glick, J.M. Canive, and . . . J.A. Lieberman. 2003. “The National Institute of Mental Health Clinical Antipsychotic Trials of Intervention Effectiveness (CATIE) Project: Schizophrenia Trial Design and Protocol Development.”
Schizophrenia Bulletin 29(1): 15–31. Doi:10.1093/oxfordjournals.schbul.a006986">http://dx.doi.org/10.1093/oxfordjournals.schbul.a006986 . - Suchman, L. and B. Jordan. 1990. “Interactional Troubles in Face-to-Face Survey Interviews (with discussion).”
Journal of the American Statistical Association 85(409): 232–253. Doi:10.1080/01621459.1990.10475331">http://dx.doi.org/10.1080/01621459.1990.10475331 . - Sudman, S., N.M. Bradburn, and N. Schwarz. 1996.
Thinking About Answers: The Application of Cognitive Processes to Survey Methodology . San Francisco: Jossey-Bass. - Sun, M., F. Li, J. Lee, K. Zhou, G. Lebanon, and H. Zha. 2013. “Learning Multiple-Question Decision Trees for Cold-Start Recommendation.” In Proceedings of the Sixth ACM International Conference on Web Search and Data Mining, 445–454. Rome, Italy: ACM. Doi:
10.1145/2433396.2433451">http://dx.doi.org/10.1145/2433396.2433451 . - Tanur, J.M. 1992.
Questions About Questions: Inquiries into the Cognitive Bases of Surveys . New York: Russell Sage Foundation. Doi:10.2307/2075046">http://dx.doi.org/10.2307/2075046 . - Tourangeau, R. 1984. “Cognitive Sciences and Survey Methods.” In
Cognitive Aspects of Survey Methodology: Building a Bridge Between Disciplines , edited by T.B. Jabine, M.L. Straf, J.M. Tanur, and R. Tourangeau, 73–100. Washington, DC: National Academies Press. - Tropp, J.A. 2004. “Greed is Good: Algorithmic Results for Sparse Approximation.”
IEEE Transactions on Information Theory 50(10): 2231–2242. Doi:10.1109/tit.2004.834793">http://dx.doi.org/10.1109/tit.2004.834793 . - U.S. Bureau of the Census. 2006.
Design and methodology: Current Population Survey . Available at:http://www.nber.org/cps/tp-66.pdf (accessed April 14, 2017). - U.S. Bureau of the Census. 2013.
American Housing Survey for the United States . Washington, D.C: U.S. Government Printing Office. - U.S. Bureau of the Census. 2014a.
American Community Survey: Design and methodology . Available at:http://www2.census.gov/programs-surveys/acs/methodology/designandmethodology/acs_design_methodology_report_2014.pdf (accessed April 14, 2017). - U.S. Bureau of the Census. 2014b.
Survey of Income and Program Participation . Available at:www.census.gov/programs-surveys/sipp (accessed April 14, 2017). - U.S. Bureau of the Census. 2016.
American Community Survey . Available at:https://www.census.gov/programs-surveys/acs/ (accessed April 14, 2017). - U.S. Energy Information Administration. 2009.
Residential Energy Consumption Survey 2009 . Available at:www.eia.gov/consumption/residential/data/2009/ . (accessed April 14, 2017). - Walston, J.T., R.W. Lissitz, and L.M. Rudner. 2006. “The Influence of Web-Based Questionnaire Presentation Variations on Survey Cooperation and Perceptions of Survey Quality.”
Journal of Official Statistics 22(2): 271–291. - Wang, L., A. Rotnitzky, X. Lin, R.E. Millikan, and P.F. Thall. 2012. “Evaluation of Viable Dynamic Treatment Regimes in a Sequentially Randomized Trial of Advanced Prostate Cancer.”
Journal of the American Statistical Association 107(498): 493–508. Doi:10.1080/01621459.2011.641416">http://dx.doi.org/10.1080/01621459.2011.641416 . - Weisberg, S. 2014.
Applied Linear Regression (4th ed.). New York: Wiley. Doi:10.1002/0471704091">http://dx.doi.org/10.1002/0471704091 . - Weiss, D.J. 1982. “Improving Measurement Quality and Efficiency with Adaptive Testing.”
Applied Psychological Measurement 6(4): 473–492. Doi:10.1177/014662168200600408">http://dx.doi.org/10.1177/014662168200600408 . - Weiss, D.J. and G.G. Kingsbury. 1984. “Application of Computerized Adaptive Testing to Educational Eroblems.”
Journal of Educational Measurement 21(4): 361–375. Doi:10.1111/j.1745-3984.1984.tb01040.x">http://dx.doi.org/10.1111/j.1745-3984.1984.tb01040.x . - Yu, E.C., S. Fricker, and B. Kopp. 2015. “Can Survey Instructions Relieve Respondent Burden?” In AAPOR. Hollywood, Florida. Available at:
http://www.bls.gov/osmr/pdf/st150260.pdf (accessed April 14, 2017).
DOI: https://doi.org/10.1515/jos-2017-0030 | Journal eISSN: 2001-7367
Language: English
Page range: 625 - 657
Submitted on: Mar 1, 2016
Accepted on: May 1, 2017
Published on: Sep 9, 2017
Published by: Sciendo
In partnership with: Paradigm Publishing Services
Publication frequency: 4 times per year
Keywords:
Related subjects:
© 2017 Kirstin Early, Jennifer Mankoff, Stephen E. Fienberg, published by Sciendo
This work is licensed under the Creative Commons Attribution-NonCommercial-NoDerivatives 3.0 License.