Have a personal or library account? Click to login
Sample Representation and Substantive Outcomes Using Web With and Without Incentives Compared to Telephone in an Election Survey Cover

Sample Representation and Substantive Outcomes Using Web With and Without Incentives Compared to Telephone in an Election Survey

Journal of Official Statistics
Volume 32 (2016): Issue 1 (March 2016)
By: Oliver Lipps and  Nicolas Pekari  
Open Access
|Mar 2016

References

  1. .AAPOR (The American Association for Public Opinion Research). 2011. Standard definitions: Final dispositions of case codes and outcome rates for surveys, 7th ed. AAPOR.
    Search in Google Scholar
  2. Alexander, G., G. Divine, M. Couper, J. McClure, M. Stopponi, K. Fortman, D. Tolsma, V. Strecher, and C. Johnson. 2008. “Effect of Incentives and Mailing Features on Online Health Program Enrolment.” American Journal of Preventive Medicine 34: 382-388. Doi: http://dx.doi.org/10.1016/j.amepre.2008.01.028.10.1016/j.amepre.2008.01.028244273718407004
    Search in Google Scholar
  3. Ansolabehere, S. and B. Schaffner. 2014. “Does Survey Mode Still Matter? Findings From a 2010 Multi-Mode Comparison.” Political Analysis 22: 285-303. Doi: http://dx.doi.org/10.1093/pan/mpt025.10.1093/pan/mpt025
    Search in Google Scholar
  4. Atkeson, L., A. Adams, and R. Alvarez. 2014. “Nonresponse and Mode Effects in Self-and Interviewer-Administered Surveys.” Political Analysis 22: 304-320. Doi: http://dx.doi.org/10.1093/pan/mpt049.10.1093/pan/mpt049
    Search in Google Scholar
  5. Baker, R., S.J. Blumberg, J.M. Brick, M.P. Couper, M. Courtright, J.M. Dennis, D.
    Search in Google Scholar
  6. Dillman, M.R. Frankel, P. Garland, R.M. Groves, C. Kennedy, J. Krosnick, P.J.
    Search in Google Scholar
  7. Lavrakas, S. Lee, M. Link, L. Piekarski, K. Rao, R.K. Thomas, and D. Zahs. 2010. “Research Synthesis AAPOR Report on Online Panels.” Public Opinion Quarterly 74: 711-781. Doi: http://dx.doi.org/10.1093/poq/nfq048.10.1093/poq/nfq048
    Search in Google Scholar
  8. Biemer, P. 1988. “Measuring Data Quality.” In Telephone Survey Methodology, edited by W. Nicholls II, R. Groves, P. Biemer, L. Lyberg, J. Massey, W. Nicholls II, and J. Waksberg, 273-283. New York: Wiley & Sons.
    Search in Google Scholar
  9. Biemer, P. and L. Lyberg. 2003. Introduction to Survey Quality. New York: John Wiley & Sons.10.1002/0471458740
    Search in Google Scholar
  10. Birnholtz, J., D. Horn, T. Finholt, and S. Bae. 2004. “The Effects of Cash, Electronic, and Paper Gift Certificates as Respondent Incentives for a Web-Based Survey of Technologically Sophisticated Respondents.” Social Science Computer Review 22: 355-362. Doi: http://dx.doi.org/10.1177/0894439304263147.10.1177/0894439304263147
    Search in Google Scholar
  11. Blumberg, S. and J. Luke. 2013. Wireless Substitution: Early Release of Estimates from the National Health Interview Survey. July-December 2012. Available at: http://www.cdc.gov/nchs/data/nhis/earlyrelease/wireless201306.pdf (accessed December 2014).
    Search in Google Scholar
  12. Bosnjak, M. 2005. “Effects of Two Innovative Techniques to Apply Incentives in Online Access Panels.” Presentation at the General Online Research Conference (GOR), Zürich, March 22-23.
    Search in Google Scholar
  13. Bosnjak, M. and T. Tuten. 2003. “Prepaid and Promised Incentives in Web Surveys: an Experiment.” Social Science Computer Review 21: 208-217. Doi: http://dx.doi.org/10.1177/0894439303021002006.10.1177/0894439303021002006
    Search in Google Scholar
  14. Brandtzæg, P., J. Heim, and A. Karahasanovic. 2011. “Understanding the New Digital Divide - A Typology of Internet Users in Europe.” International Journal of Human- Computer Studies 69: 123-138. Doi: http://dx.doi.org/10.1016/j.ijhcs.2010.11.004.10.1016/j.ijhcs.2010.11.004
    Search in Google Scholar
  15. Braunsberger, K., H. Wybenga, and R. Gates. 2007. “A Comparison of Reliability Between Telephone and Web-Based Surveys.” Journal of Business Research 60: 758-764. Doi: http://dx.doi.org/10.1016/j.jbusres.2007.02.015.10.1016/j.jbusres.2007.02.015
    Search in Google Scholar
  16. Burden, B.C. 2000. “Voter Turnout and the National Election Studies.” Political Analysis 8: 389-398.10.1093/oxfordjournals.pan.a029823
    Search in Google Scholar
  17. Busse, B. and M. Fuchs. 2012. “The Components of Landline Telephone Survey Coverage Bias. The Relative Importance of No-Phone and Mobile-Only Populations.” Quality and Quantity 46: 1209-1225. Doi: http://dx.doi.org/10.1007/s11135-011-9431-3.10.1007/s11135-011-9431-3
    Search in Google Scholar
  18. Callegaro, M. and C. DiSogra. 2008. “Computing Response Metrics for Online Panels.” Public Opinion Quarterly 72: 1008-1032. Doi: http://dx.doi.org/10.1093/poq/nfn065.10.1093/poq/nfn065
    Search in Google Scholar
  19. Chang, L. and J. Krosnick. 2009. “National Surveys via RDD Telephone Interviewing Versus the Internet.” Public Opinion Quarterly 73: 641-678. Doi: http://dx.doi.org/10.1086/346010.10.1086/346010
    Search in Google Scholar
  20. Christian, L., D. Dillman, and J. Smith. 2008. “The Effects of Mode and Format on Answers to Scalar Questions in Telephone and Web Surveys.” In Advances in Telephone Surveys, edited by J.M. Lepkowski, 250-275. New York: Wiley & Sons.10.1002/9780470173404.ch12
    Search in Google Scholar
  21. Cobben, F. and J. Bethlehem. 2005. “Adjusting Undercoverage and Nonresponse Bias in Telephone Surveys.” Discussion paper 05006. CBS, Statistics Netherlands, Voorburg/ Heerlen. Available at: http://www.cbs.nl/nr/rdonlyres/7fd00f42-15a3-4151-9daa-2d54-566cf59a/0/200506x10pub.pdf (accessed February, 2016).
    Search in Google Scholar
  22. Curtin, R., E. Singer, and S. Presser. 2007. “Incentives in Random Digit Dial Telephone Surveys: a Replication and Extension.” Journal of Official Statistics 23: 91-105.
    Search in Google Scholar
  23. De Leeuw, E. 2005. “To Mix or Not To Mix Data Collection Modes in Surveys.” Journal of Official Statistics 21: 233-255.
    Search in Google Scholar
  24. Dillman, D. 2000. Mail and Telephone Surveys: The Tailored Design Method. New York: John Wiley & Sons.
    Search in Google Scholar
  25. Dillman, D. 2011. Mail and Internet surveys: The Tailored Design Method - 2007 Update With New Internet, Visual, and Mixed-Mode Guide. New York: John Wiley & Sons.
    Search in Google Scholar
  26. Dillman, D. and L. Christian. 2005. “Survey Mode as a Source of Instability in Responses Across Surveys.” Field Methods 17: 30-52. Doi: http://dx.doi.org/10.1177/1525822X04269550.10.1177/1525822X04269550
    Search in Google Scholar
  27. Dillman, D., G. Phelps, R. Tortora, K. Swift, J. Kohrell, J. Berck, and B. Messer. 2009. “Response Rate and Measurement Differences in Mixed-Mode Surveys Using Mail, Telephone, Interactive Voice Response (IVR) and the Internet.” Social Science Research 28: 1-18. Doi: http://dx.doi.org/10.1016/j.ssresearch.2008.03.00.
    Search in Google Scholar
  28. Ernst Stähli, M. 2012. “Telephone Surveys in Switzerland: Spotlight.” In Telephone Surveys in Europe: Research and Practice, edited by M. Häder, S. Häder and M. Kühne, 25-36. Berlin: Springer.10.1007/978-3-642-25411-6_3
    Search in Google Scholar
  29. Fricker, S., M. Galesic, R. Tourangeau, and T. Yan. 2005. “An Experimental Comparison of Web and Telephone Surveys.” Public Opinion Quarterly 69: 370-392. Doi: http://dx.doi.org/10.1093/poq/nfi027.10.1093/poq/nfi027
    Search in Google Scholar
  30. Göritz, A. 2004. “The Impact of Material Incentives on Response Quantity, Response Quality, Sample Composition, Survey Outcome, and Cost in Online Access Panels.” International Journal of Market Research 46: 327-345. Gö ritz, A. 2006. “Incentives in Web Studies: Methodological Issues and a Review.” International Journal of Internet Science 1: 58-70.
    Search in Google Scholar
  31. Groves, R., R. Cialdini, and M. Couper. 1992. “Understanding the Decision to Participate in a Survey.” Public Opinion Quarterly 56: 475-493. Doi: http://dx.doi.org/10.1086/269338.10.1086/269338
    Search in Google Scholar
  32. Groves, R., F. Fowler, M. Couper, J. Lepkowski, E. Singer, and R. Tourangeau. 2004a. Survey Methodology, Wiley Series in Survey Methodology. New York: Wiley.
    Search in Google Scholar
  33. Groves, R., S. Presser, and S. Dipko. 2004b. “The Role of Topic Interest in Survey Participation Decisions.” Public Opinion Quarterly 68: 2-31. Doi: http://dx.doi.org/10.1093/poq/nfh002.10.1093/poq/nfh002
    Search in Google Scholar
  34. Holbrook, A.L. and J.A. Krosnick. 2010. “Social Desirability Bias in Voter Turnout Reports Tests Using the Item Count Technique.” Public Opinion Quarterly 74: 37-67. Doi: http://dx.doi.org/10.1093/poq/nfp065.10.1093/poq/nfp065
    Search in Google Scholar
  35. Joye, C. 2012. “Srph-Castem.” FORS - SFSO workshop, June 21. Neuchâtel.
    Search in Google Scholar
  36. Joye, D., A. Pollien, M. Sapin, and M. Ernst Stähli. 2012. “Who Can Be Contacted by Phone? Lessons from Switzerland.” In Telephone Surveys in Europe: Research and Practice, edited by M. Häder, S. Häder and M. Kühne, 85-102. Berlin: Springer- Verlag.10.1007/978-3-642-25411-6_8
    Search in Google Scholar
  37. Karp, J.A. and D. Brockington. 2005. “Social Desirability and Response Validity: A Comparative Analysis of Overreporting Voter Turnout in Five Countries.” The Journal of Politics 67: 825-840. Doi: http://dx.doi.org/10.1111/j.1468-2508.2005.00341.x.10.1111/j.1468-2508.2005.00341.x
    Search in Google Scholar
  38. Kreuter, F., S. Presser, and R. Tourangeau. 2008. “Social Desirability Bias in CATI, IVR, and Web Surveys.” Public Opinion Quarterly 72: 847-865. Doi: http://dx.doi.org/10.1093/poq/nfn063.10.1093/poq/nfn063
    Search in Google Scholar
  39. Krosnick, J. 1991. “Response Strategies for Coping With the Cognitive Demands of Attitude Measures in Surveys.” Applied Cognitive Psychology 5: 213-236. Doi: http://dx.doi.org/10.1002/acp.2350050305.10.1002/acp.2350050305
    Search in Google Scholar
  40. Link, M. and M. Fahimi. 2008. “Telephone Survey Sampling.” In Sampling of Populations: Methods and Applications, edited by P.S. Levy and S. Lemeshow, 455-487. New York: Wiley.
    Search in Google Scholar
  41. Lipps, O. and K. Kissau. 2012. “Nonresponse in an Individual Register Sample Telephone Survey in Lucerne (Switzerland).” In Telephone Surveys in Europe: Research and Practice, edited by M. Häder, S. Häder and M. Kühne, 187-208. Berlin: Springer-Verlag.10.1007/978-3-642-25411-6_13
    Search in Google Scholar
  42. Lipps, O. and N. Pekari. 2013. Mode and Incentive Effects in an Individual Register Frame Based Swiss Election Study. FORS Working Paper Series, paper 2013-3. Lausanne: FORS.
    Search in Google Scholar
  43. Lipps, O., N. Pekari, and C. Roberts. 2015. “Coverage and Nonresponse Errors in an Individual Register Frame Based Swiss Telephone Election Study.” Survey Research Methods 9: 71-82.
    Search in Google Scholar
  44. Little, R.J. and S. Vartivarian. 2003. “On Weighting the Rates in Non-Response Weights.” Statistics in Medicine 22: 1589-1599. Doi: http://dx.doi.org/10.1002/sim.1513.10.1002/sim.151312704617
    Search in Google Scholar
  45. Lozar Manfreda, K., M. Bosnjak, J. Berzelak, I. Haas, and V. Vehovar. 2008. “Web Surveys Versus Other Survey Modes: a Meta-Analysis Comparing Response Rates.” International Journal of Market Research 50: 79-104.10.1177/147078530805000107
    Search in Google Scholar
  46. Malhotra, N. and J. Krosnick. 2007. “The Effect of Survey Mode and Sampling on Inferences About Political Attitudes and Behavior: Comparing the 2000 and 2004 ANES to Internet Surveys With Nonprobability Samples.” Political Analysis 15: 286-323. Doi: http://dx.doi.org/10.1093/pan/mpm003.10.1093/pan/mpm003
    Search in Google Scholar
  47. McDonald, M.P. 2003. “On the Overreport Bias of the National Election Study Turnout Rate.” Political Analysis 11: 180-186. Doi: http://dx.doi.org/10.1093/pan/mpg006.10.1093/pan/mpg006
    Search in Google Scholar
  48. Messer, B.L. and D.A. Dillman. 2011. “Surveying the General Public Over the Internet Using Address-Based Sampling and Mail Contact Procedures.” Public Opinion Quarterly 75: 429-457. Doi: http://dx.doi.org/10.1093/poq/nfr021.10.1093/poq/nfr021
    Search in Google Scholar
  49. Mohorko, A., E. de Leeuw, and J. Hox. 2013a. “Internet Coverage and Coverage Bias in Europe: Developments Across Countries and Over Time.” Journal of Official Statistics 29: 609-622. Doi: http://dx.doi.org/10.2478/jos-2013-0042.10.2478/jos-2013-0042
    Search in Google Scholar
  50. Mohorko, A., E. de Leeuw, and J. Hox. 2013b. “Coverage Bias in European Telephone Surveys: Developments of Landline and Mobile Phone Coverage Across Countries and Over Time.” Survey Methods: Insights from the Field. Doi: http://dx.doi.org/10.13094/SMIF-2013-00002.
    Search in Google Scholar
  51. Mood, C. 2010. “Logistic Regression: Why We Cannot Do What We Think We Can Do, and What We Can Do About It.” European Sociological Review 26: 67-82. Doi: http://dx.doi.org/10.1093/esr/jcp006.10.1093/esr/jcp006
    Search in Google Scholar
  52. Nagelhout, G., M. Willemsen, M. Thompson, G. Fong, B. van den Putte, and H. de Vries. 010. “Is Web Interviewing a Good Alternative to Telephone Interviewing? Findings from the International Tobacco Control (ITC) Netherlands Survey.” BMC Public Health 10: 351. Doi: http://dx.doi.org/10.1186/1471-2458-10-351.10.1186/1471-2458-10-351289779620565838
    Search in Google Scholar
  53. Omnibus 2010. Survey on Information and Communication Technology, Swiss Federal Statistical Office 2010. Excel result sheets (in German; accessed October 21, 2013). Available at: http://www.bfs.admin.ch/bfs/portal/de/index/themen/16/04/data.html (accessed December 2014).
    Search in Google Scholar
  54. Parsons, N. and M. Manierre. 2014. “Investigating the Relationship Among Prepaid Token Incentives, Response Rates, and Nonresponse Bias in a Web Survey.” Field Methods 26: 191-204. Doi: http://dx.doi.org/10.1177/1525822X13500120.10.1177/1525822X13500120
    Search in Google Scholar
  55. Peytchev, A. 2009. “Survey Breakoff.” Public Opinion Quarterly 73: 74-97. Doi: http://dx.doi.org/10.1093/poq/nfp014.10.1093/poq/nfp014
    Search in Google Scholar
  56. Revilla, M.A. and W.E. Saris. 2013. “A Comparison of the Quality of Questions in a Face-to-Face and a Web Survey.” International Journal of Public Opinion Research 25: 242-253. Doi: http://dx.doi.org/10.1093/ijpor/eds007.10.1093/ijpor/eds007
    Search in Google Scholar
  57. Ryu, E., M. Couper, and R. Marans. 2006. “Survey Incentives: Cash vs. In-Kind, Face-to- Face vs. Mail, Response Rate vs. Nonresponse Error.” International Journal of Public Opinion Research 18: 89-106. Doi: http://dx.doi.org/10.1093/ijpor/edh089.10.1093/ijpor/edh089
    Search in Google Scholar
  58. Sánchez-Fernández, J., F. Muñoz-Leiva, F.J. Montoro-Ríos, and J. Ángel Ibáñez-Zapata. 2010. “An Analysis of the Effect of Pre-Incentives and Post-Incentives Based on Draws on Response to Web Surveys.” Quality and Quantity 44: 357-373. Doi: http://dx.doi.org/10.1007/s11135-008-9197-4.10.1007/s11135-008-9197-4
    Search in Google Scholar
  59. Schaurer, I., B. Struminskaya, L. Kaczmirek, and W. Bandilla. 2012. “The Price We Have to Pay: Incentive Experiments in the Recruitment Process for a Probability-Based Online Panel.” Presentation at the General Online Research Conference (GOR) March 5-7, 2012, Mannheim.
    Search in Google Scholar
  60. Scherpenzeel, A. and V. Toepoel. 2012. “Recruiting a Probability Sample for an Online Panel. Effects of Contact Mode, Incentives, and Information.” Public Opinion Quarterly 76: 470-490. Doi: http://dx.doi.org/10.1093/poq/nfs037.10.1093/poq/nfs037
    Search in Google Scholar
  61. Schonlau, M., A. van Soest, A. Kapteyn, and M. Couper. 2009. “Selection Bias in Web Surveys and the Use of Propensity Scores.” Sociological Methods and Research 37: 291-318. Doi: http://dx.doi.org/10.1177/0049124108327128.10.1177/0049124108327128
    Search in Google Scholar
  62. Selb, P. and S. Munzert. 2013. “Voter Overrepresentation, Vote Misreporting, and Turnout Bias in Postelection Surveys.” Electoral Studies 32: 186-196. Doi: http://dx.doi.org/10.1016/j.electstud.2012.11.004.10.1016/j.electstud.2012.11.004
    Search in Google Scholar
  63. Sinclair, M., J. O’Toole, M. Malawaraarachchi, and K. Leder. 2012. “Comparison of Response Rates and Cost-Effectiveness for a Community-Based Survey: Postal, Internet and Telephone Modes with Generic or Personalised Recruitment Approaches.” BMC Medical Research Methodology 12: 132. Doi: http://dx.doi.org/10.1186/1471-2288-12-132.10.1186/1471-2288-12-132
    Search in Google Scholar
  64. Singer, E. and R. Bossarte. 2006. “Incentives for Survey Participation. When are they ‘Coercive’?” American Journal of Preventive Medicine 31: 411-418. Doi: http://dx.doi.org/10.1016/j.amepre.2006.07.013.10.1016/j.amepre.2006.07.013
    Search in Google Scholar
  65. Singer, E. J. van Hoewyk, and M. Maher. 2000. “Experiments with Incentives in Telephone Surveys.” Public Opinion Quarterly 64: 171-188. Doi: http://dx.doi.org/10.1086/317761.10.1086/317761
    Search in Google Scholar
  66. Singer, E. and C. Ye. 2013. “The Use and Effects of Incentives in Surveys.” The ANNALS of the American Academy of Political and Social Science 645: 112-141. Doi: http://dx.doi.org/10.1177/0002716212458082.10.1177/0002716212458082
    Search in Google Scholar
  67. Stephenson, L. and J. Creˆte. 2011. “Studying Political Behavior: A Comparison of Internet and Telephone Surveys.” International Journal of Public Opinion Research 23: 24-55. Doi: http://dx.doi.org/10.1093/ijpor/edq025.10.1093/ijpor/edq025
    Search in Google Scholar
  68. Struminskaya, B., L. Kaczmirek, I. Schaurer, and W. Bandilla. 2014. “Assessing Representativeness of a German Probability-Based Panel.” In Online Panel Research: A Data Quality Perspective, edited by M. Callegaro, R. Baker, J. Bethlehem, A. Gö ritz, J. Krosnick, and P. Lavrakas, 61-85. New York: John Wiley & Sons.10.1002/9781118763520.ch3
    Search in Google Scholar
  69. Su, J., P. Shao, and J. Fang. 2008. “Effect of Incentives on Web-Based Surveys.” Tsinghua Science and Technology 13: 344-347. Doi: http://dx.doi.org/10.1016/S1007-0214(08)70055-5.10.1016/S1007-0214(08)70055-5
    Search in Google Scholar
  70. Teisl, M., B. Roe, and M. Vayda. 2006. “Incentive Effects on Response Rates, Data Quality, and Survey Administration Costs.” International Journal of Public Opinion Research 18: 364-373. Doi: http://dx.doi.org/10.1093/ijpor/edh106.10.1093/ijpor/edh106
    Search in Google Scholar
  71. Vannieuwenhuyze, J. and G. Loosveldt. 2013. “Evaluating Relative Mode Effects in Mixed-Mode Surveys: Three Methods to Disentangle Selection and Measurement Effects.” Sociological Methods & Research 42: 82-104. Doi: http://dx.doi.org/10.1177/0049124112464868.10.1177/0049124112464868
    Search in Google Scholar
  72. Van Veen, F., A. Göritz, and S. Sattler. 2011. “The Impact of Monetary Incentives on Completion and Data Quality in Online Surveys.” Presentation at the European Survey Research Association (ESRA) Conference, Lausanne, July 18-22 and General Online Research (GOR) Conference, Düsseldorf, March 14-16.
    Search in Google Scholar
  73. Von der Lippe, E., P. Schmich, and C. Lange. 2011. “Advance Letters as a Way of Reducing Non-Response in a National Health Telephone Survey: Differences Between Listed and Unlisted Numbers.” Survey Research Methods 5: 103-116. Doi: http://dx.doi.org/10.18148/srm/2011.v5i3.4657#sthash.qLueRYqS.dpuf.
    Search in Google Scholar
  74. Warren, J. and A. Halpern-Manners. 2012. “Panel Conditioning in Longitudinal Social Science Surveys.” Sociological Methods and Research 41: 491-534. Doi: http://dx.doi.org/10.1177/0049124112460374.10.1177/0049124112460374
    Search in Google Scholar
  75. Yeager, D., J. Krosnick, L. Chang, H. Javitz, M. Levendusky, A. Simpser, and R. Wang. 2011. “Comparing the Accuracy of RDD Telephone Surveys and Internet Surveys Conducted with Probability and Non-Probability Samples.” Public Opinion Quarterly 75: 709-747. Doi: http://dx.doi.org/10.1093/poq/nfr020.10.1093/poq/nfr020
    Search in Google Scholar
  76. Zickhur, K. and A. Smith. 2012. Digital Differences. Pew Internet & American Life Project 13. Available at: http://www.pewinternet.org/2012/04/13/digital-differences/ (accessed August 2014).
    Search in Google Scholar
DOI: https://doi.org/10.1515/jos-2016-0008 | Journal eISSN: 2001-7367
Journal RSS Feed
Language: English
Page range: 165 - 186
Submitted on: Feb 1, 2014
|
Accepted on: Apr 1, 2015
|
Published on: Mar 10, 2016
Published by: Sciendo
In partnership with: Paradigm Publishing Services
Publication frequency: 4 issues per year
Keywords:
Web surveys,
mode experiment,
incentive effects,
individual register frame,
national election survey
Related subjects:
Mathematics,
Probability and statistics

© 2016 Oliver Lipps, Nicolas Pekari, published by Sciendo
This work is licensed under the Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 License.

Previous articleVolume 32 (2016): Issue 1 (March 2016)Next article