Have a personal or library account? Click to login
Using Eye-Tracking Methodology to Study Grid Question Designs in Web Surveys Cover

Using Eye-Tracking Methodology to Study Grid Question Designs in Web Surveys

Open Access
|Mar 2023

References

  1. Antoun, C., J. Katz, J. Argueta, and L. Wang. 2018. “Design Heuristics for effective Smartphone Questionnaires.” Social Science Computer Review 36(5): 557–574. DOI: https://doi.org/10.1177/0894439317727072.
  2. Brosnan, K., B. Grün, and S. Dolnicar. 2021. “Cognitive load reduction strategies in questionnaire design.” International Journal of Market Research 63(2): 125–133. DOI: https://doi.org/10.1177%2F1470785320986797.10.1177/1470785320986797
  3. Callegaro, M., J. Shand-Lubbers, and J.M. Dennis. 2009 “Presentation of a Single Item versus a Grid: Effects on the Vitality and Mental Health Scales of the SF-36v2 Health Survey.” 64th Annual Conference of the American Association for Public Opinion Research (AAPOR), May 14, 2009: 5887–5897. Hollywood, Florida. Available at: http://www.asasrms.org/Proceedings/y2009/Files/400045.pdf (accessed July 2022).
  4. Cohen, J. 1960. “A Coefficient of Agreement for Nominal Scales.” Educational and psychological measurement 20(1): 37–46. DOI: https://doi.org/10.1177/001316446002000104.
  5. Couper, M.P. 2008. Designing effective Web surveys. Cambridge University Press. DOI: https://doi.org/10.1017/CBO9780511499371.
  6. Couper, M.P., R. Tourangeau, F.G. Conrad, F.C. Zhang. 2013. “The Design of Grids in Web Surveys.” Social Science Computer Review 31(3): 322–345. DOI: https://doi.org/10.1177/0894439312469865.417236125258472
  7. Couper, M.P., M.W. Traugott, and M.J. Lamias. 2001. “Web Survey Design and Administration.” Public Opinion Quarterly: 65(2): 230–253. DOI: https://doi.org/10.1086/322199.11420757
  8. DeBell, M., C. Wilson, S. Jackman, and L. Figueroa. 2021. “Optimal Response Formats for Online Surveys: Branch, Grid, or Single Item?” Journal of Survey Statistics and Methodology 9(1): 1–24. DOI: https://doi.org/10.1093/jssam/smz039.
  9. De Leeuw, E.D., Z.T. Suzer-Gurtekin, and J.J. Hox. 2018. “The Design and Implementation of Mixed-mode Surveys.” In Advances in Comparative Survey Methods: Multinational, Multiregional, and Multicultural Contexts (3MC), edited by T.P. Johnson, B. Pennell, I.A.L. Stoop, and B. Dorer: 387–409.
  10. Hoboken: Wiley. Dillman, D.A., J.D. Smyth, and L.M. Christian. 2014. Internet, Phone, Mail, and Mixed-mode Surveys: the Tailored Design Method (4th edition). Hoboken: Wiley.
  11. Galesic, M., and T. Yan. 2011. “Use of Eye Tracking for Studying Survey Response Processes.” In Social and Behavioral Research and the Internet, edited by M. Das, P. Ester, and L. Kaczmirek: 349–370. New York: Taylor and Francis.10.4324/9780203844922-14
  12. GLES. 2019. “Longterm-Online-Tracking, Cumulation 2009–2017 (GLES).” GESIS Data Archive, Cologne. ZA6832 Data file Version 1.1.0, DOI: https://doi.org/10.4232/1.13416.
  13. Grady, R.H., R.L. Greenspan, and M. Liu. 2019. “What Is the Best Size for Matrix-Style Questions in Online Surveys?” Social Science Computer Review 37(3): 435–445. DOI: https://doi.org/10.1177/0894439318773733.
  14. Gummer, T., F. Quoß, and J. Roßmann. 2019. “Does Increasing Mobile Device Coverage Reduce Heterogeneity in Completing Web Surveys on Smartphones?” Social Science Computer Review 37(3): 371–384. DOI: https://doi.org/10.1177/0894439318766836.
  15. Heerwegh, D. 2009. “Mode Differences Between Face-to-Face and Web Surveys: an Experimental Investigation of Data Quality and Social Desirability Effects.” International Journal of Public Opinion Research 21(1): 111–121. DOI: https://doi.org/10.1093/ijpor/edn054.
  16. ISSP Research Group (2016): International Social Survey Programme: Citizenship II – ISSP 2014. GESIS Data Archive, Cologne. ZA6670 Data file Version 2.0.0. DOI: https://doi.org/10.4232/1.12590.
  17. Jenkins C.R, and D.A. Dillman. 1995. “Towards a Theory of Self-Administered Questionnaire Design.” In Survey Measurement and Process Quality, edited by L. Lyberg, P. Biemer, M. Collins, E. de Leeuw, C. Dippo, N. Schwarz, and D. Trewin: 165–196. New York Wiley.
  18. Just, M.A., and P.A. Carpenter. 1980. “A Theory of Reading: From Eye Fixations to Comprehension.” Psychological Review 87: 329–354. DOI: https://doi.org/10.1037/0033-295X.87.4.329.
  19. Kaczmirek, L. 2005. “A Framework for the Collection of Universal Client Side Paradata (UCSP).” Available at: http://kaczmirek.de/ucsp/ucsp.html (accessed January 2021).
  20. Kamoen, N., B. Holleman, P. Mak, T. Sanders, and H. van den Bergh. 2017. “Why are Negative Questions Difficult to Answer? On the Processing of Linguistic Contrasts in Surveys.” Public Opinion Quarterly 81(3): 613–635. DOI: https://doi.org/10.1093/poq/nfx010
  21. Koffka, K. 1935. Principles of Gestalt psychology. New York: Harcourt.
  22. Krosnick, J.A. 1991. “Response Strategies for Coping with the Cognitive Demands of Attitude Measures in Surveys.” Applied Cognitive Psychology 5(3): 213–236. DOI: https://doi.org/10.1002/acp.2350050305.
  23. Krosnick, J.A., and D.F. Alwin. 1988. “A Test of the Form-Resistant Correlation Hypothesis. Ratings, Rankings, and the Measurement of Values.” Public Opinion Quarterly 52(4): 526–538. DOI: https://doi.org/10.1086/269128.
  24. Krosnick, J.A., and S. Presser. 2010. “Question and Questionnaire Design.” In Handbook of Survey Research, edited by P.V. Marsden, and J.D. Wright: 263–314. Emerald Group Publishing.
  25. Landis, J.R., and G.G. Koch. 1977. “The Measurement of Observer Agreement for Categorical Data.” Biometrics 33(1): 159–174. DOI: https://doi.org/10.2307/2529310.
  26. Liu, M., and A. Cernat. 2018. “Item-by-Item versus Matrix Questions: A Web Survey Experiment.” Social Science Computer Review 36(6): 690–706. DOI: https://doi.org10.1177/0894439316674459.10.1177/0894439316674459
  27. Mavletova, A., and M.P. Couper. 2015. “A Meta-Analysis of Breakoff Rates in Mobile Web Surveys.” In Mobile Research Methods: Opportunities and Challenges of Mobile Research Methodologies, edited by D. Toninelli, R. Pinter, and P. de Pedraza: 81–98. London: Ubiquity Press. DOI: http://dx.doi.org/10.5334/bar.f.10.5334/bar.f
  28. Mavletova, A., M.P Couper, and D. Lebedev. 2018. “Grid and item-by-item formats in PC and mobile web surveys.” Social Science Computer Review 36(6): 647–668. DOI: https://doi.org/10.1177%2F0894439317735307.10.1177/0894439317735307
  29. Mayerl, J. 2013. “Response Latency Measurement in Surveys. Detecting Strong Attitudes and Response Effects”. Survey Methods: Insights from the Field. Retrieved December 17, 2020, from https://surveyinsights.org/?p=1063. DOI: https://doi.org/10.13094/SMIF-2013-00005.
  30. McCarty, J.A., and L.J. Shrum. 2000. “The Measurement of Personal Values in Survey Research: A Test of Alternative Rating Procedures.” Public Opinion Quarterly 64(3): 271–298. DOI: https://doi.org/10.1086/317989.11114269
  31. Neuert, C.E., and T. Lenzner. 2017. “Incorporating Eye Tracking into Cognitive Interviewing to Pretest Survey Questions.” International Journal of Social Research Methodology 19(5): 501–519. DOI: https://doi.org/10.1080/13645579.2015.1049448.
  32. Peytchev, A. 2005. “How Questionnaire Layout Induces Measurement Error.” Paper presented at the 60th annual meeting of the American Association for Public Opinion Research, May, 2005. Miami Beach, FL,USA. Available at: http://www.websm.org/db/12/3636/Bibliography/Causes%20of%20Context%20Effects:%20How%20Questionnaire%20Layout%20Induces%20Measurement%20Error/.
  33. Rammstedt, B., and O.P. John. 2007. “Measuring Personality in One Minute or Less: A 10-Item Short Version of the Big Five Inventory in English and German.” Journal of Research in Personality 41: 203–212. DOI: https://doi.org/10.1016/j.jrp.2006.02.001.
  34. Rayner, K. 1998. “Eye Movements in Reading and Information Processing: 20 Years of Research.” Psychological Bulletin 124: 372–422. DOI: https://doi.org/10.1037/0033-2909.124.3.372.9849112
  35. Romano Bergstrom, J., and A. Schall. 2014. Eye Tracking in User Experience Design. San Francisco, CA: Morgan Kaufmann.
  36. Roßmann, J., T. Gummer, and H. Silber. 2018. “Mitigating Satisficing in Cognitively Demanding Grid Questions: Evidence from Two Web-Based Experiments.” Journal of Survey Statistics and Methodology 6(3): 376–400. DOI: https://doi.org/10.1093/jssam/smx020.
  37. Roßmann, J., and H. Silber. 2020. “Satisficing and Measurement Error.” In SAGE Research Methods Foundations, edited by P. Atkinson, S. Delamont, A. Cernat, J.W. Sakshaug, and R.A. Williams. London: SAGE Publications. DOI: https://dx.doi.org/10.4135/9781526421036912794.10.4135/9781526421036912794
  38. Silber, H., J. Roßmann, and T. Gummer. 2018. “When Near Means Related: Evidence from Three Web Survey Experiments on Inter-Item Correlations in Grid Questions.” International Journal of Social Research Methodology 21(3): 275–288. DOI: https://doi.org/10.1080/13645579.2017.1381478.
  39. Staub, A., and K. Rayner. 2007. “Eye movements and on-line comprehension processes.” In The Oxford Handbook of Psycholinguistics, edited by G. Gaskell: 327–342. Oxford, UK: Oxford University Press.10.1093/oxfordhb/9780198568971.013.0019
  40. Toepoel, V., M. Das, and A. van Soest. 2009. “Design of Web Questionnaires: The Effects of the Number of Items per Screen.” Field Methods 21(2): 200–213. DOI: https://doi.org/10.1177/1525822X08330261.
  41. Tourangeau R., M.P. Couper, and F. Conrad. 2004. “Spacing, Position, and Order. Interpretive Heuristics for Visual Features of Survey Questions.” Public Opinion Quarterly 68(3): 368–393. DOI: https://doi.org/10.1093/poq/nfh035.
  42. Tourangeau, R., and K. Rasinski. 1988. “Cognitive Processes Underlying Context Effects in Attitude Measurement.” Psychological Bulletin 103(3): 299–314. DOI: 10.1037/0033-2909.103.3.299.10.1037/0033-2909.103.3.299
  43. Tourangeau, R., L.J. Rips, and K. Rasinski. 2000. The Psychology of Survey Response. Cambridge: Cambridge University Press.10.1017/CBO9780511819322
  44. Wertheimer, M. 1923. Laws of organization in perceptual forms: A source book of Gestalt psychology. London: Routledge.
  45. W3C: World Wide Web consortium. 2018. “Web Content Accessibility Guidelines (WCAG) 2.1.” Available at: https://www.w3.org/TR/WCAG21/ (accessed June 2021).
  46. Zhang, C., and F. Conrad. 2014. “Speeding in Web Surveys: The Tendency to Answer very Fast and its Association with Straightlining.” Survey Research Methods 8(2): 127–135. DOI: https://doi.org/10.18148/srm/2014.v8i2.5453.
Language: English
Page range: 79 - 101
Submitted on: Jan 1, 2021
|
Accepted on: Aug 1, 2022
|
Published on: Mar 16, 2023
Published by: Sciendo
In partnership with: Paradigm Publishing Services
Publication frequency: 4 issues per year

© 2023 Cornelia E. Neuert, Joss Roßmann, Henning Silber, published by Sciendo
This work is licensed under the Creative Commons Attribution-NonCommercial-NoDerivatives 3.0 License.