On the Limitations of Manipulation Checks: An Obstacle Toward Cumulative Science

Bless H. Burger A. M. A closer look at social psychologists’ silver bullet: Inevitable and evitable side effects of the experimental approach Perspectives On Psychological Science 2016 11 296 308 10.1177/1745691615621278

Brandt M. J. IJzerman H. Dijksterhuis A. Farach F. J. Geller J. Giner-Sorolla R. van’t Veer A. The Replication Recipe: What makes for a convincing replication? Journal of Experimental Social Psychology 2014 50 217 224 10.1016/j.jesp.2013.10.005

Brewer M. B. Crano W. D. Reis H. T. Judd C. M. Research design and issues of validity Handbook of Research Methods in Social and Personality Psychology 2014 2nd ed. New York, NY, US Cambridge University Press 11 26

Chaiken S. Heuristic versus systematic information processing and the use of source versus message cues in persuasion Journal of Personality and Social Psychology 1980 39 752 766 10.1037/0022-3514.39.5.752

Cohen J. Things I have learned (so far) American Psychologist 1990 45 1304 1312 10.1037/0003-066X.45.12.1304

Cook T. D. Campbell D. T. Peracchio L. Dunnette M. D. Hough L. M. Quasi Experimentation Handbook of Industrial and Organizational Psychology 1990 Palo Alto, CA Consulting Psychologists Press

Deutsch M. Krauss R. M. Theories in Social Psychology 1965 NY Basic Books

Fiedler K. Schott M. Meiser T. What mediation analysis can (not) do Journal of Experimental Social Psychology 2011 47 1231 1236 10.1016/j.jesp.2011.05.007

Finkel E. J. Eastwick P. W. Reis H. T. Best research practices in psychology: Illustrating epistemological and pragmatic considerations with the case of relationship science Journal of Personality and Social Psychology 2015 108 275 297 10.1037/pspi0000007

Flake J. K. Pek J. Hehman E. Construct Validation in Social and Personality Research: Current Practice and Recommendations Social Psychological and Personality Science 2017 10.1177/1948550617693063

Foschi M. Webster M. J. Sell J. Hypotheses, operationalizations, and manipulation checks Laboratory Experiments in the Social Sciences 2014 2nd ed. San Diego, CA, US Elsevier Academic Press 247 268 10.1016/B978-0-12-404681-8.00011-X

Haslam S. A. McGarty C. Sansone C. Morf C. C. Panter A. T. Experimental design and causality in social psychological research Handbook of Methods in Social Psychology 2003 Thousand Oaks, CA Sage 237 264

Hüffmeier J. Mazei J. Schultze T. Reconceptualizing replication as a sequence of different studies: A replication typology Journal of Experimental Social Psychology 2016 66 81 92 10.1016/j.jesp.2015.09.009

Jacoby J. Sassenberg K. Interactions do not only tell us when, but can also tell us how: Testing process hypotheses by interaction European Journal of Social Psychology 2011 41 10.1002/ejsp.762

John L. K. Loewenstein G. Prelec D. Measuring the prevalence of questionable research practices with incentives for truth-telling Psychological Science 2012 23 524 532 10.1177/0956797611430953

Kühnen U. Manipulation checks as manipulation: Another look at the ease-of-retrieval heuristic Personality and Social Psychology Bulletin 2010 36 47 58 10.1177/0146167209346746

Maxwell S. E. The Persistence of Underpowered Studies in Psychological Research: Causes, Consequences, and Remedies Psychological Methods 2004 9 147 163 10.1037/1082-989X.9.2.147

Messick S. Standards of validity and the validity of standards in performance assessment Educational Measurement: Issues and Practice 1995 14 5 8 10.1111/j.1745-3992.1995.tb00881.x

O’Keefe D. J. Message Properties, Mediating States, and Manipulation Checks: Claims, Evidence, and Data Analysis in Experimental Persuasive Message Effects Research Communication Theory 2003 13 251 274 10.1111/j.1468-2885.2003.tb00292.x

Platt J. R. Strong inference Science 1964 146 347 353 10.1126/science.146.3642.347

Reis H. T. Judd C. M. Handbook of Research Methods in Social and Personality Psychology 2014 2nd ed. New York Cambridge University Press

Shadish W. R. Cook T. D. Campbell D. T. Experimental and quasi-experimental designs for generalized causal inference 2002 Boston, MA, US Houghton, Mifflin and Company

Sigall H. Mills J. Measures of independent variables and mediators are useful in social psychology experiments: But are they necessary? Personality and Social Psychology Review 1998 2 218 226 10.1207/s15327957pspr0203_5

Stroebe W. Strack F. The alleged crisis and the illusion of exact replication Perspectives on Psychological Science 2014 9 59 71 10.1177/1745691613514450

Wilson T. D. Aronson E. Carlsmith K. Fiske S. T. Gilbert D. T. Lindzey G. The art of laboratory experimentation Handbook of Social Psychology 2010 NJ Wiley 1 51 81 10.1002/9780470561119.socpsy001002