Have a personal or library account? Click to login
How to Run Behavioural Experiments Online: Best Practice Suggestions for Cognitive Psychology and Neuroscience Cover

How to Run Behavioural Experiments Online: Best Practice Suggestions for Cognitive Psychology and Neuroscience

By: Nathan Gagné and  Léon Franzen  
Open Access
|Jan 2023

Figures & Tables

Table 1

List of costs and benefits of online studies. To facilitate evaluation of the importance and consequences of items, costs and benefits indicate a potential trade-off any experimenter would engage in when conducting research online.

COSTS
  • Limited control of testing environment, higher risk of distractions

  • Limited possibility for intervention during testing in mass online testing, physical absence of researchers

  • Noisy data

  • Increased lack of motivation and attention due to extended general computer time usage

  • Higher dropout rates

  • Compensation of cognitive or perceptual differences by expensive hardware

  • Varying computer processing capabilities, timing inaccuracies for brief stimulus display

  • Greater potential for cheating (providing invalid data) and participant fraud (pretending to be someone one is not)

  • Potentially greater temptation to pay participants non-adequately, no in-person interaction

  • If a bug is present and data was not collected in batches, much data needs to be discarded

  • Reliable access to online studies required (internet access and proper equipment)

  • Sampling of non-naïve participants (particularly via platforms)

BENEFITS
  • No physical presence needed

  • Time reduction for experimenter(s)’ testing time commitment – without supervision requirements

  • Rapid data collection (in parallel)

  • Easily collect more data for larger sample sizes

  • Increased possibilities for recruitment, generally more accessible for most people (depending on socioeconomic background)

  • Collect more representative and heterogenous samples (depending on sampling method)

  • Increased autonomy for participants about the time and location of participation

  • Reduced social pressure and feelings of obligation to finish a study

  • Access for more trainees to run their own study earlier in their career

  • No lab equipment and space needed; all-in-one solutions provided by platforms

  • Potential for increased data anonymity (e.g., for special populations)

  • Reduced equipment and research costs

spo-3-1-34-g1.png
Figure 1

Experimental design, dyslexia scores, and data screening visualisations. a) Schematic of the fictional paradigm and trial sequence. First, participants saw an encoding period in which they encoded the location of various stimuli of the same type presented in different locations. A total of eight fixed locations were available on each trial and either three or six locations were filled with items. Then, a spatial retrieval cue was followed by a decision screen presenting three different stimuli of the same type. Participants were instructed to respond as quickly as possible using their physical keyboard. Further experimental details are available from the study’s Open Science Framework repository (Franzen et al., 2022). b) Raincloud plots (Allen et al., 2019) of the simulated dyslexia checklist scores that served as screening tool after the removal of excluded participants. Dyslexia data is depicted in blue colour, while the yellow colour indicates data of the control group. Overlaid boxplots show the median, upper and lower quartile. A maximum score of 40 was used to delineate between participants included in the control group and others without an official dyslexia diagnosis who were excluded from further analyses and this plot. c) Scatterplots of accuracy as a function of reaction time across all conditions (top: measures of central tendency; bottom standard deviations). One mean value per participant computed across mean accuracy or median reaction times of both working memory conditions. Colours indicate groups. Blue dots depict single-participant values of the dyslexia group, whereas yellow dots depict values of the control group. Dashed lines indicate the lower and upper bounds of the 95% confidence interval for all participants included in the analyses.

spo-3-1-34-g2.png
Figure 2

Overview of suggestions for online research by study stage. Flowchart following the workflow of experimental studies in cognitive psychology and neuroscience.

DOI: https://doi.org/10.5334/spo.34 | Journal eISSN: 2752-5341
Language: English
Submitted on: Jan 23, 2022
|
Accepted on: Dec 23, 2022
|
Published on: Jan 4, 2023
Published by: Ubiquity Press
In partnership with: Paradigm Publishing Services
Publication frequency: 1 issue per year

© 2023 Nathan Gagné, Léon Franzen, published by Ubiquity Press
This work is licensed under the Creative Commons Attribution 4.0 License.