Figure 1
The amount of effect size overestimation after truncation for different statistical power levels (for the Student t-test).
Table 1
Overview of included journals for the four APA-disciplines and the number of articles analyzed within each journal and year. The JIF quartiles of each journal are displayed between brackets for the time periods 2015–2016 (first position) and 2020–2021 (second position).
| DISCIPLINE | JOURNAL | YEAR OF PUBLICATION | |||
|---|---|---|---|---|---|
| 2015 | 2016 | 2020 | 2021 | ||
| Clinical | Psychological Trauma Theory Research Practice and Policy (Q2 – Q2) | 20 | 20 | ||
| Rehabilitation Psychology (Q3 – Q3) | 20 | 20 | |||
| Health Psychology (Q1 – Q1) | 20 | 20 | |||
| Journal of Consulting and Clinical Psychology (Q1 – Q1) | 20 | 20 | |||
| Journal of Experimental and Clinical Psychopharmacology (Q2 – Q2) | 20 | 20 | |||
| Psychological Services (Q3 – Q3) | 20 | 20 | |||
| Experimental | Journal of Experimental Psychology: General (Q1 – Q1) | 20 | 20 | ||
| Journal of Comparative Psychology (Q2 – Q3) | 20 | 20 | |||
| Journal of Experimental Psychology: Human Perception and Performance (Q2 – Q2) | 20 | 20 | |||
| Emotion (Q1 – Q1) | 20 | 20 | |||
| Journal of Experimental Psychology: Learning, Memory, and Cognition (Q2 – Q2) | 20 | 20 | |||
| Canadian Journal of Experimental Psychology (Q4 – Q4) | 20 | 20 | |||
| Social | Journal of Personality and Social Psychology (Q1 – Q1) | 20 | 20 | ||
| Group Dynamics: Theory, Research, and Practice (Q4 – Q4) | 16 | 13 | |||
| Law and Human Behavior (Q1 – Q2) | 20 | 20 | |||
| American Journal of Orthopsychiatry (Q1 – Q1) | 20 | 20 | |||
| Cultural Diversity & Ethnic Minority Psychology (Q2 – Q2) | 20 | 20 | |||
| Psychology of Men & Masculinities (Q2 – Q3) | 20 | 20 | |||
| Educational | Journal of Educational Psychology (Q1 – Q1) | 20 | 20 | ||
| Training and Education in Professional Psychology (Q3 – Q4) | 13 | 9 | |||
| Journal of Diversity in Higher Education (Q4 – Q2) | 4 | 8 | |||
| Journal of Counseling Psychology (Q1 – Q1) | 20 | 20 | |||
| School Psychology (Quarterly) (Q1 – Q2) | 20 | 20 | |||
| Canadian Journal of Behavioural Science (Q4 – Q1) | 20 | 20 | |||
Figure 2
Evolution of power analysis prevalence in general (panel A), per discipline (panel B), and per JIF quartile (panel C).
Figure 3
Number of articles with and without a power analysis across the two time periods within each of the included journals.
Figure 4
Ratio of a priori power analysis versus sensitivity analysis over time.
Figure 5
Standardized versus unstandardized effect sizes used in power analysis.
Note. Panel A and B are based on the 173 studies that report an a priori power analysis or sensitivity analysis. Panel C is based on the 112 articles that employed (and reported the type of) standardized effect in the power analysis and shows the values used for each type of standardized effect.8 The mean of the distribution is indicated by the cross.
Figure 6
Distribution of the (log-transformed) sample sizes A) per type of power analysis, aggregated across time periods, disciplines, and JIF quartiles (no power analysis, n = 725; a priori power analysis, n = 119;, sensitivity analysis, n = 54; post hoc power analysis, n = 5), and B) for articles where the sample size was either pre-determined (2015–2016, n = 428; 2020–2021, n = 356) or power analysis-based (2015–2016, n = 25; 2020–2021, n = 94) per time period.
Table 2
Overview of the minimal detectable effect size dav with a total sample size of 150 participants (ppt) to reach a power of 80% for different values of correlation between the repeated measures (not applicable for the independent samples t-test). Results are based on 5000 simulations.
| SETTING | N | CORRELATION | ||
|---|---|---|---|---|
| r = 0.3 | r = 0.5 | r = 0.7 | ||
| Paired t-test | Same 150 ppt in both conditions | 0.27 | 0.23 | 0.18 |
| Independent t-test | Different 75 ppt per group | 0.46 | ||
| ANOVA 2 × 2 rep. meas. | Same 150 ppt in all 4 conditions | 0.39 | 0.33 | 0.26 |
| ANOVA 2 × 2 split-plot | Different 75 ppt per group | 0.54 | 0.46 | 0.36 |