Figure 1
Weight-of-evidence strategies in public discussions. The displayed materials represent the weight-of-evidence strategies outnumbering and forewarning as used in the respective experiments. Following the social-cue mechanism of outnumbering, the number of science advocates and deniers varied between conditions (3:3 vs. 5:1) in all experiments. Following the additional multiple-source mechanism, all guests in the discussion contributed as speakers in Experiment 3. The presented forewarning text is a translated version of the original German forewarning used in Experiments 2 and 3.
Table 1
Overview of outcome measures. Reliability of multiple-item scales is indicated by Cronbach’s alpha; numbers behind alphas relate to the respective experiments. All outcome measures were converted into percentages of maximum possible scores of the original scales (POMP), with higher values indicating a more positive attitude, greater confidence in vaccination and stronger intention.
| Construct | Incl. | Scale type | Wording | Source |
|---|---|---|---|---|
| attitude towards vaccination | Exp. 1–3 | mean score of 5-point rating scales (α1pre = .72; α1post = .85; α2pre = .81; α2post = .86; α3pre = .84; α3post = .88) | Please indicate how much you agree with the following statements. 1. Vaccinating against dysomeria is necessary. 2. Vaccinating against dysomeria is a good idea. 3. Vaccinating against dysomeria is beneficial. (1 = I strongly disagree, 5 = I strongly agree) | (Schmid & Betsch, 2019) |
| intention to get vaccinated | Exp. 1–3 | visual analog scale | If you had the opportunity to get vaccinated against dysomeria, what would you do? (1 = I will definitely not get vaccinated, 100 = I will definitely get vaccinated) | (Schmid & Betsch, 2019) |
| confidence in vaccination | Exp. 2–3 | mean score of 5-point rating scales (α2pre = .71; α2post = .83; α3pre = .77; α3post = .85) | Please indicate how much you agree with the following statements. 1. Vaccination against dysomeria is effective. 2. I am completely confident that the vaccine against dysomeria is safe. 3. Regarding the vaccine against dysomeria, I am confident that public authorities decide in the best interest of the community. (1 = I strongly disagree, 5 = I strongly agree) | (Betsch et al., 2018) |
Table 2
Weight-of-evidence strategies’ effects on changes in attitude. The results presented in Tables 2–4 are based on a 2 (rebuttal vs. advocate silent; between subjects) × 2 (outnumbering 5:1 vs. equal proportion of discussants 3:3; between subjects) × 2 (forewarning vs. no forewarning) × 2 (measurement before vs. after the debate; within subjects) repeated-measures ANOVA (Type II sum of squares). Significant effects are shown in boldface for the significance level of p < 0.05.
| Attitude | Experiment 1 n = 101 | Experiment 2 n = 390 | Experiment 3 n = 396 | ||||||
|---|---|---|---|---|---|---|---|---|---|
| Effects | F | p | η2p | F | p | η2p | F | p | η2p |
| Time | 99.16 | <.001 | .506 | 165.42 | <.001 | .302 | 132.11 | <.001 | .254 |
| Rebuttal × Time | 10.01 | .002 | .094 | 24.13 | <.001 | .059 | 50.91 | <.001 | .116 |
| Outnumbering × Time | 0.11 | .737 | .001 | 0.27 | .605 | .001 | 2.10 | .148 | .005 |
| Forewarning × Time | — | — | — | 7.52 | .006 | .019 | 1.11 | .293 | .003 |
| Rebuttal × Outnumbering × Time | 4.68 | .033 | .046 | 0.02 | .883 | <.001 | 2.83 | .093 | .007 |
| Rebuttal × Forewarning × Time | — | — | — | 0.47 | .495 | .001 | 0.16 | .687 | <.001 |
| Outnumbering × Forewarning × Time | — | — | — | 1.40 | .239 | .004 | 0.02 | .886 | <.001 |
| Rebuttal × Outnumbering × Forewarning × Time | — | — | — | 0.54 | .462 | .001 | 3.37 | .067 | .009 |
Table 3
Weight-of-evidence strategies’ effects on changes in intention. Significant effects are shown in boldface for the significance level of p < 0.05.
| Intention | Experiment 1 n = 101 | Experiment 2 n = 390 | Experiment 3 n = 396 | ||||||
|---|---|---|---|---|---|---|---|---|---|
| Effects | F | p | η2p | F | p | η2p | F | p | η2p |
| Time | 43.98 | <.001 | .312 | 156.64 | <.001 | .291 | 92.10 | <.001 | .192 |
| Rebuttal × Time | 3.65 | .059 | .036 | 32.37 | <.001 | .078 | 57.58 | <.001 | .129 |
| Outnumbering × Time | 0.19 | .661 | .002 | 0.15 | .703 | <.001 | 2.44 | .119 | .006 |
| Forewarning × Time | — | — | — | 14.75 | <.001 | .037 | 6.92 | .009 | .018 |
| Rebuttal × Outnumbering × Time | 3.36 | .070 | .033 | 0.03 | .868 | <.001 | 0.03 | .865 | <.001 |
| Rebuttal × Forewarning × Time | — | — | — | <0.01 | .948 | <.001 | 0.60 | .441 | .002 |
| Outnumbering × Forewarning × Time | — | — | — | 0.03 | .864 | <.001 | 1.11 | .293 | .003 |
| Rebuttal × Outnumbering × Forewarning × Time | — | — | — | 0.07 | .790 | <.001 | 0.25 | .615 | .001 |
Figure 2
Effects from outnumbering on damage from science denialism in public discussions in Experiment 1. The results reveal that outnumbering mitigated the damage from denialism only if a rebuttal was delivered. The y-axes represent mean changes in attitude (left graph) and in intention (right graph) in POMP values. Descriptive data are provided in Supplementary Tables 2–4. The x-axes represent experimental conditions. Error bars represent 95% CIs. Dots indicate changes in individual participants’ attitudes and intentions.
Table 4
Weight-of-evidence strategies’ effects on changes in confidence. Significant effects are shown in boldface for the significance level of p < 0.05.
| Confidence | Experiment 2 n = 390 | Experiment 3 n = 396 | ||||
|---|---|---|---|---|---|---|
| Effects | F | p | η2p | F | p | η2p |
| Time | 96.20 | <.001 | .201 | 61.59 | <.001 | .137 |
| Rebuttal × Time | 46.15 | <.001 | .108 | 70.75 | <.001 | .154 |
| Outnumbering × Time | 1.35 | .246 | .004 | 0.81 | .369 | .002 |
| Forewarning × Time | 10.44 | .001 | .027 | 8.76 | .003 | .022 |
| Rebuttal × Outnumbering × Time | 1.72 | .190 | .004 | 0.03 | .853 | <.001 |
| Rebuttal × Forewarning × Time | 2.27 | .132 | .006 | 0.04 | .949 | <.001 |
| Outnumbering × Forewarning × Time | 0.04 | .849 | <.001 | 1.14 | .287 | .003 |
| Rebuttal × Outnumbering × Forewarning × Time | 0.49 | .485 | .001 | 2.21 | .138 | .006 |
Figure 3
Effects from outnumbering and forewarning on damage from science denialism in public discussions in Experiment 2. The results reveal a significant mitigation in damage from denialism on all outcome measures when forewarning was used. The mitigating effect was not a function of whether the advocate uses a rebuttal or remains silent. Comparison of the lower (3:3) and upper (5:1) panels reveals no evidence that outnumbering mitigated the damage from denialism on any outcome measure. The y-axes represent mean changes in attitude (left graph), intention (centre graph) and confidence (right graph) in POMP values. Descriptive data are provided in Supplementary Tables 2–4. The x-axes represent experimental conditions. Error bars represent 95% CIs. Dots indicate changes in individual participants’ outcome measures.
Figure 4
Effects from outnumbering and forewarning on damage from science denialism in public discussions in Experiment 3. Results reveal a significant mitigation of damage from denialism in individuals’ intention to get vaccinated and confidence in vaccination when forewarning was used. The mitigating effect was not a function of whether the advocate uses a rebuttal or remains silent. Comparison of the lower (3:3) and upper (5:1) panels reveals no evidence that outnumbering mitigated the damage from denialism on any outcome measure. The y-axes represent mean changes in attitude (left graph), intention (centre graph) and confidence (right graph) in POMP values. Descriptive data are provided in Supplementary Tables 2–4. The x-axes represent experimental conditions. Error bars represent 95% CIs. Dots indicate individual changes in individual participants’ outcome measures.