Limited and Mixed Evidence for System-Sanctioned Change to Protect the Environment: A Replication Study

System Justification as a Barrier to Pro-Environmentalism

System justification theory (SJT; Jost & Banaji 1994) provides one answer for why we endure existing social, economic, and political arrangements that disproportionately disadvantage some, even if we are the ones facing the disadvantage (Jost et al. 2004). System justification is a motivation to rationalise and justify the existing system as fair, good, and inevitable. People with system justification tendencies may actively defend the status quo by, for example, opposing redistributive policy and accepting stereotypes that reinforce inequality (Jost et al. 2004). System justification therefore dissuades people from taking action that challenges the status quo, even if doing so would improve it to their benefit.

Despite its consequences, engaging in system justification is theorised to reduce the anxiety and uncertainty of current unequal sociopolitical systems (Jost & Hunyady 2002). Jost and Hunyady (2002) summarise that system justification prevents stress by giving the impression of a stable and just society; serves as a coping resource by making people feel hopeful and in control; and as a coping response when exposed to threat (thus, threats may bolster this defensive motivation). Putting these functions in the context of climate change, system justification may entail denial of the problem or acceptance of unconstructive hope, as examples.

Environmental problems can be perceived as a threat to the system in several ways. Feygina et al. (2010) argue that being motivated to see the current economic, political, and social systems as legitimate and beneficial prevents people from acknowledging the scale of impending environmental disaster. Accepting that these favoured systems have caused such destruction would challenge perceptions of legitimacy and fairness. They also noted the conflict between environmental protection and the current economic model; environmental solutions may unacceptably threaten the prevailing status quo that those engaging in system justification seek to defend. Supporting the theorised link between system justification and environmentalism, Feygina et al. (2010) showed that stronger endorsement of system justification corresponds with greater denial of environmental problems (Studies 1 and 2) and less frequent environmental behaviour (Study 2).

Building on Feygina et al.’s (2010) work, Campbell and Kay (2014) argued that the political opinion gap on climate change can be explained by solution aversion: a motivated denial of climate change stemming from the view that pro-climate reforms entail unacceptable threats. They showed that Republican climate denial is partly explained by the perception that climate solutions will adversely affect the American economy. Clarke et al. (2019) similarly found that the effects of right-wing ideological attitudes (in this case, right-wing authoritarianism and social dominance orientation) on climate denial were mediated by ‘mitigation threat’: the perception that climate policy threatens the social and economic status quo. Indeed, when environmental protection is framed as having negative implications for economic growth, there is a stronger ideological resistance (Harring & Sohlberg 2016). Ideologies that favour the current system, including system justification, may therefore be important barriers to pro-environmentalism.

The Possibility of System-Sanctioned Change

If anti-environmentalism stems from a belief that acceptance of environmental problems (and their solutions) poses a threat to the system, then system-supportive reforms may be more palatable. This is the logic behind Feygina et al.’s (2010, Study 3) experimental research. Feygina et al.’s study began by reminding all participants (control condition and experimental condition) of their dependence on the country they live in. They did this using a paragraph designed to make participants’ relationship to the prevailing socioeconomic system salient. The text (see Table 1) stressed the importance of country of residence for participants’ life and wellbeing, including their financial wellbeing (e.g., via taxes they must pay, jobs available to them), social and personal wellbeing (e.g., quality of social services and access to leisure activities), and even their chances of finding happiness with a life partner. Participants then read an excerpt about research interest in environmentalism. After this, only the experimental (system-preservation) condition participants read the following additional text: ‘Being pro-environmental allows us to protect and preserve the American way of life. It is patriotic to conserve the country’s natural resources’. This short message avoided framing environmentalism as a challenge to the system, and instead positioned environmental action as a way to preserve the system. That is, it framed environmentalism in a way that works ‘with rather than against system justification motivation’ (Feygina et al. 2010, p. 333).

Although the manipulation was brief, Feygina et al. reported that it was effective. Specifically, they described significant interactions between system justification and messaging condition on pro-environmental intentions and the likelihood of signing a petition. System justification predicted lower behavioural intentions and lower likelihood of signing petitions in a control condition, and this association vanished for the system-preservation experimental condition.

Justification for Replication

There is a growing attention to the importance of replication of research results. In general, framing studies in environmental communication are rarely replicated, and several recent replication failures (e.g., Kim et al. 2021, 2023; Stanley et al. 2021a) highlight that our field is not immune to replicability concerns. When considering the changes in environmental awareness and narratives in society, it would be crucial to not only develop new communication frames, but to also re-evaluate the effectiveness of the ones that have been developed in previous research.

The current empirical evidence for system-preservation framing is limited to Feygina et al. (2010, Study 3), which was conducted more than a decade ago and had some limitations, and a global mega-study (Vlasceanu et al. 2024). Feygina et al. (2010, Study 3) used a small (N = 41, 73% women) student sample from New York University (NYU), spread across two conditions. They described finding a significant system justification – experimental condition interaction for both pro-environmental behaviour intentions and signing environmental petitions. However, these key findings were interactions with a continuous variable (system justification), suggesting the impact of the framing depends on the level of system justification, and there is no detail about the distribution of responses on this scale. Mean levels of system justification tend to be low (Vargas-Salfate et al. 2018). While Feygina et al. (2010) did not report descriptive statistics for system justification in their study, Cichocka and Jost (2014) published a review of mean levels of system justification that includes data collected at a similar time using NYU students. In all cases, mean scores are below the scale midpoint of the same system justification measure.¹

To interpret the significant overall interaction effect on intentions to engage in pro-environmental behaviours, Feygina et al. (2010, Study 3) reported results from two follow-up tests, which we summarise in Table 2. They examined those high and low in system justification (defined by M ± 1SD mean), reporting higher pro-environmental intentions for high system justifiers in the experimental condition: p = .09. This p-value is above the conventional alpha level, and a p-value reported as p > .10 was taken as evidence that low system justifiers did not respond differently to the messages. They also examined the relationship between system justification and pro-environmental intentions, separately by condition. They reported a negative association between system justification and pro-environmental intentions in the control condition (reported as ‘marginal’, p < .09), which ‘was eliminated’ (p. 334) in the system-preservation condition (p = .14).

To follow up the significant condition × system justification interaction on the likelihood of signing pro-environmental petitions, Feygina et al. (2010, Study 3) described a marginal negative relationship between system justification and likelihood of signing petitions in the control condition, and no relationship between these variables in the system-preservation condition. Statistics were not completely reported, though a figure showing the likelihood of signing petitions at high (M + 1SD) and low (M – 1SD) levels of system justification implies a greater likelihood of signing petitions for high system justifiers in the system-preservation condition than in the control condition. Besides the interaction itself (where p = .03), no p-values were reported for the analyses on the likelihood of signing petitions. Therefore, while Feygina and colleagues (2010, Study 3) developed an interesting and well-reasoned theoretical account of the system justification-environmentalism association that has had a profound impact on the field, the evidence they presented does not yet make a compelling case for system-preservation framing. An intervention based on system-preservation framing was recently included in Vlasceanu et al.’s (2024) 63-country study examining the effects of 11 pro-environmental interventions. Their intervention emphasised participants’ dependence on where they live, gave examples of the ways in which climate change is threatening this way of life, and invited pro-environmental action to protect and preserve their way of life. The intervention showed small effects on belief in climate change, policy support, and intentions to share climate information on social media by (each increased by 0.85%, 0.74%, and 5.81%, respectively), and did not affect effort in a tree planting activity. Reanalysis of these data across political liberals and conservatives indicated that the intervention increased climate beliefs of both groups, increased climate policy support only among political liberals, and backfired to decrease tree planting efforts among political conservatives (Berkebile-Weinberg et al. 2024). These mixed results for interactions with political orientation further warrant a closer replication of Feygina et al.’s study that directly measures system justification.

Current Study

We conducted a close replication of Feygina et al. (2010, Study 3) using a larger, general U.S. population sample. We tested the effects of system justification across conditions (control, system-preservation condition) on intentions to act pro-environmentally and on agreement to sign pro-environmental petitions. This was not a direct replication. Deviations from the original study protocol are summarised in Table 3 and included a shift to online recruitment (the original experiment was completed in a controlled lab setting) and implementing strategies to ensure the quality of data collected in this setting (i.e., attention and naivety checks). Given that we were conducting the replication more than ten years on from the original, the petitions were also updated to reflect environmental issues still relevant today.

A successful replication of Feygina et al.’s (2010, Study 3) conclusions ought to show a significant interaction between system justification and experimental condition on pro-environmental behavioural intentions. Probing this interaction using the same two methods as the original study should show that for participants high in system justification (defined by Feygina et al. as scoring at M + 1SD), behavioural intentions are significantly higher among participants assigned to the system-preservation condition than the control condition. For those with low levels of system justification (M – 1SD), there should be no significant differences in responses across conditions. This pattern should also be reflected in the simple slopes analysis: system justification ought to be significantly, negatively related with pro-environmental behavioural intentions in the control condition, and unrelated in the system-preservation condition. This would provide evidence that the system-preservation frame successfully eliminated the negative effect of system justification on environmental intentions.

As an extension to the replication attempt, we also introduced a second control condition, which we refer to as the ‘true control’ condition. In it, participants received no message or information invoking salience of the system. Jost et al. (2004) explain that people ‘want to hold favourable attitudes about social and political systems that affect them’ (p. 887). The initial text Feygina et al. (2010, Study 3) showed all participants was designed to emphasise one’s reliance on the system and may have affected system justification tendencies to shape responses in the experiment. Specifically, people who feel more dependent on the system should be more motivated to defend it (Feygina et al. 2010), and in turn, respond less pro-environmentally. A similar ‘system dependence’ message was used in Hennes et al. (2016) to manipulate system justification, with effects on the accuracy of recall of information of climate change. Thus, the salience of the system manipulation could have depressed pro-environmental responding in both original study conditions, meaning that the positive results in the system-preservation condition could reflect a restoration of typical pro-environmental responding, rather than a true increase relative to baseline. We test this possibility by comparing responses within the original study conditions to our true control condition. Finally, if the system-preservation framing successfully engages political conservatives, then the patterns described above ought to replicate with political orientation in place of system justification.

Participants

Feygina et al.’s (2010, Study 3) sample was 41 New York University students (73% women). We simulated power at a critical p-value = .05 in a two-tailed test using the paramtest package (Hughes 2017) in R (R Core Team, 2022), based on the effect sizes reported in the original study for pro-environmental behavioural intentions (R² = .10, b1 = –0.36, b2 = 0.06, b3 = 0.49; see https://osf.io/6b8kz for the R script and https://osf.io/q7wr9 for the output). A final sample of 170 would secure 90% power for this interaction effect. Although our goals involving the true control condition were exploratory, we expected that the salience of system manipulation could heighten the negative association between system justification and pro-environmentalism in the original study control condition and amplify the positive moderating effects of system-preservation framing between system justification and pro-environmentalism (see Diamond 2020). Based on these expectations, we simulated a graph and conducted the power analysis (see https://osf.io/q7wr9), R² = .10, b1 = –0.25, b2 = –0.20, b3 = –0.15, b4 = –0.35, b5 = 0.50; see https://osf.io/295wt for the R script and https://osf.io/q7wr9 for the output and simulated graph). To attain 90% power to detect significant interaction effects in the analysis including all the three conditions, our minimum sample size was 480. We increased this figure by 20% (to N = 576) to ensure sufficient power after applying our preregistered exclusion criteria.

Between 25–26th August 2022, we recruited participants living in, and identifying their nationality as, the United States of America using Prolific’s balanced sample feature to recruit men and women equally. Participants received 0.97GBP for completing the 7-minute Qualtrics survey.² We elected to recruit participants via Prolific because comparisons to other platforms suggest the data collected through Prolific are of higher quality (Peer et al. 2021). We advertised the survey using the title ‘Evaluating the clarity of news articles’ to align with the cover story presented in the original experiment, with the Participant Information Sheet claiming that the project explores ‘how the language used in newspaper articles affects how clear and interesting this information is to readers’. Following our preregistered exclusion criteria, we removed three participants who withdrew consent and six who did not identify their nationality as the U.S. We report analyses with the remaining 567 participants,³ ranging in age from 18 to 93 years (M = 37.06, SD = 13.92; 48.68% women, 47.97% men, 3.35% non-binary or prefer not to say) and predominantly identifying their race as White (76.37%), followed by Asian (9.00%), Black or African American (6.88%), Some other race (4.23%), Prefer not to say (1.76%), American Indian or Alaska Native (1.24%), and Native Hawaiian or Other Pacific Islander (0.53%). The sample leaned liberal (M = 34.62, SD = 27.85 on a 0–100 liberal-conservative scale).

Salience of the System Manipulation

Participants in the original study conditions (system-preservation condition, control condition) read instructions obscuring the aims of the study (‘First, please read this excerpt from a newspaper article. Later, you will be asked some questions about it, so please read it carefully.’). Next, a paragraph noted participant’s dependence on the country they live in to make their relationship to the prevailing socioeconomic system salient (see Table 1).

Environmental Passage and Manipulation

Original study participants read: ‘Researchers have always been interested in the state of the natural environment, and have paid attention to how it has changed over the years. Today, researchers are especially interested in the relationship between people and the environment.’ In the system-preservation condition, participants additionally read: ‘Being pro-environmental allows us to protect and preserve the American way of life. It is patriotic to conserve the country’s natural resources.’ To bolster the cover story, the participants then rated in both conditions how clear, easy to understand, interesting, and persuasive, these passages were. ‘True control’ condition participants began the experiment with the questions below.

Pro-Environmental Behaviour Intentions and System Justification

As in the original study, we used the 10-item scale that measures intentions to engage in pro-environmental behaviour (α = .93, item example: ‘I intend to join and provide financial support to pro-environmental organisations in the near future’), and the 8-item system justification scale (α = .88, item example: ‘Everyone has a fair shot at wealth and happiness’, Kay & Jost 2003), each rated from 1 (‘strongly disagree’) to 9 (‘strongly agree’). Feygina et al. (2010) reasoned that system justification ought to be measured towards the end of the experiment to further obscure the study aims and reported no difference in system justification by condition. Our results are consistent with all three conditions compared: F(2, 564) = 0.65, p = .521, η² = .002.

Pro-Environmental Petitions

Feygina et al. (2010) concluded their study using a false debrief, then presented participants with seven pro-environmental petitions ostensibly arranged by an on-campus environmental group and unrelated to the study. Due to the difficulty of displaying an onscreen false debrief followed by petitions that are believably unlinked to the study, our method deviated here: instead, we updated the original petitions by removing outdated references (see https://osf.io/gbfnh) and asked participants whether they would like to sign each petition (yes/no), with (false) information that they would receive links to their chosen petitions at the end of the survey. Following Feygina et al. (2010), we recoded responses into ordered categories (see Table 2).

Survey End

We recorded age, gender, political orientation, and subjective social status using the MacArthur Scale (Adler et al. 2000). We also asked two questions to verify participants met the eligibility criteria for the study (located in, and identifying their nationality as, the United States). The in-person nature of the original experiment gave the researchers more experimental control and oversight of participants. To ensure a similar level of data quality in the online, anonymous setting, we captured time spent on the survey and included a multiple-choice attention check question, and two yes/no closed-ended naivety check questions based on Kim et al. (2021), detailed in the Supplementary Materials. Finally, we disclosed the true study aims and gave participants the option to withdraw from the study in line with our ethics protocol.

Descriptive Statistics of Outcome Variables in Each Condition

Table 4 presents descriptive statistics for PEB intentions and total number of pro-environmental petitions signed across each experimental condition. It shows that on average, participants scored just below the scale midpoint in their intentions to engage in pro-environmental behaviour, indicating slight disagreement that they will take the actions specified in the measure. Of the seven pro-environmental petitions they were presented, participants on average agreed to sign just over three.

Close Replication: Effects in the Original Study Control Condition vs System-Preservation Condition

We present results from analyses that aimed to replicate those reported in Feygina et al. (2010, Study 3) alongside the original study findings in Table 2. In contrast to our predictions and Feygina and colleague’s findings, the relationship between system justification and PEB intentions did not depend on messaging condition in our study. We appeared to replicate the significant interaction between system justification and condition on likelihood of signing petitions. Table 2 shows that simple slope analyses identified a significant negative effect of system justification on likelihood of signing petitions in the original study control condition, and no effect in the system-preservation condition. This finding aligns with the pattern of results described by Feygina et al. (2010, Study 3) and indicates that in the absence of a system-preservation frame, a stronger tendency to justify the system predicts a lower likelihood of signing environmental petitions; an effect that is non-significant with the frame.

Proceeding with robustness checks, we found that the significant interaction between system justification and condition on the number of petitions signed remained even when repeating these analyses using the actual count of petitions signed, using negative binomial regression (b = 0.16, SE = 0.08, p = .035).⁴ There was a significant, negative relationship between system justification and petitions signed in the original study control condition (b = –0.53, SE = 0.18, 95% Asymptotic CI [–0.88, –0.18], p = .003), and no significant relationship in the system-preservation condition (b = –0.01, SE = 0.17, 95% Asymptotic CI [–0.35, 0.32], p = .930).

The difference in the number of petitions signed across conditions was not found for participants at M – 1SD (2.23) and M + 1SD (5.30) on system justification. However, we used exploratory Johnson-Neyman analyses to examine at what level of system justification participants responded differently depending on the condition. This analysis identified a significant effect of condition (original study control versus system-preservation) on the number of petitions signed among those with extremely high levels of system justification (i.e., values above 7.48 on the scale, which ranged from 1 to 9). This result ought to be interpreted with caution, because it is based on a fitted model, and individuals with extremely high values on system justification are rare.

Extension to the Replication: Analyses with a True Control Condition

Pro-Environmental Behaviour Intentions Across Three Conditions

We then examined PEB intentions across all three conditions, setting true control condition as a reference group. This analysis returned two significant interactions: the first suggests the effect of system justification on PEB intentions depends on whether participants are in the original or true control condition (b = 0.29, p = .015), and the second suggests the effect on PEB intentions depends on whether participants are in the system-preservation experimental condition or true control condition (b = 0.34, p = .003). Analysis of simple slopes showed a significant negative relationship between system justification and PEB intentions only in the true control condition (b = –0.24, SE = .08, p = .003). The association was non-significant in both the original study control condition (b = 0.05, SE = .09, p = .553) and system-preservation condition (b = 0.11, SE = .09, p = .213), raising the intriguing possibility that both original study conditions eliminated the negative effect of system justification on PEB intentions (see Figure 1).

Figure 1

Comparing PEB intentions of those with lower system justification (M – 1SD, 2.18) highlights significantly lower PEB intentions in the system-preservation condition (M = 4.30, SE = 0.20) than the true control condition (M = 4.97, SE = 0.18), t(561) = –2.49, p = .025. The system-preservation message therefore lowered the PEB intentions of those rejecting system justification. For those higher in system justification (M + 1SD, 5.37), PEB intentions were not significantly higher in the system-preservation condition (M = 4.64, SE = 0.18) than in the true control condition (M = 4.21, SE = 0.18), t(561) = 1.65, p = .179. Although these comparisons at M ± 1SD align with Feygina et al.’s (2010, Study 3) analytic approach, the values are somewhat arbitrary for probing the interaction. We therefore conducted an exploratory Johnson-Neyman analysis to identify at what level(s) of system justification there is a significant effect of condition on PEB intentions. This indicated that the effect of the system-preservation framing versus the true control condition on PEB intentions was statistically significant both at levels of system justification below 2.87 and above 5.83 on the 1–9 Likert-type scale. Thus, there was some evidence that participants higher in system justification responded differently depending on condition. As shown in Figure 1, the true control condition is a context where PEB intentions appear lowest for high system justifiers, and highest for low system justifiers.

Petitions Signed across Three Conditions

The effect of system justification on signing none, a few, or most petitions did not differ between original and true control conditions (b = –0.09, SE = 0.13, 95% CI [–0.34, 0.16], OR = 0.92, p = .498). However, it differed between true control and system-preservation conditions (b = 0.28, SE = .12, 95% CI [0.04, 0.52], OR = 1.33, p = .021). Simple slope analysis revealed significant negative relationships between system justification and petition signing in the original (b = –0.35, p < .001) and true control conditions (b = –0.26, p = .002), and non-significant association in the system-preservation condition (b = 0.02, p = .786). At 1SD below the mean, lower system justifiers were more likely to indicate willingness to sign most petitions in the true control condition (60.68%) or original control condition (61.36%), compared to the system-preservation condition (48.01%). For higher system justifiers, the highest percentage were willing to sign most petitions in the system-preservation condition (49.90%) compared to the true control condition participants (40.39%) and original study control condition (34.58%; see Table 5 for full results and percentages at M ± 1SD and M ± 1.5SD).

Table 5

Percentages of Those Lower and Higher in System Justification Signing None, a Few, and Most Pro-Environmental Petitions in All Study Conditions.

	LOWER SYSTEM JUSTIFICATION		HIGHER SYSTEM JUSTIFICATION
	M – 1.5SD	M – 1SD	M + 1SD	M + 1.5SD
Original study control condition
None	18.80	23.36	47.79	54.65
A few	13.56	15.28	17.62	16.70
Most	67.65	61.36	34.58	28.65
System-preservation condition
None	34.82	34.39	32.70	32.29
A few	17.65	17.60	17.40	17.34
Most	47.53	48.01	49.90	50.37
True control condition
None	20.34	23.87	41.67	46.74
A few	14.20	15.45	17.94	17.71
Most	65.47	60.68	40.39	35.55

However, our robustness checks that involved negative binomial regression using the count data identified no significant interaction between system justification and true control versus original study control condition (b = –0.05, SE = 0.08, p = .516) or between system justification and true control versus system-preservation condition (b = 0.12, SE = 0.07, p = .114). Thus, there was not robust evidence that system-preservation framing enhanced the likelihood of individuals with higher levels of system justification signing pro-environmental petitions relative to our true control condition (see also Figure 2).

Figure 2

Effects with Conservative Political Orientation

Political orientation did not interact with conditions to predict either outcome variable (see Supplementary Materials for full details).

Potential Explanations for the Replication Failure

The move to online data collection was necessary to secure a large sample, however it offered less experimenter oversight than the original experiment. It also prevented us from offering petitions ostensibly unrelated to the study. The way Feygina et al. (2010, Study 3) covertly measured petition signing was a key strength of their research that we were not able to incorporate into the design of our close replication, which could have affected results. Our participants were likely more aware that petition signing was a dependent variable of interest, potentially leading them to (dis)agree to sign at different rates. Social psychological research is increasingly moving online (Sassenberg & Ditrich 2019). Clifford and Jerit (2014) suggest that participants might respond differently to lab- and online-based studies on a few domains, such as looking up answers in knowledge questions, though attentiveness and socially desirable responding does not appear to differ. Nevertheless, in-person replication attempts of the system-preservation frame are needed to rule out the possibility that the move online affected responses and as a potential explanation for our findings.

It is also important to note that our participants did not give responses consistent with a motivation ‘to defend, bolster, and justify the social, economic, and political systems on which they depend’ (Jost et al. 2017, p. 101). Mean scores on the 1–9 system justification scale were below the theoretical scale midpoint (i.e., M = 3.77, SD = 1.60, scale midpoint = 5). As a result, at ‘high system justification’ (M + 1SD), participants endorsed the items only weakly (i.e., scoring 5.37). Vargas-Salfate et al.’s (2018) 19-nation investigation of system justification found that means are typically low; on a 1–7 scale, the mean in their U.S. sample was 3.41, and levels ranged from 2.43 (in Brazil) to 4.21 (in China: the only nation with mean scores above the theoretical scale midpoint). Feygina et al. (2010) did not report mean system justification in their sample, though given the overall sample size and recruitment from NYU, we could speculate they would have had few participants nearing the top end of the scale. Examining mean system justification tendencies among NYU undergraduates between 2007–2011 reported in Cichocka and Jost (2014) further reinforces concerns that our sample overall scored lower in system justification than Feygina et al.’s sample (i.e., reported means range from 4.41 to 4.53; see our Note 1). While we cannot exclude the possibility that system-preservation framing effectively motivates those strongly endorsing system justification, which could be tested through recruitment of more conservative samples in future, nevertheless the exploratory Johnson-Neyman analyses examined responses across the full range of system justification in our sample. These analyses found significant differences in PEB intentions among those high in system-justification (above 5.83) only when contrasting the system-preservation with the true control conditions, not with the original study control condition; and differences in petitions signing only at extremely high levels of system justification (above 7.48) when contrasting the system-preservation with the original study control conditions.

Another difference between our replication and the original research is the temporal context. The U.S. in August 2022, when our data were collected, was not a stagnant reflection of the U.S. pre-2010, when Feygina et al. (2010, Study 3) were preparing their manuscript. Indeed, the changing context is one potential explanation for why Baldwin and Lammers’ (2016) findings that conservatives respond more pro-environmentally to past-focused than future-focused messages has failed to replicate (Kim et al. 2021; Stanley et al. 2021a). Given the existing polarisation and abundance of environmental information, participants may have set views before entering the experiment (Bernauer & McGrath 2016), and the brief system-preservation message might no longer resonate with American conservatives or be powerful enough to shift responses in the pro-environmental direction. We call for more research on this topic that also considers plausible updates in the research design (see Stroebe & Strack 2014), including whether system justifiers would respond to a different message in today’s context. At the same time, the manipulation interacted with system justification and not with another ideological variable (political orientation), suggesting it was appropriately targeted to system justification.

Sotola and Credé (2022) recently examined experimental research on system justification to estimate threats to replicability, such as low power, publication bias, and questionable research practices. Their analysis indicated that given its low power, main effects within the SJT literature have an estimated replicability rate of just 14%, and that false positives may make up an estimated 60% of reported effects; for interaction effects, the estimated replicability rate is 19%, with a predicted false discovery rate of 25%. Interestingly, the system dependence manipulation (which was included, but not tested, in Feygina et al. 2010, Study 3) had the highest expected replication rate (67%) and lowest estimated false discovery rate (.03). Sotola and Credé conclude that overall, the replicability of SJT research is ‘likely to be low’ (p. 906).

The Effects of System Justification in the Absence of Manipulation

The inclusion of our true control condition revealed that participants who were low in system justification reported higher intentions to engage in PEB when they were presented with no message than if they saw the system-preservation message and/or the system dependence message. Thus, seeing no information was better for maintaining intentions to engage in environmentally protective behaviours than presenting information aimed at changing people’s responses. There are a number of possible explanations for this backfire effect, but as it occurred in both original study conditions, one possibility is that the initial message designed to make their reliance on the system salient caused low system justifiers to respond more similarly to higher system justifiers.

Hennes et al. (2016, Study 3) reported on a system justification experiment that showed this is possible. In a control group exposed to no information, endorsement of system justification predicted poorer recall of climate change information, indicating lower acceptance of the seriousness of climate change. In the experimental group that was told about a U.S. recession, there was no main effect of system justification on recall, but an interaction effect was found showing reduced recall among those low in system justification (defined as M – 1SD), with no effect on those high in system justification (M + 1SD). Thus, when those who typically reject system justification motives were led to believe that the economy was under threat, they engaged in similar information processing as higher system justifiers to lessen the perceived severity of climate change. In other words, the manipulation made higher- and lower-system justifiers respond in the same way (and, interestingly, it did so without changing levels of system justification). Moreover, as further evidence, Kay et al. (2009, Studies 2 and 3) showed that the same system dependence manipulation that was used in Feygina et al. (2010) and in our replication leads to a greater deference to the current system one is motivated to justify.

Together, this literature supports the possibility that the system dependence manipulation engaged the situational system-serving needs of participants who otherwise tend to reject such motives. The result – that is, the flatter line we observe between system justification and pro-environmental behaviour intentions in both original study conditions – may be explained by system dependence affecting higher and lower system justifiers to respond in similar ways. Based on their reporting, it is unclear whether this effect was observed in Feygina et al.’s (2010) data or if it is unique to our replication effort. However, the effect could be specific to the pro-environmental intention outcome variable; for petition signing, both control groups (including the original study control, which viewed the system dependence message) responded similarly. Jost et al. (2012) argued that collective actions like petition signing likely reflect a venting of anger or attempt to persuade authority to change course. Thus, a possible explanation is that the system dependence message demotivated low system justifiers from making personal changes, though they still felt that signing pro-environmental petitions could alleviate their anger about environmental degradation in the original study control condition, unless they additionally read the system-preservation message that framed doing so as entrenching an undesirable status quo.

Motivated to Reject the Status Quo

On average, participants in our sample tended not to endorse system justifying beliefs. We discussed this above, but here further consider that it lends a different interpretation of our results. Jost et al. (2011) stated that when system justification is low, people might engage in efforts to change the current system. Those who strongly reject the status quo may not simply lack a motivation to defend the system but feel compelled to actively seek change. What might be happening to depress low system justifiers’ pro-environmental intentions in the original study conditions in our sample? One explanation is that people who are open to changing and challenging the system (i.e., low in system justification) might have been less willing to do so after a reminder of the unchanging nature of the system and its influence on them. Osborne et al. (2018) previously warned that system-preservation messaging could backfire: ‘framing a progressive social movement as an extension of the status quo could ironically mobilize opposition among liberals, thus offsetting any potential gains in public support obtained by reaching out to conservatives’ (p. 15). When told that environmentalism will protect the system, those who reject the system may reject environmentalism. Interesting future directions are to omit the system dependence passage and test the system-preservation message alone and add a system-challenging message: framing environmental protection as a way to overthrow the status quo. We might expect this message to heighten low-system justifiers’ pro-environmental responses, including collective action.

Until recently, system justification motives have been studied as barriers to collective action. Jost et al. (2017) addressed this oversight by articulating a system-justification model of collective action (SJMCA). In it, they argued that system justification dampens perceptions of injustice and experiences of moral outrage. Anger is a key motivator of system-challenging protest (van Zomeren et al. 2008), including climate activism (Stanley et al. 2021b), so this model has implications for who is likely to engage in what sorts of collective action. They predicted that while activism against the prevailing system is more likely among those who are low in system justification, high system justifiers might take part in protest in defense of the system if it is under threat. One of the dependent variables studied here, petition signing, is a form of non-disruptive protest (Jost et al. 2012). Testing the effect of system-preservation framing within the context of the SJMCA could elucidate whether our unexpected effects are mediated by a reduction of eco-anger among low system justifiers caused by the system dependence manipulation, and thus a low motivation to take part in collective action.

In the absence of any manipulation (i.e., in our true control condition), we replicated the negative effect of system justification on environmental outcomes. Thus, efforts to experimentally manoeuver this association to increase environmentalism are still worthwhile. Jost et al.’s (2012, Study 3) research asked some teachers to detail which elements of the political, legal, social, or economic system they would not recommend other countries model on their nation. This manipulation reduced system justification and increased both anger directed at the government and willingness to protest. While learning of outsiders’ criticisms of the system is a common method to evoke threat that increases system justification motives (Kay & Friesen 2011, but see Sotola & Credé 2022), this finding suggests that generating ones’ own critiques could promote system challenging action. Given increasing public demands for ‘system change, not climate change’, a system-rejection task could be tested as a promising intervention in the climate domain.

Alternative Explanations for Backfire Effects

Researchers use the terms ‘backfire effects’ (and sometimes, ‘boomerang effects’) to describe when climate messages have the opposite effect to that intended, such as increasing political polarisation on climate solutions (Hart & Nisbet 2011). This generally reflects a strengthening of right-wing opposition to climate action, though here we found some evidence of a depression of left-wing intentions to behave pro-environmentally. Swire-Thompson et al. (2020) has previously raised concerns about the replicability of backfire effects in the literature on misinformation corrections. They had several alternative explanations for these effects. For example, they suggested that the effect could be caused by measurement error given the misinformation literature is relatively new and that the majority of work that identified a backfire effect relied on single item measurement. Measurement of PEB is contentious and often uses ad hoc items or scales (Lange & Dewitte 2019), so this critique could apply to our research. Possible advances in measurement of PEB are discussed below.

Another possibility is that participants are responding based on their expectations of the research aims, and that the backfire is therefore caused by demand characteristics. Swire-Thompson and colleagues warn about this possibility because some research has found that ‘virtually all participants’ in a memory updating study correctly identified the study aims. Our two naivety checks identified 67 participants who either recalled taking part in a similar study before or having heard about research indicating that people who prefer the status quo are more persuaded by messages that describe environmental protection as preserving the American way of life. Our results did not change when these individuals were excluded. These and other alternative explanations, such as effects on emotions, ought to be investigated further in future research.

Strengths, Limitations, and More Future Directions

Sotola and Credé (2022) called for researchers to adopt open science practices and for journals to encourage attempts to replicate SJT findings, both conceptually and directly. We contribute to this call by focusing on one of the most impactful SJT publications in our area. The open science practices, including preregistration and publicly available data, are strengths of our replication project. We also recruited a large sample to power examination of the interaction effects, which enabled us to look in an exploratory way across the range of system justification in our data. Inclusion of a true control condition allowed us to generate new insights about the effects of the original study conditions. Finally, including political orientation was an important advance that allowed us to examine the value of system-preservation framing for encouraging conservative environmentalism. However, all research programs have limitations, and the discussion above regarding potential explanations for the null findings identify several limitations of our work that warrant addressing in future replications.

Future research could also further interrogate the role of chronic individual differences in system justifying ideologies in peoples’ responses to climate change. Researchers are increasingly recognising that awareness of the reality of climate change is causing emotional challenges like eco-anxiety (e.g., Clayton 2020). Climate change is a profound global threat and may cause anxiety owing to uncertainty about how and when exactly we will be affected, and whether humankind will do enough to address the problem (Ojala et al. 2021; Pihkala 2020). As a motivation that addresses epistemic and existential needs to reduce both uncertainty and perceptions of threat (Jost et al. 2011), system justification tendencies could protect people from climate distress. Indeed, Jylhä (2016) found that self-reported tendency to avoid experiencing anxiety correlates with both system justification and climate change denial. However, Wullenkord et al. (2021) found no association between climate anxiety and system justification in cross-sectional research. If system justification processes buffer effects of climate threat on anxiety, experimental or longitudinal tests could be more appropriate to detect an association.

Further efforts to adequately measure behaviour are needed to understand the true effects of system justification on meaningful outcomes, and the success of interventions to curb these effects. The PEB intentions measure had some strengths, such as that it successfully avoided ceiling effects, and the modified petitions measure allowed us to deploy the study at scale. However, the intentions measure falls victim to the intention-action gap (i.e., we did not follow up to find out if people actually performed these behaviours), and the petitions measure may not have been believable or maintain the level of ecological validity with the cover story used in the original study. Lange and Dewitte (2019) reviewed alternative measures of behaviour to explore in future research, such as lab-based observation of a target behaviour or recording whether a participant elects to donate a portion of their participation incentive to an environmental cause. Some measures also require a personal sacrifice or effort on behalf of the participant, such as the Work for the Environment task (Lange & Dewitte 2022). Selecting appropriate measures for future research ought to involve considerations around which behaviours are relevant to system justification, such as forgoing high-emission traditions (e.g., abstaining from meat during celebrations where it is typically eaten).

Attitudes are another suitable target for this research to test theoretical predictions about the effects of system justification on rated support for policies and petitions for actions that variously challenge or protect the status quo. Some environmental policies have been criticised for being unjust for certain populations or too mild and ineffective, in part because politicians may aim to keep their supporters satisfied by proposing environmental solutions that maintain the status quo. Researchers could examine whether individuals high in system justification prefer tackling emissions through ‘techno-optimistic’ solutions that do not challenge the status quo (Marquardt & Nasiriyousi 2022), and conversely whether low system justification predicts greater support for policies and protest action that explicitly challenges the status quo through broader reform.