Introduction
Groundwater, a critical common pool resource in many agricultural communities, is under considerable stress due to increased water demand (Frankel 2015). Groundwater management is also scientifically and socially complex, with financial and environmental implications for a wide range of stakeholders, including irrigators, developers, regulators, residents, and environmental groups. Increasingly, difficult decisions about how to allocate scarce water resources are made via community-based, collaborative governance approaches, rather than government or market-oriented approaches (Milman et al. 2018; Lubell et al. 2020). To inform their decisions, these collaborative groups’ resource users must rely on hydrological models, which are inherently imperfect or incomplete. Because groundwater is invisible to most users, measuring groundwater, its recharge, and its long-term availability involves multiple types of uncertainty (Gungle et al. 2016). Case study evidence suggests that scientific uncertainty can affect collaborative performance, but findings in the literature are conflicting, and questions remain about whether and why uncertainty affects performance.
In this study, we ask: how does scientific uncertainty affect collaborative performance? Our study is motivated in large part by the real-world case of Southern Arizona’s San Pedro River. The San Pedro is the last free flowing river in Southern Arizona, and the watershed provides considerable ecological, cultural, and economic benefits (Stromberg and Tellman 2009). For decades, stakeholders have worked collaboratively in the region to manage tradeoffs between the river’s environmental and economic uses. In the 1990s, tensions rose between developers and environmentalists, who held differing views about long-term water availability in the region and whether human activity contributed to declining water levels. The two groups commissioned separate hydrological models that produced conflicting predictions about groundwater availability in the region (Glennon 2002). Since then, a new collaborative partnership has formed to better integrate science and policy (Richter et al. 2014). But it is difficult to measure groundwater levels accurately in large areas like the San Pedro watershed, and studies show that stakeholders continue to perceive and interpret hydrological model uncertainty in diverging ways, potentially affecting the collaborative partnership’s long-term performance (Gungle et al. 2016; Ahn 2023). The case suggests that uncertainty may undermine performance, but individual case studies alone cannot isolate the causal impact of uncertainty on group performance.
To overcome this methodological challenge, we use a game experimental approach, which is commonly used to isolate the effects of individual variables on sustainable resource use in settings where users share a common resource (Ostrom 2006). Previous experiments have identified key factors that affect users’ ability to manage shared resources sustainably, including communication, resource visibility (Janssen 2013), power asymmetry (Javaid and Falk 2015), and informational uncertainty (Apesteguia 2006). Here, we modify a groundwater game initially developed by Meinzen-Dick et al. (2016), where users must choose between crops of varying water intensity under conditions of limited and shared groundwater resources. Our version is modified to take place online via Zoom, and tests hypotheses about whether different types of scientific uncertainty affect different measures of collaborative performance. To gain a qualitative understanding of the interactive strategies that groups develop to cope with uncertainty, we also analyzed groups’ communications via Zoom chat during the game.
Our results show that some forms of scientific uncertainty can prompt participants to use less water, contributing to the protection of shared resources. More specifically, uncertainty in the form of a range of estimates tends to promote sustainable resource use, while the use of competing hydrologic models has limited influence. Our qualitative analysis of participants’ communications suggests that groups respond to uncertainty in strategic ways, but that their approaches diverge. Some groups committed to collaborative strategies that emphasized caution to ensure sustainable resource use, while others were reluctant to commit to reducing their resource use in the face of uncertainty. Overall, we find that some forms of scientific uncertainty may prompt resource users to exercise caution and use resources more sustainably, but that this result depends upon stakeholders’ strategic actions in response to scientific information.
Our results also have practical relevance for many groundwater basins that are in the process of establishing collaborative, community-based approaches to groundwater management, even in the absence of hydrological certainty. For instance, recent groundwater reform in California has prompted every irrigation district in the state to form collaborative groups to negotiate plans for long-term, sustainable groundwater use (Kiparsky et al., 2017). These collaborative groups may well find that, as in the San Pedro, hydrological uncertainty poses challenges that could threaten their collaborative performance. Our results suggest that on the one hand, imprecise hydrological estimates may prompt cautious conservation behavior and thus protect shared groundwater resources. On the other hand, if hydrological models contradict one another and suggest widely different long-term water availability, this may allow some users to behave strategically in ways that undermine the collaboratives’ ability to use groundwater sustainably and equitably.
Theory and hypotheses
In this section, we bring together literature on collective action (e.g., Ostrom 1990) and collaborative governance (e.g., Emerson & Nabatchi 2012) to develop hypotheses about how scientific uncertainty affects the performance of collaborative groups. While these two literatures both deal with common themes about how groups of interdependent actors work together to achieve (or not achieve) shared goals, they have different epistemological origins, assumptions, and methodologies. In the literature review below, we draw on and synthesize studies from both literatures.
Collaborative Performance as a Dependent Variable
Collaborative performance is a multi-dimensional construct that can include both process-based performance and substantive impacts on social or ecological conditions (Koontz et al 2019). In the common-pool resource literature, performance often refers to the sustainable use of small-scale natural resources such as fisheries, groundwater, or forests (e.g., Meinzen-Dick et al. 2016; Janssen et al. 2010). In the collaborative governance literature, scholars measure performance in multiple ways, including ecological and environmental outcomes (Scott 2015; Baldwin 2020), governance outputs (Koontz et al. 2019), and equity outcomes (Naime et al. 2022; Cook, Grillos, Andersson 2023).
Performance can also be measured at the level of either the group or the individual (Emerson and Nabatchi 2015). In behavioral experiments, individual economic gains –payoffs – are often used to measure performance at the individual level. In the collaborative governance literature, individual level performance might be conceived as participant satisfaction with the process, or individual willingness to cooperate with others (Koontz et al. 2019).
Scientific Uncertainty and Collaborative Performance
Collaborative governance is emerging as a dominant model for groundwater governance, and recent policy change in California now requires collaborative groundwater sustainability planning (Sabatier et al. 2005; Lubell et al. 2020). But it is not entirely clear that the collaborative governance model will work well under real-world conditions of groundwater basins, where groundwater recharge is invisible to most users, many users are newcomers to the basin, and incomplete information about groundwater recharge rates may undermine the effectiveness of collaborative processes. In Arizona, where groundwater management has been required for decades, the San Pedro case offers a cautionary tale. The basin created a collaborative group that was instrumental in getting diverse stakeholders to develop shared vision and values for a groundwater basin. But stakeholders held divergent views about how human activities impacted water availability and recharge rates, and two different groups of stakeholders each commissioned two different hydrological models, with divergent long-term implications for sustainable water use in the basin (Glennon 2002; Ahn 2023). Uncertainty about which model to use prevented the group from developing a shared scientific understanding of recharge rates, and prompted a split into two groups (Ahn 2023). Disputes or divergent views about basic ecological conditions within groundwater basins can lead to conflicts that complicate or even undermine actors’ ability to work together effectively (Ahn 2023).
Scientific uncertainty and the use of uncertain information have been important topics among environmental policy scholars for decades, coinciding with the rise of environmental disruptions and climate change, both of which are subject to considerable uncertainties (Abbott 2005; Ulibarri 2019; Koontz and Thomas 2021). Drawing on this work, we define scientific uncertainty broadly as a lack of commonly accepted knowledge about the causes and consequences of an environmental problem within a collaborative group. In managing groundwater systems, natural and artificial water recharge has been a major point of disagreement among key stakeholders (Laukka et al. 2021; Gungle et al. 2016). Socio-hydrological studies show that uncertainty about water recharge affects political decision making (Siade, Nishikawa, and Martin 2015), and observational studies suggest that under scientific uncertainty, stakeholders tend to act strategically to benefit their own interests by exploiting scientific uncertainty (Knaggård 2014).
Competing hypotheses have been suggested about the relationship between scientific uncertainty and ecological sustainability. Uncertainty can lead to information asymmetry, increased transaction costs, and free riding, and these conditions can be exploited by actors with a strategic interest in delaying action or manipulating outcomes. Some studies have found that uncertainty negatively affects shared resources (Dannenberg et al. 2015; Barrett and Dannenberg 2012; Palm-Forster and Suter 2022). Other studies, however, have challenged this argument and provided evidence that threshold uncertainty actually helps protect shared resources (Santos and Pacheco 2011; Schill and Rocha 2023).
Less is known about whether scientific uncertainty might affect equitable distribution of outcomes in collaborative governance. A growing number of scholars have raised concerns about equity in collaborative governance, particularly collaborative groundwater governance. Some studies suggest that collaborative engagement tends to mitigate distributional equity (Ahn and Baldwin 2022), and laboratory studies also show that communication tends to improve distributional equity in payoffs among participants (Ghate, Ghate, and Ostrom 2013), but other studies find that collaboration can reproduce inequities for marginalized groups (Méndez-Barrientos et al. 2020). Uncertainty may also allow some users to exploit collaborative processes for their own gain at the expense of others. Some evidence suggests that power asymmetry between resource users tends to yield less equitable distribution of earnings (Janssen et al. 2012).
Studies operationalize uncertainty in different ways. In experiments replicating international negotiations, uncertainty has been operationalized in terms of unknown but potentially dangerous thresholds or tipping points (Barrett and Dannenberg 2012). In their study, Barrett and Dannenberg (2012) find that the fear of crossing a dangerous threshold can turn climate negotiation into a coordination game, but uncertainty about location of the threshold turns the game back into a prisoners’ dilemma, causing cooperation to collapse. Palm-Forster and Suter (2022) operationalize three types of threshold treatments – a known threshold (certainty), an uncertain threshold with a known probability distribution of possible thresholds (risk), and an uncertain threshold with an unknown probability distribution (ambiguity). They find that threshold uncertainty – risk and ambiguity – tends to increase use of shared natural resources, but that this risk is mitigated by communication.
Uncertainty can also focus on imprecision or ambiguity in hydrologic studies. Scientists may be more or less confident about their models depending on a range of factors, including the quality of data, the target precision, and the model fit. Effective and impactful science explicitly conveys the uncertainty that is generated from simplified models. Often, scientific models include a range of values where the best estimate of the model is presented along with the margin of error. In the context of groundwater management, USGS studies report a range of estimated water recharge from different sources and the resultant uncertainty varies significantly (Siade, Nishikawa, Martin 2015; Gungle et al. 2016).
Scientific uncertainty can also be framed as competing models of explanation or prediction in media and policy community, especially when underlying scientific issue is contentious. In the case of the San Pedro, for example, two different models pointed to different long-term predictions about water availability, prompting stakeholders to make strategic choices about which model should be used as the basis for collaboration (Ahn 2023). Competing hydrological models can themselves be the result and process of on-going tension between stakeholders, where stakeholders choose and work with hydrologists operating under different baseline assumptions. Competing models may further undermine the credibility of neutral science and reduce the social trust between stakeholders. While competing science has been observed in the real world, to the authors’ knowledge, this form of uncertainty has not been explored in experimental settings.
Hypotheses
We draw on the literature review above to hypothesize that the nature of uncertainty may determine whether uncertainty improves or undermines sustainability of resource use. More specifically, we hypothesize that imprecise estimates and competing hydrological models prompt very different types of strategic behaviors. When models are imprecise, resource users may be more likely to use water conservatively, ensuring that water use remains well within sustainable levels. Recent study suggests that scientific uncertainty in the form of numerical range does not undermine public’s trust in science (Van Der Bles et al. 2020). Competing models, in contrast, allow users to justify higher levels of consumption because perceived conflict among experts tends to undermine stakeholders’ perception that the science is credible (Shi et al. 2023; Jensen and Hurley 2012). Stakeholders may resist reducing their resource consumption if they lack confidence in scientific models that suggest such reductions are critical for sustained water availability.
We thus hypothesize the following:
H1: Scientific uncertainty in the form of imprecise estimates improves sustainable resource use, but uncertainty in the form of competing models undermines it.
Our second hypothesis considers the effects of uncertainty on equitable distribution of benefits from resource use. The San Pedro case suggests that competing estimates created in silos by separate experts tend to increase tensions among stakeholders and prevent transparency that is needed for stakeholders (Glennon 2002; Stromberg and Tellman 2009; Ahn 2023). These tensions undermine coordination and may make equitable distribution of resources more difficult to achieve. But when stakeholders work together in hydrological modelling and produce single scientific studies bounded by a degree of uncertainty, these are more well received (Ahn 2023). Thus, we hypothesize that imprecise estimates may prompt actors to work together for the benefit of the group, while competing estimates of available resources may allow some actors to behave strategically and capture greater benefits for themselves. More specifically:
H2: Scientific uncertainty in the form of imprecise estimates improves equitable distribution of benefits, but uncertainty in the form of competing models undermines it.
Next, we turn to the effects of collaborative performance on individual level outcomes. Starting with cooperative behavior, we follow a similar logic to Hypothesis 2, hypothesizing that imprecise estimates will encourage players to work together and increase their willingness to forego rewards to benefit the group, but competing models will have the opposite effect:
H3: Scientific uncertainty in the form of imprecise estimates improves individuals’ willingness to cooperate, but uncertainty in the form of competing models undermines it.
Finally, we draw on the literature to hypothesize that uncertainty in general will undermine individuals’ ability to maximize earnings from resource use:
H4: All forms of scientific uncertainty will reduce average individual resource earnings when compared with groups given scientific information with less uncertainty about recharge.
Experimental Design, Methods, and Data
Our game experiment is based on Meinzen-Dick et al. (2016)’s field experimental crop choice game. Over repeated rounds of game play, participants make decisions about whether to plant rice, a high-water intensity crop that offers higher economic returns, or corn, a low-water intensity crop with smaller returns (Cooley et al. 2015). Water is shared among all players and decreases as players choose their crops, with limited recharge after each round. While actual recharge rates were constant across all rounds and treatment conditions, we tested our hypotheses about scientific uncertainty by varying the information that participants were given about water recharge rates.
Our initial planning and experiment pilot-testing took place in 2021, when Covid-related restrictions made normal laboratory experiments infeasible. Building on previous experimental work (Janssen et al. 2010; Baggio et al. 2015), we chose to use Zoom as the primary platform for our experiment. To make the most out of the online environment, we used Python graphics to communicate the basic game set-up to players, the Qualtrics survey to record players’ crop choices, and the Zoom chat function to record players’ communications with one another. Unlike previous experimental games, which isolated and varied communication rules, we allowed players to communicate during all rounds via Zoom chat, which we later analyzed to qualitatively examine how participants cope with uncertainty. To mimic real-world conditions where individuals have an incentive to maximize their earnings, we provided participants with financial compensation based on their crop choices (Meinzen-Dick et al. 2018). Each game dollar was worth 50 cents in real-world U.S. dollars at the end of the game, allowing participants to earn up to $25. The monetary incentive provided in the lab makes a difference in environmental behavior in a way that the incentive scheme should feel more real to participants compared to hypothetical incentive (Xu et al. 2018).
Recruitment and Game Play
Participants were recruited from the University of Arizona student pool to play 30 games from July to November 2022. An email went out to students across campus, and some instructors offered students extra credit for participating. Students could schedule their game individually and virtually using the Calendly system, which mitigated the concern of sampling bias, e.g., when students from a single class or friend group sign up for the same game. Due to scheduling constraints, treatments could not be assigned entirely at random. To rule out selection bias, we conducted a balance test that shows similar demographics across control and treatment groups [See Supplementary Materials, Appendix B].
Groups of 4–5 participants played up to 10 rounds of the game. They first played two practice rounds, during which they were allowed to ask any questions to ensure comprehension of instruction. To ensure anonymity, when participants entered the Zoom room, they were instructed to turn off their camera and microphone and to change their name into a random animal name. Players were then given basic game instructions. For consistency across games, a facilitator used a pre-recorded script to provide instructions and did not interrupt the game unless there were technical difficulties.
Participants were instructed that the goal of the game was to maximize their earnings by choosing a crop to plant in each round. Players could choose to plant corn, which consumes one unit of water and produces two dollars of game income, or rice, which consumes three units of water and produces five dollars of game income. Earnings were individual, and players were told that the facilitator would monitor the chat traffic and stop the game if players attempted to share earnings [See Figure 1 in Supplementary Materials].
Players were also told that their water resources were shared and limited. Players were given accurate information about their initial shared groundwater resource (10 units per player, for a total of 40 or 50 units, depending on group size). In all games, 4 or 5 units of groundwater were recharged after each round depending on group size. In the control group, players were given accurate information about groundwater recharge. In the first treatment group, players were told that recharge would fall between 0 and 10 units. In the second treatment group, players were told that two different hydrologists produced conflicting estimates, with one predicting 1 unit of recharge and the other predicting 10 units of recharge (See Table 1).
Table 1
Framed Treatment Information.
| Framed scientific uncertainty about water recharge | |
| Control condition | “Based on the calculations of our hydrologists, 5 units of water will be replenished after each round of the game.” |
| Treatment 1: A range of estimates | “Based on the calculations of our hydrologists, 0 – 10 units of water will be replenished after each round of the game.” |
| Treatment 2: Competing hydrological models | “Our hydrologists disagree about the groundwater recharge. Based on the calculations of the hydrologist 1, there would be 1 unit of recharge per round for the group. In contrast, the hydrologist 2 suggests that there would be 10 units of water recharge per round for the group.” |
In each round, players were asked to make their crop choices in secret via a Qualtrics decision form, shown in Figure 2 in the Supplementary Materials. After making their decisions, players received an update on their cumulative earnings and contribution to groundwater depletion, shown in Figure 2 (Supplementary Materials). After making their crop choice, players were instructed to return to the zoom room, where all players received an update on the group’s current season and available groundwater. Between rounds, players were allowed to strategize with one another via Zoom chat for about 60 seconds [see Figure 3, Supplementary Materials]. Games could include up to 10 rounds, but players were not told the number of rounds to avoid strategic behavior in the final rounds. After the game, participants completed survey questions about their demographic information and perceived outcome satisfaction and received payment via Venmo or Zelle. [See Supplementary Materials, Appendix A for the survey questions].
Operationalizing the Dependent and Independent Variables
Because our dependent variable of interest, collaborative performance, is multi-dimensional, we use multiple measures at both the individual and group levels. At the group level, we calculate a proxy for sustainable water use by calculating the percentage of groundwater remaining after the end round of the game, and we measure equity in two ways: with a measure of the standard deviation of individual earnings across all group members, and by calculating the Gini coefficient for within-group earnings (Anderies et al. 2013). We measure performance at the individual level in terms of individual profit and willingness to cooperate. The former is measured by calculating total payoff across all rounds of play, and the latter is measured by the percentage of rounds where an individual chose the lower-value, less-water intensive corn crop. (See Table 2.)
Table 2
Definition and operationalization of dependent and independent variables.
| VARIABLES | DEFINITION | OPERATIONALIZATION |
|---|---|---|
| Dependent variables | ||
| Group-level Dependent Variables | ||
| Sustainability | Shared groundwater availability | % of groundwater remaining after the end round of the game |
| Equity | Distributional equality of payoffs in each group | (1) Standard deviation of earning distributions within each group (2) Gini-coefficient of payoffs in each group |
| Individual-level Dependent variables | ||
| Cooperation | Players’ willingness to forego profit for the common good | % of corn choices in 10 rounds |
| Profit | Participant’s total rewards | Individual total earnings in 10 rounds |
| Primary independent variables | ||
| Scientific Uncertainty | A range of estimates | Water recharge units between seasons are 0~10. |
| Competing hydrological models | One hydrologist argues that the recharge unit would be 1 and the other hydrologist argues that it would be 10. | |
Our primary independent variables are two types of scientific uncertainty about water recharge rates provided to participants. In the first treatment condition, players are given a range of estimates (e.g., they are told that recharge is estimated at 0–10 units per round). In the second treatment condition, players are given competing estimates (e.g., they are told that hydrologists disagree about whether recharge is 1 or 10 units per round). Since the actual recharge rounds were consistent across all rounds, in practice participants might have updated their understanding of recharge based on their own observations, potentially magnifying the conflict between the two competing models.
Data and Analysis
This study included 130 individuals in 30 groups. Because our dependent variable is measured at the group and the individual level, we use different analytical strategies. To test the impact of uncertainty on group-level performance variables, we use one-way ANOVA to compare means of each treatment:
In equation (1), Yij represents performance variables for round i and group j. µ is the overall mean of all observations. αj * Ui represents the interaction between the treatment indicator variable Ui and the effect of the treatment for each group αj. ɛij is the standard error term in individual rounds within each group. While we ran a series of ANOVA tests to compare means of treatment groups, we use the Bonferroni correction for our main results in Table 4 since it allows detailed comparisons between multiple conditions and corrects for the increased probability of making Type I error – false positive – by adjusting the significance level (StataCorp 2023; Cabin and Mitchell 2000).
To test individual level outcomes, we use linear regression estimation. To account for relatively small sample size and potentially skewed error distribution in game data, we use clustered standard error to mitigate concerns of heteroskedasticity. We also run bootstrapped clustered standard error with sufficient number of computational resampling for robustness checks (Cameron, Gelbach, and Miller 2008). The results were consistent. The model is listed below:
In equation (2), Yij measures the performance of individual i within group j. Uij is a vector of dummy variables representing the treatment conditions given to individual i within group j condition. Di is a vector of individual level demographic variables, and ɛij is the error term.
Our models include individual covariates that might affect decision-making, drawn from a post-game exit survey. These covariates include education level, gender, and major. To address the possibility that students majoring in natural resource management might bring conservation-related knowledge and attitudes to the game, we created a dummy variable to control for these majors. Descriptive statistics are presented in Table 3.
Table 3
Descriptive Statistics.
| VARIABLE | OBSERVATIONS | MINIMUM | MEAN | MAXIMUM | STANDARD DEVIATION |
|---|---|---|---|---|---|
| Dependent variables | |||||
| Group-level | |||||
| Sustainability: Remaining groundwater % | 30 | .05 | 0.44 | 0.8 | 0.19 |
| Equity 1:Gini-earning | 30 | 0 | 0.06 | 0.14 | 0.04 |
| Equity 2:Earning SD | 30 | 0 | 4.15 | 9.32 | 2.41 |
| Individual-level | |||||
| Cooperation | 130 | 0.2 | 0.70 | 1 | 0.17 |
| Profit | 130 | 20 | 28.74 | 44 | 5.24 |
| Individual Covariates | |||||
| Graduate student | 130 | 0 | 0.10 | 1 | 0.31 |
| Natural Resource Major | 130 | 0 | 0.18 | 1 | 0.39 |
| Gender | 130 | 0 | 0.72 | 1 | 0.44 |
Table 4
The Difference in Collaborative Performance between Treatment and Experimental Conditions (group level). These estimates are based on equation (1).
| # OF GROUPS | SUSTAINABILITY: REMAINING GROUNDWATER % | EQUITY (SD) | EQUITY (GINI) | |
|---|---|---|---|---|
| Control (“X unit of recharge”) | n = 9 | M = 0.32 (0.15) | M = 4.21 (2.00) | M = 0.06 (0.03) |
| Treatment 1 (“A range of estimates”) | n = 11 | M = 0.49 (0.17)*** | M = 3.69 (2.57) | M = 0.06 (0.04) |
| Treatment 2 (“Competing hydrologic models”) | n = 10 | M = 0.48 (0.21)** | M = 4.64 (2.52) | M = 0.07 (0.04) |
[i] ***p < .01, **p < .05, *p < .1, standard deviations are in parentheses, M: mean, SD: standard deviation.
Qualitative Analysis of Group Strategies
We also collected text data from Zoom chats to examine how groups cope with scientific uncertainty. During each game, groups were allowed to use Zoom’s chat function between each round of game play. Players were not able to monitor which player invested in which crops, but they could choose to discuss their choices via Zoom chat, and could choose whether or not to reveal their choices from past rounds, or to discuss and commit to a particular choice in future rounds. After reading all chat transcripts, we selected chat interactions in which players discussed strategy for additional analysis. Using an inductive process, we identified three main strategic interactions that groups used: a “Collaborative” strategy, where players engaged with one another collaboratively; a “Collaborative Caution” strategy, where players engaged with one another collaboratively but expressed a need to exercise caution as a means of coping with uncertainty about water use; and an “Uncertain Commitment” strategy, where players were reluctant to commit to a particular course of action. Table 6 presents example chat text for each of these strategies.
Results
Group-level outcomes
Table 4 presents the ANOVA results to test the effects of uncertainty on group-level outcomes. Results suggest that both forms of uncertainty had a significant and positive effect on remaining groundwater. While the control groups had, on average, 32% of their original groundwater resources remaining after each round, treatment groups 1 and 2 had 49% and 48%, respectively, and this result was statistically significant. To unpack this result temporally across rounds, Figure 1 shows the remaining water after each round for all 3 groups. Figure 1 suggests that the groups made similar crop choices for the first few rounds, but began to diverge around Round 4, when groups subjected to scientific uncertainty began to use less water compared to the control group.
Figure 1
Remaining Groundwater (percentages).
Table 4 also shows the effect of scientific uncertainty on equitable distribution of earnings. Here, the results are conflicting and statistically insignificant. When equity is measured in terms of the standard deviation of earnings, uncertainty in the form of imprecise estimates tends to improve equity and uncertainty in the form of conflicting estimates tends to undermine it. When equity is measured as a Gini coefficient, competing hydrological models show a slightly worse equity, but none of these results is statistically significant.
Individual outcomes
Table 5 presents the results of regression models used to test the effects of uncertainty on individual-level outcomes. Model 1 presents the results for individual cooperation as a dependent variable (e.g., foregoing the more lucrative but more water intensive crop). In groups where players were given a range of estimates, there was an average increase in cooperative behavior of 0.08 when compared to the control group, a result that is statistically significant (p < .05). In groups that received competing model treatment, cooperative behavior was not statistically different from the control group. Model 2 presents the results for individual earnings and shows that players in groups given a range of estimates earn 2.4 less game dollars than control groups (p < .05). Earnings for participants in the competing estimate groups were not significantly different than the control groups.
Table 5
The Effects of Scientific Uncertainty on Collaborative Performance (individual-level). Model (1) and (2) are based on equation (2).
| (1) COOPERATION | (2) PROFIT | ||
|---|---|---|---|
| Primary Independent Variable | Treatment 1 (“A range of estimates”) | 0.0811** (0.0351) | –2.432** (1.052) |
| Treatment 2 (“competing hydrologic models”) | 0.0333 (0.0470) | –0.998 (1.411) | |
| Covariates | Graduate Student | 0.0632 (0.0516) | –1.896 (1.556) |
| Natural Resource Major | 0.0963** (0.0369) | –2.888** (1.107) | |
| Gender (Female) | 0.0032 (0.0273) | –0.095 (0.794) | |
| Constant | 0.640*** (0.0356) | 30.79*** (1.067) | |
| Observations | 130 | 130 |
[i] ***p < .01, **p < .05, *p < .1, standard deviations are in parentheses.
Table 6
Theoretical mechanisms that explain quantitative results and supporting qualitative empirical evidence from communication analysis.
| COLLABORATIVE STRATEGY | COLLABORATIVE CAUTION STRATEGY | UNCERTAIN COMMITMENT STRATEGY |
|---|---|---|
| Example chat text from a group under competing hydrological models [g20] R1. Hello everyone! Should we all alternate which resources we pick, so we can play more rounds? I say we all go with less groundwater so we can hopefully get more rounds everyone has to be down though otherwise it wont work Perfect. Should we alternate? 2 pick rice and 2 corn? Sounds good Sure ill go corn okay Same I’ll do rice R2. Capybara and I picked corn. This round, should the two of us pick rice and Turtle and Cat pick corn? Yes okay sweet, now we rotate Ya i’ll do corn now Sounds good. Aye love this teamwork lol Same here. We want to avoid the tragedy of the commons yeaaaa | Example chat text from a group under the range of estimates treatment [g8] R2. Did that even deplete any? No it did not It looks like we earned back the same that we spent maybe if we choose the same card it doesn’t deplete any? It said in the instructions we will gain back anywhere from 0–10 each round I chose rice that time and it went down four R3. Not too bad so far I say we just stay conservative, we will be making more money in the long run compared to how much water we use. Should we maybe try and go low again? I agree Awesome I agree, maybe a rice every couple rounds just to see but not too much R4. Did anyone pick rice that time? No I didn’t I didn’t either, that’s interesting that it went down two Should we go low again? Yes Lets go low again maybe until we gain some more water? Yes Honestly, I say stay low for awhile Works for me | Example chat text from a group under certain recharge information[g19] R1. We could all choose rice this round to try to maximize our water Bet R2. Done we should do corn to maximize the water ok sounds good R5. I think we should all keep doing corn to hold out as long as possible i agree R7. We are about to do season H right R8. Should we keep doing corn we only have 6 points until the game is ended pls keep doing corn it should stay at the same # |
Among covariates, students majoring in natural resource management earn approximately 3 game dollars less than others (p < .05). They also tend to be more cooperative in water conservative crop choices. Gender and graduate student status does not show any systematic effect on crop choice behavior.
Qualitative analysis
To unpack potential mechanisms behind our results, we also analyzed Zoom chat transcripts, using an inductive process to identify 3 preliminary strategies that groups used to inform their crop choices. In the “collaborative strategy” approach, participants recognized the need to balance individual and group benefits, and over the course of several rounds decided on a strategy that group members would share. The specific strategy varied: in some groups, players used a rotation strategy to allow different players to choose the more profitable rice crop each round, while other groups chose universal or optimization strategies. In the “collaborative caution” approach, participants coped with uncertainty by delaying immediate consumption of resources. They tended to observe how crop choices, water recharge, and resulting water quantity are related for a couple of rounds. Finally, the “uncertain commitment” strategy was an alternative approach that some groups used to cope with uncertainty. Here, instead of deciding as a group to exercise caution, individual players were reluctant to share information about their choice, commit to a discussed strategy, or follow through on their commitments.
Based on qualitative evidence, groups under scientific uncertainty tend to deploy a variety of strategies. Table 6 provides an example chat discussion illustrating each of the three strategies. In collaborative strategy, an example group under competing hydrological models demonstrate that they are transparent about individual crop choices and adopt rotation strategies to cope with uncertain recharge situations. The collaborative caution strategy, where an example group is under the range of estimates treatment, shows that groups stay conservative on their water use and carefully examine hydrological conditions before making further adjustments. A group under certain recharge information demonstrates uncertain commitment strategy where a few individuals attempt to take water saving initiatives, but players are hesitant to share information about their choices and to make meaningful progress in devising rules and committing to those rules. Given the relatively small number of observations, we avoid drawing conclusions about the correlation between treatment information and kinds of strategies adopted by different groups, and simply note that groups generally respond to uncertainty in strategic ways, but that specific strategies diverged across groups.
Discussion and Conclusions
Experiments are often used to better understand how groups of resource users engage in collective action (Janssen et al. 2010; Baggio et al. 2015; Meinzen-Dick et al. 2016; Barrett and Danberg 2012; Schill and Rocha 2023). Early experimental work demonstrated that communication within groups is critical to sustainable resource use (Janssen et al. 2010; Barret and Danberg 2012; Meinzen-Dick et al. 2016). In our study, we build on and advance this work. We develop a digital version of Meinzen-Dick et al. (2016)’s groundwater game, test novel hypotheses about the effects of multiple types of uncertainty, and operationalize multiple dimensions of collaborative performance. Our first hypothesis posited that imprecise scientific information would improve long-term water availability, while competing hydrological estimates would undermine it. Our results partially confirm this hypothesis. Both of the uncertainty treatments prompted players to choose less water-intensive crops. The differences between the treatment and control groups did not emerge until around the third round of play, suggesting that there is a learning curve across all groups, but that as players begin to better understand how recharge works in the game setting, uncertainty around the exact recharge amount prompted players to be more conservative with their water use. Our findings are consistent with a handful of other studies that suggest a positive relationship between uncertainty and sustainable resource use (Santos and Pacheco 2011; Schill and Rocha 2023).
Analysis of chat transcripts confirms this result, suggesting that while control group users tended to pursue an optimization strategy, uncertainty prompted other groups to develop strategies that would protect their interests under a range of conditions. In other words, when users have full and perfect information about water resources, they will tend to optimize their water use, but when information is uncertain, they must respond strategically. Sometimes, groups exercise caution, behaving in ways that are sub-optimal but that strategically avoid the risk of catastrophic loss, resulting in more sustainable resource use overall. In other cases, however, uncertainty may prompt users to avoid committing to a common strategy.
Our results provide no support for our second hypothesis, which posited that imprecise scientific information would improve equitability of earnings distribution, while competing hydrological models would undermine it. Distributional equity was slightly lower under both uncertainty treatments, but these results were not statistically significant. The null effects of uncertainty on equity could also stem from the way equity is operationalized in this study by using the Gini-coefficient, which makes an underlying assumption that equitability is synonymous with equal earnings across participants (Naime et al. 2022). Since our participants were students who entered the game with equal information, social capital, and strategic advantage, this is a reasonable assumption. In the real world, however, participants’ capacity, relative power, and history of marginalization can differ considerably, and equal distribution of benefits may not be considered equitable. Field experimental studies sometimes operationalize equity outcomes in more complex ways that reflect community conditions, including equity perceptions (Cook, Grillos, Andersson 2023) and participation differences across marginalized groups (Cook 2024). While our lab experiment does not require these nuanced measures, we encourage future studies to include diverse ways of equity measurement and consider potential effects of other institutional variables that importantly shape power imbalance and thus equity.
Turning now to our hypotheses about individual-level outcomes, we find mixed support. Our third hypothesis posited that imprecise scientific information would improve individuals’ willingness to cooperate, while competing hydrological estimates would undermine it. Our results show that individuals receiving the imprecise range of estimates treatment were significantly more likely to cooperate by foregoing the more lucrative rice crop, but show that competing estimates had no effect on individuals’ cooperative behavior. In terms of individual earnings, we hypothesized that both forms of uncertainty would reduce individual earnings, again with partial support: individual earnings were lower in both treatment groups, but these results were statistically significant only for the groups receiving a range of estimates.
With any lab experimental work, it is critical not only to assess the findings themselves, but also to discuss whether those findings are generalizable to real-world settings. Given the many differences between our student research subjects and real-world farmers and other water users, it may be naïve to assume that our most basic finding – e.g., that scientific uncertainty leads to more sustainable water use – will hold in all settings. Instead, we argue that our findings reveal an underlying causal mechanism that is far more generalizable: that scientific uncertainty prompts resource users to engage in strategic behaviors to avoid potential harms. In our lab experimental setting, these strategic behaviors were cautious and focused on conserving water. In the real world, where resource users have a wider range of options available to them, they might choose different strategic behaviors.
This finding holds significant policy implications for watershed communities, highlighting how the presentation of scientific uncertainty to stakeholders can influence their behaviors. Given inherent uncertainty in hydrological models, scientists and stakeholders may do well to work together to co-produce knowledge. Recent studies on coproduction (Wall, Meadow, Horganic 2017) suggest that when conflict between stakeholders is high regarding socially and scientifically complex issues, stakeholder groups can improve environmental outcomes of shared resource governance by developing an integrated model that provides an agreeable range of uncertainties rather than different hydrological estimates. In that process of coproduction, stakeholders can have additional benefits such as increased trust, shared strategies, and increased coordination.
Theoretically, we bring insights from the collaborative governance literature to the study of common pool resources, diversifying the conceptualization of group-level performance. We also draw from a real-world collaborative governance case – San Pedro – to derive novel hypotheses (Ahn 2023). Methodologically, integrating case study insights and game experiment is a promising way to produce findings that are rigorous via experimental design and relevant, bringing in hypotheses from real world observations (Paluck 2010; Ostrom 2006).
In an era of global change, scientific uncertainty about existing and future environmental conditions is prevalent, and many scholars have begun to examine how this uncertainty affects groups’ ability to cooperate around natural resource use. These studies have produced conflicting findings. Some experimental and theoretical work suggests that uncertainty undermines sustainable resource use (Dannenberg et al. 2015; Barrett and Dannenberg 2012; Palm-Forster, and Suter 2022), while other studies come to the opposite conclusion (Santos and Pacheco 2011; Schill and Rocha 2023). In the related field of climate change research, there is general consensus that powerful stakeholders within the fossil fuel industry have exploited scientific uncertainty to cast doubt on the need to take action on climate change (e.g., Oreskes & Conway 2011). While our study does not resolve these tensions, our qualitative results suggest that groups may respond to uncertainty in divergent ways, based on heterogenous stakeholder interests. In cases of water governance, where most stakeholders share a long-term interest in sustained access to water, stakeholders may behave strategically in ways that exercise caution in the face of uncertainty. In the climate change case, where powerful actors have a short-term financial interest in avoiding policy change, these stakeholders may use uncertainty to avoid or delay policy action.
Additional Files
The additional files for this article can be found as follows:
Acknowledgements
We are grateful to special issue editors and IJC reviewers for their constructive comments, which have helped to improve the manuscript. We are also grateful to Kirk Emerson, Tom Koontz, Tomás Olivier, Marco Janssen, and seminar participants at ASU’s Center for Behavior, Institutions, and the Environment for their feedback. We received valuable feedback at the 2022 Duck Family Graduate Workshop, 2023 International Association for the Study of the Commons Conference, and 2023 American Political Science Association Annual Conference. Support for article publication and editing of this special issue on groundwater governance was provided by the CGIAR Research Initiative on NEXUS Gains: https://www.cgiar.org/initiative/nexus-gains/. We also thank the University of Arizona’s Undergraduate Research Initiation Funding for their support in designing and implementing groundwater games with undergraduate students.
Funding Information
This research received funding support from University of Arizona’s Undergraduate Research Initiation Funding, GPSC research grant, and Dr. Kirk Emerson.
Competing Interests
The authors have no competing interests to declare.