Introduction
Becoming a parent is a significant shift in the lives of parents, and the need for both health and social support are well documented (Oakley 2018). For many, there is a sense of uncertainty and anxiety, particularly in new mothers (who in most cases, still do the bulk of childcare and night wakings), in the shift to having responsibility for someone other than themselves (Leahy Warren 2005). Particularly in times of austerity with budget cuts to health and social services, including in the post-natal period, information about parenting may come predominantly from friends and family (Leahy Warren 2005), along with sources such as the internet and media (Lagan, Sinclair, and George Kernohan 2010, Hay et al. 2022, Johansson et al. 2010, Krimsky 2007). Such sources are often not evidence-based, i.e. have not been proven, beyond anecdote or chance, to work (Asmussen 2012). Parenting Science Gang was established by a group of mothers to conduct research that answered the questions new parents have about parenting. Whilst parent-led research endeavours already existed (particularly in the rare diseases space; see for example Parent Project Muscular Dystrophy https://www.parentprojectmd.org/), Parenting Science Gang was novel because it sought to give free-reign to parents to research anything parenting related. Parenting Science Gang was funded by Wellcome after a successful precursor project Nappy Science Gang (Nappy Science Gang 2016), which explored the topic of how best to clean cloth nappies. In Parenting Science Gang (PSG), parent groups, predominantly women, designed and ran their own studies to answer questions of importance to them, collaborating with professional researchers who provided advice where required. The project team provided parents with the scaffolding to support them in deciding which research questions they wanted to answer and the methods needed to answer those questions, and then to collect and analyse data and disseminate findings. The project was run almost exclusively on Facebook, one of the key places where parents, particularly mothers, go to get parenting advice (Bartholomew et al. 2012). The project ran from 2017 to 2019.
Parenting Science Gang aimed to build parents’ confidence talking about science and build their science capital. Science capital is a way of looking at science-related forms of cultural and social capital, and is important because it helps foster engagement with science, technology, engineering, and maths (DeWitt, Archer, and Mau 2016; Archer et al. 2010).
The team also wanted the project to be enjoyable, because this is an important part of retaining participants (West and Pateman 2016), as well as parent-led. This goes beyond being a co-created project, as none of the project team were working as scientists. The lead (SC) is a professional public engagement practitioner. The project brought in scientists (termed “experts”) to answer the questions parents had. This paper is authored by paid members of the project team: the project lead (SC), project manager (AS), project coordinators (TGG and RB), and the internal evaluator (SW).
The structure of the project and its scientific outputs are reported elsewhere (see Collins et al. 2020), but here we focus on the impact on Parenting Science Gang participants, and the extent to which we achieved our aim of being parent-led. Both these topics are important, because often citizen science projects claim to be transformative in terms of outcomes on participants, but robust evaluation is often lacking (Tredick et al. 2017; Wehn et al. 2021; Davis et al. 2022). Figure 1 shows the different dimensions we consider herein. We begin by considering the effectiveness, or success, of the approach, looking at inclusivity through the demographics of our participants, the extent of co-production, and enjoyment. “Know” explores people’s science capital and parenting knowledge, “Think” examines participants’ confidence and attitudes towards science and parenting, and “Do” considers whether there have been changes in parenting actions or engagement with science. We do not examine scientific or socio-ecological and economic dimensions of evaluation; rather, we focus on the participant dimension, covering both process evaluation (aspects of how the project was run) and outcome evaluation (benefits for participants) (Kieslinger et al. 2018). At the time of project design and evaluation, we were not aware of any frameworks for measuring learning outcomes or other changes in participants, and now well-used tools such as Philips and colleagues’ framework for measuring individual learning outcomes from citizen science (Philips et al 2018), the Measuring Impact of Citizen Science (MICS) tool (MICS tools, undated), and Kieslinger and colleagues’ criteria for evaluating citizen science (Kieslinger et al 2018) did not exist. As such, we had to draw on our experiences of being involved with and evaluating previous projects, designing an evaluation approach that included formative and summative elements to allow us to iteratively improve the project as it progressed, and try to evidence any changes in participants over time.

Figure 1
The different dimensions of the project evaluated.
In this paper, we describe the methods used for evaluation and the results from the evaluation, and we end with some reflections on the process of evaluating.
Project approach
The project was facilitated by a project leader (SC), a project manager (AS), and two project coordinators or facilitators (TGG and RB working as a job-share). They are all mothers, and none of them were embedded in scientific institutions. In addition to these core members of the team, the project had two evaluators (one of whom was SW), both from academia. The project ran for two years (2017 to 2019) with four groups started in the first year and a further four in the second year. Two of the first-year groups were focused on breastfeeding: Breastfeeding Older Babies and Beyond PSG, and UK Breastfeeding and Parenting Support PSG, and they jointly ran a project exploring the composition of breastmilk produced for children over two years old, and another on the experiences of healthcare support for breastfeeding. Science Aware Natural Parenting PSG group ran a project on the effect of babywearing (sling use) on the skin and core temperature of babies, and Dumfries and Galloway Bumps Babies and Beyond PSG researched flexischooling in Scotland. Let Toys Be Toys looked at gender stereotyping in books, Big Birthas looked how mothers’ weight impacted experiences of pregnancy, and Mealtime Hostages explored characteristics of children with extreme selective eating. Across the eight groups, there were 2,608 unique members. Some people joined more than one group. Members were typically female (701 out of 719 in pre-survey, 97%), white (93%), a parent to one or two children (80%), based in the UK (87%) and 25–44 years old (94%) (Collins et al. 2020). A timeline of the project can be seen in Figure 2. The time each activity took varied from days to months. For example, some data collection activities took place in a day (collecting breastmilk samples) and others took weeks (reading books with children).

Figure 2
Timeline of the Parenting Science Gang project, showing the different activities that took place.
Evaluation Methods
Our approach to evaluation was pragmatic, designed to meet the funders expectations, give us ongoing feedback to iteratively improve the project, and not overburden participants. We used a range of methods to evaluate the project to understand the impacts of the project on participants and their families. These included a pre- and post-survey, in-depth interviews and discussion threads in the groups, as well as snapshot evaluations using the “hand, head, heart, bin” method at two in-person events, which involved participants writing down what they would take away for future projects (hand), things they had learned (head), things they had enjoyed (heart), and things they didn’t enjoy (bin). A full evaluation report was written for the funder (Wellcome) and to share learning from the project (Parenting Science Gang 2019a). This paper reports solely on the results of the pre- and post-project surveys, focusing on any effects of participation as shown in Figure 1.
After ethical approval was granted by a University of York Ethics Committee, a survey with categorical and free-text responses was administered via Google Forms. Information about the survey and how responses would be used was included at the top of the form, and in the Facebook posts promoting it. The survey included questions about science capital from the ASPIRES questionnaire (see UCL 2024). The survey questions can be seen in Supplemental File 1: Table 1. Some questions were common to both surveys, so we could assess any changes in our participants, whilst others were asked only in the pre- or post-survey.
The survey was posted on each of the Facebook groups at the beginning of the project, and members were asked to complete it when the groups were first set up. Then posts were made regularly so that members joining later could respond, although this was not a prerequisite for involvement. We had 719 responses to the pre-survey (28% response rate). Respondents were asked to give the name they used on Facebook as an identifier. At the end of the project, the post-survey was posted on the Facebook group, and again participants were asked to complete it. This had 272 respondents (10% response rate), many of whom unfortunately had not completed the first survey. Only 149 people completed both surveys (6% of total Parenting Science Gang participants).
Data analysis
Each respondent in the pre-survey was given a unique identifying code, the post-survey was searched for their Facebook name, and then the code replaced their personal data (Facebook name). Data were then analysed in two ways. Statistical analysis was conducted on quantitative data, whilst free-text comments were read and coded into broad categories. Data from each survey were analysed separately and together for the 149 participants who completed both surveys. No responses were compulsory, so some questions have a lower number of respondents.
Data was analysed using IBM SPSS Statistics to help answer our overarching evaluation questions: “How effective was our method?” (of engaging diverse parents in a parent-led science project) and “Have we helped parents change what they know, think, and do?”
Results
In this section, we begin by giving some demographic details about our participants to support evaluation of the inclusivity of our approach and the extent to which the project was co-produced, and then we discuss levels of enjoyment—all different elements of the approach’s effectiveness as a whole. We then discuss the three dimensions of “Know” (knowledge relating to science and parenting), “Think” (confidence and attitude), and “Do” (behaviour changes).
How effective were the methods employed?
Demographics
Across the eight Parenting Science Gang groups, there were 2,608 unique members, although we do not have demographic details for most of these as only 719 people completed the pre- and the post-survey. Participants were mainly female (97%), white (93%), and based in the UK, with smaller numbers of participants from other countries such as the United States, Canada, and Germany. This arose because of Facebook’s global platform, despite the team being based in the UK and all experiments being done there.
Most of our participants were in the 35–44 age bracket (see Figure 3), although there was a spread from 18 to 55 or over.

Figure 3
Age profile of Parenting Science Gang members. Data is from the post-survey; n = 272.
Although we tried to reach parents who tend to be underrepresented in science activities (particularly people identifying as Black) through gatekeeper organisations, the majority of our participants were white. Our participants were also more educated than the general public, with 63% of those completing the pre-survey having a science qualification at undergraduate degree level or higher (see Figure 4). This was higher still (72%) for those who were engaged enough to complete the post-project survey. This compares with 34% having any form of degree in England and Wales, according to the 2021 census (Office for National Statistics 2023).

Figure 4
Highest level of science qualification of Parenting Science Gang members, showing that the majority had at least an undergraduate degree. Data is from the pre-survey; n = 719.
The groups
Each Parenting Science Gang group had different numbers of members, with the smallest numbers (80) in the local parenting group in Dumfries and Galloway (Scotland) and in the Science Aware Natural Parenting group. The largest group was Breastfeeding Older Babies and beyond (900).
The proportion of participants responding to the post-survey from each of the groups largely reflects group size, but Science Aware Natural Parenting and UK Breastfeeding and Parenting Support groups were slightly overrepresented in the end survey, and the Mealtime Hostage group was underrepresented.
Participants were able to join as many groups as they wished, with around half joining more than one group, and just over a quarter joining three or more. Just under half of respondents visited the groups weekly or daily, showing that at least those responding to the survey tended to be well-engaged with the groups.
Attendance varied throughout the course of the project. Some parents became less engaged as the project went on (for example, if the topic the group chose to focus on wasn’t what they were interested in), some dropped out after contributing data (see Parenting Science Gang 2019b for case studies), and this influenced the number of respondents for the post-survey. One respondent noted: “Logistically I think a large part of the future of our groups project (analysis and future experiments) will fall to our scientist researcher, particularly as the SANP [Science Aware Natural Parenting] group has unfortunately seems to have lost steam a bit!”
The project was designed to be flexible so people could dip in and out when it suited them, but this was tricky for some of the groups when it came to analysis and dissemination of findings, as some had so few active members that the analysis and dissemination burden fell to a small number of parents: “There have been too few of us involved and so the burden has been too great on each of us.”
Was it parent-led?
The vast majority (88% of those responding to the post-survey question [n = 220]) said the project was parent-led. One parent said, “This was genuinely participatory. We asked the questions, set the parameters, analysed the results and others helped us in that. We weren’t just data collectors as in a lot of other “citizen science” kind of projects”
Another member said, “It was user led but heavily facilitated to ensure groups stays on task, especially when deciding on questions and topic,” and some felt that initially yes it was but “less so once it got to the collecting data and results stage.” Another found this frustrating:
“Ultimately I felt annoyed that after we came up with research questions, researched them, thought of experiments, etc, we ended up just doing something that one of the experts had suggested. Although we did end up voting on it, by that point we’d already spent a lot of time discussing that question (as if it was already the one we’d chosen), so it was pretty much a done deal.”
Another noted “To some extent. Eventually my sense was that the group leader(s) and the researchers who joined determined the direction of the project, the specific research questions, etc.” Certainly the project required lots of support and nudging from the co-ordinators, highlighted by numerous comments, but despite this support, one person noted that more coordination might have been helpful at times.
For some people, the user-led nature was confusing: “Very confusing as to who was leading it or what was going on. There were a lot of pointless posts so I had to switch off in the end”. And another said “I couldn’t tell, I didn’t know who the main people were, if they were paid etc”. A third felt although it was user led, it wasn’t very democratic:
“I did [participate regularly] till I had to take a break. Now I feel it’s mostly run by the core person [another parent] & they don’t have enough time to do it properly. I think a lot of user comment is lost or ignored due to lack of resources & I think this rather betrays the projects aim.”
The vast majority of participants also felt they had the opportunity to contribute ideas (94% of those responding, n = 214). The majority felt they were part of the team, at least to some extent (23% absolutely yes, 24% yes, 28% somewhat). There was a large minority who said Not really (17%, or 43 respondents) and a further 6 who said Not at all (2%). Some of those who said they had not had an opportunity to contribute noted that this was because looking after their children took priority, leaving them with little time or energy to participate. A few were confused about how to contribute.
Enjoyment
The majority of respondents in the post-survey quite enjoyed (36%) or really enjoyed (43%) participating in the project, though 20% (50 people) were neutral, and one said they didn’t enjoy it much. Some of the comments from those responding neutrally indicated that they had not been very involved:
“I don’t think I’ve been involved enough to really take anything from it.”
“I don’t really know what involvement I have had if any. Social media is so busy and I usually look at it in the middle of the night so I don’t concentrate much. Might be an idea to reiterate what the group does every so often.”
“I haven’t had the time I had hoped to dedicate to it. My responses reflect that rather than anything negative about the PSG.”
“I wouldn’t say I was a strong participant – more of an observer.”
The vast majority (88%) would recommend the project to someone else, 11% were unsure, and two people would not—both of whom were part of Mealtime Hostages. One of them (also a scientist) responded to the question, “What (if anything) do you feel YOU got out of being involved in Parenting Science Gang?” with “Honestly, it was a frustrating experience, both as a parent and as a researcher. I very much appreciate the efforts of the group leaders, but in the end I don’t think the research was representative of the broader ARFID [Avoidant/Restricted Food Intake Disorder] population, including my own child.” However, 95% of people said they would take part in a future project.
Have we helped parents change what they know?
The majority responded positively to the question, “Do you think you learned anything new from the project?” (see Figure 5), with 51% responding that they had learned a few things, and 26% saying that they had learned lots of things. Comments such as “I learnt more about how to actually “do” various qualitative and textual analysis methods taking part in PSG than I did academically (which included a social sciences methods MA, and a qualitative methods based PhD)” indicated that even some highly qualified participants learned things.

Figure 5
Responses to the post-survey (n = 272) about whether participants had learned anything new.
Again, 20% were neutral on this, with comments such as “I can’t remember what I’ve learnt so couldn’t say either way,” and “As a scientist I am already familiar with the scientific process.”
Science capital
We hoped that participants would speak to their families about the work, thereby increasing their child/ren’s understanding of science. In the post-survey, of our 272 respondents, 35% said they had spoken to their child/ren about the project, 18% said they had not because their child was too young, and 44% (90 people) said they had not, but did not give a reason.
For the 149 people who completed both surveys, there was no statistically significant change, using the Wilcoxon signed Rank test, in parents’ responses to any of the science capital questions (77% in both pre- and post-surveys selected “I usually understand what they are talking about” in response to the question, “Thinking of the stories about science you see or hear in the news, which of the statements below would you say best describes you?”) (see Supplemental File 2: Statistics). There was no difference in the pre- and post- surveys in what the parents reported they like doing. There was also no difference in whether people said they knew scientists (McNemar test).
The free-text responses revealed that sometimes what might have shown in the quantitative data as a positive outcome of increasing confidence, actually had some downsides. For example: “I’m far more confident in my choices. I’m a lot less anxious about my daughter’s eating & we discovered Forest School which brings out the very best in her. The only downside is that I can’t do everything as I’d like to e.g. flexischooling [the focus of one of the PSG groups] etc so I feel conflicted where I probably wouldn’t have done if I didn’t know there were alternatives to mainstream education.”
Unfortunately, due to the relatively small sample size of those who had school-aged children repeating the survey, it was not possible to assess the impact of the project on children.
Have we helped parents change what they think?
We asked parents about their confidence discussing science stories that they hear about in the news, and there was no statistical difference in these according to the Wilcoxon signed Rank test (see Supplemental File 2: Statistics): 92% were very or fairly confident in pre- and 91% in post-surveys, and for “I am capable of being a ‘scientist’ in certain contexts,” 82% strongly agree/agree in pre-survey and 84% in post-survey. Whilst statistical analysis of the paired survey data showed no difference in the science capital or attitude of parents (Figure 6), using the responses to both surveys separately does show some potential shifts in participants as a whole (see Figure 7), with 58% of respondents in the post-survey saying they strongly agreed that they were interested in science, compared with 28% in the pre-survey.

Figure 6
Percentage of participants who completed both surveys responding to “I am interested in science.”

Figure 7
Percentage of participants responding to “I am interested in science.”
There was a large increase in the percentage of positive responses to people feeling able to make a contribution to science (see Figure 8), with 46% of respondents in the post- survey agreeing and 27% strongly agreeing, compared with 26% and 4%, respectively, in the pre-survey, although again no statistical difference was found in the sample who completed both surveys (Figure 9).

Figure 8
Percentage of participants responding to “I am able to make a contribution to science.”

Figure 9
Percentage of participants who completed both surveys responding to ‘I am able to make a contribution to science.”
Changes in attitude to science
The post-survey asked respondents if it had changed their attitude to science. A slight majority of respondents (52%) said it had not, but 45% said that it had. This may be because our participants were more educated than the general population. Many of those who answered no commented that they already had positive attitudes. For example: “I believed in science before and strongly do now, so no.” Another noted “not really, except that it reinforces that everyone regardless of education can have good ideas!” Many of those saying yes talked about it being more accessible: “Yes I feel the “doing” is more accessible to people like me, not just the reading!” and “It feels closer, more accessible and real. I have a greater understanding of the process behind the results.” Others commented that the project made science seem more relevant: “I now see it as much more accessible. I used to think the science gene had skipped me (parents & grandparents all scientists, but I studied arts!), but now I realise it’s much more relevant to my daily life as a parent than I thought or realised it could be.”
Even some of those involved in scientific research expressed changes in attitudes. For example, someone who was doing a PhD noted “Yes, I have started to see myself as a scientist. Ridiculous maybe that I didn’t whilst doing a PhD, but I didn’t. I saw myself as a mum and a student, but not a scientist.”
Another scientist said: “Yes, actually – might seem strange because I am a scientist, but the enthusiasm of everyone involved has helped rekindle my enthusiasm for science in general, when my day-to-day work has lots of slog, often followed by rejection and disappointment, with bright spots of cool results or something being accepted seeming few and far between: being able to see real science being done, and people actually getting to do something they designed and being so excited about it all, has helped me look more positively towards my job.”
Have we helped parents change what they do?
Respondents were provided with a list of sources that might provide advice and were asked to select who they “Trust most” and “Would ignore/distrust.” A McNemar sign test showed no change in responses for the 149 people who completed two surveys (see Supplemental File 2: Statistics). A high percentage (n = 719) chose scientists as people they would “Trust most” in the pre- and post-surveys (see Supplemental File 3: Figures 1 and 2), and there was a significant increase in the number of people selecting scientists in their post-test compared with their pre-test (n = 149, Z = –2.240, p = 0.025 Sign test), with 34 people who had not ticked “Scientists” in the pre-rest ticking this in the post-test. It should be noted, however, that there were 17 people who selected “Scientists” as a place they used for advice in the pre-survey but not the post-survey. One parent in their pre-survey said “I don’t trust scientists to give advice but I value good evidence in informing my decisions.” and in their post-survey wrote “I trust scientists to provide credible evidence on which to base a decision.”
Books and friends were also selected by a high proportion of people as sources they would “Trust most.” Trust in health professionals was much more mixed, with a respondent to the post-survey noting, “GP & health visitors seem to have outdated information, poor knowledge of current evidence base & a lack of time/resources to address issues properly.” Another wrote, “Health professionals don’t all seem to be up to date, a few have been very good but with most, I would definitely be fact checking.”
Changes in parenting style, or other side effects
We also wanted to explore whether there were any changes in the way people parented: 34% of respondents to this question (n = 200) said it had changed their parenting style, with 58% saying it had not. Many of those saying no said that it had confirmed for them that their previous decisions were the right ones, or given them confidence in their approach. One example is, “No, it has helped to further cement my views though in a positive manner thank you.” Another response was, “No, but more secure in trusting my instincts on breastfeeding.” A few said it had not changed their parenting but they had passed the information along to other parents: “No because I rarely wear [carry in a sling] a baby anymore. But I do tell others about it.”
Some who said yes said it was little things: “In small ways – book recommendations to read with my kids, that kind of thing.” Other positive comments included: “Yes – more aware of avoiding gender stereotypes with my son than before the experiment.” For some of these parents, it led to conversations with others about parenting: “It led to long discussions with my partner, so we both now base our parenting decisions on doing research and interpreting findings.”
In some cases, the project gave parents the confidence to trust their own instincts, and question sources:
“I’m less terrified by pseudoscientific scaremongering type of parenting advice; I question it vigorously. In one case PSG gave me a better understanding of the actual evidence which informed the (simplified) advice I was given about bedsharing and I felt confident to make a change which made our family life a lot happier. And generally I actually feel more confident to follow my own instinct as a parent because I see how science provides a few trustworthy pieces of the puzzle and plenty is as yet unknown, so the rest is up to me.”
Particularly for the breastfeeding studies, many mothers commented that it had given them more confidence to continue breastfeeding longer than they might have otherwise, partly because meeting with other mothers also breastfeeding older infants normalised it for them: “Breastfeeding wise I’ve just felt validated feeding to natural term, being involved in something run by other parents who have also continued breastfeeding older babies.”
Summary of results
Whilst the project was conceived before the Kieslinger et al. (2018) evaluation framework, here we evaluate Parenting Science Gang against their framework for participant outcomes, as it allows comparison with other projects. This is shown in Table 1. This shows that some areas saw more change than others, with good target group alignment of activities, high degrees of participation, and more mixed outcomes for the other participant dimensions, including aspects such as providing training for participants. As a whole, we were unable to quantitatively demonstrate changes for most of our variables, but the quotes presented from participants here show that, at least for some participants, changes did occur, and they highlight the value of using mixed-methods to understand impacts of projects, and the challenges with using a pre- and post-survey design. Our cohort of participants who responded to the surveys were overall highly educated and quite confident talking about science, meaning the scope for improvements was lower than if we had had a different baseline.
Table 1
Evaluation of Parenting Science Gang using “Participant dimension” from Kieslinger et al. 2018.
| CRITERIA RELATING TO THE PARTICIPANT DIMENSION | ||
|---|---|---|
| Target group alignment | Does the project have an involvement plan that considers specifics of different target groups? | Yes – each group designed their own engagement strategy |
| Are the options for participation and the degree of involvement diversified (e.g., gamification)? | Partly – had to be on Facebook, but lots of flexibility in how much and which stages to be involved in | |
| Degree of participation intensity | Can citizens participate in various project phases? | Yes – from deciding topic through to dissemination |
| Do citizens and scientists work as mutually respected partners in the knowledge generation process? | Yes – scientists brought in as experts and co-designed methods with parents | |
| Facilitation and communication | Are support and training measures adapted to the different participant groups? | Partly – limited training through the project (mainly through question and answer sessions) |
| Are objectives and results clearly and transparently communicated? | Partly – objectives unclear at times and extent of results communication determined by group and scientist enthusiasm | |
| Do citizens receive regular feedback? | Yes – regularly during lifetime of project (Facebook updates, newsletter, website) | |
| How interactively is communication and collaboration between scientists and citizens organised? | Very – Question and answer sessions led by parents. | |
| Knowledge, skills and competencies | What are the learning outcomes with regards to new knowledge, skills and competencies for the participants? | Varied depending on the participant – high for some. |
| Science literacy | Does the project contribute to a better understanding of science? | Partly - depending on the participant – high for some. |
| Does the project contribute to a better understanding of the scientific topic? | Partly – depending on the participant – high for some. | |
| Behaviour and ownership | Does the project foster ownership amongst participants? | Partly – depending on the participant, but most felt it was parent-led. |
| Does the project contribute to facilitating personal change in behaviour or political citizenship? | Yes – for some participants | |
| Motivation and engagement | Does the project raise motivation, self-esteem and empowerment amongst participants? | No significant difference in confidence talking about science but yes for some participants. |
| Are participants motivated to continue the project or involve in similar activities? | Yes – 95% would take part in a similar project | |
Whilst not all aspects of the framework were met, the project was entirely parent-led, and as such, the focal areas studied, project objectives, any training provided (mainly through question-and-answer sessions with relevant researchers), and how results were communicated were determined by parent participants themselves.
Discussion
Parenting Science Gang was indeed parent-led and has previously been described as a “radical” form of co-created citizen science (Collins et al. 2020), but for some this was not a positive but confusing experience. As Fischer et al. (2021) notes, more “signposting” or explanation of the stages of the research process is required to orient participants. This can help to reassure participants that, for example, determining research questions is time-consuming and complicated. Without this, confusion can lead to disengagement. The Parenting Science Gang team perceived that posts were regularly made in the groups about the overall plan, where we were in it, and what was happening next. It may be that this confusion was inevitable as participants were dropping into the group whilst visiting Facebook for social reasons, and therefore only seeing sporadic posts from the group. The project was designed so that people could participate as much or as little as they wanted; however, we acknowledge that those who participated less frequently may have felt “lost” or confused at times. Future projects may want to conduct even more “signposting.” Co-created citizen science projects in particular require large amounts of staff time to support them. Some of the confusion felt by some Parenting Science Gang participants could be down to insufficient resourcing. The project had two part-time project co-ordinators (TGG and RB) working across the eight groups, which made keeping up with all activities challenging. Herodotou et al. (2022) surveyed participants in online “missions” on a platform called nQuire and found that the majority were not keen to lead their own studies. They note that the shift from participant to creator requires significant effort to build capacity in research design and lead people through the research process. In their study, participants cited lack of time as a key barrier to greater engagement, and Fischer et al (2021) highlight that considerable time, and interest, is required from participants for designing projects. The large number of participants in most of the PSG Facebook groups meant that parents could opt in and out when they wanted, although some of the smaller groups did struggle in the latter stages with drop outs, as evidenced by our previously quoted participant: “the burden has been too great on each of us.” Having a large pool of participants to draw on for participant-led projects is recommended.
Regarding diversity, we aimed to reach a representative proportion of society, but this was extremely challenging. In the UK in 2015, 86% of the population were white, 34% were over 55, and just over half (52%) were female (Pateman et al. 2021). In comparison, our project reached more women, and younger participants, than the population, which is unsurprising given our topic. Unfortunately, despite reaching out to “gatekeeper” organisations, we were not successful in reaching a representative proportion of participants who identified as being from ethnic groups other than white. Sardo et al. (2025), in their UK citizen science project, also found this approach was not successful in recruiting more representative participants. Humm and Schrögel (2020) emphasise the importance of taking time to build trusting relationships with partner organisations, including spending time in their spaces to understand their needs before involving them in projects. This was tricky in Parenting Science Gang because it was a relatively short-term project and all online, meaning we could not physically go and visit groups to build relationships.
Previous commentators have suggested that co-created projects can have more transformative effects on participants than less participatory forms of citizen science (Gunnell et al. 2021). Parenting Science Gang could be considered a radical form of co-creation, as it was parent-led, but we were not able to statistically demonstrate evidence of transformation. However, the statistical analysis shows only part of the picture, as the project did have profound impacts on some participants, which is evidenced through free-text comments from the survey, for example:
“I found new areas of interest and reawoke old ones. I’ve made friends and had my faith in myself and abilities restored. I can still use my brain and read academic texts (and science ones at that!) and have intelligent, grown up conversations! I’ve been waffling about returning to work and what I could do, I suddenly have more ideas and enthusiasm thanks to working with PSG and meeting the others involved.”
Other participants said the project had given them “The courage to re-apply for my masters and begin a career in science” and that it had “shaped my current career direction!”. The evaluation report and case studies (Parenting Science Gang 2019a,b) contains other examples derived from in-depth interviews and comments made in the Facebook groups. The lack of statistical evidence for change may be due to flaws in the evaluation design, as discussed in our Limitations section.
There was some evidence that trust in scientists increased through participating in Parenting Science Gang, although as Bedessem et al. 2023 demonstrate in their study of biodiversity citizen science participants, trust in science and scientists and its relationship to engagement and levels of education can be complex and multi-faceted. In our survey, 17 people selected scientists as a source of parenting advice in their pre-survey, but then not in their post-survey, suggesting perhaps interactions with scientists lowered their trust in them to give parenting advice. More research is needed into how citizen science participation impacts trust in the scientific process and in scientists themselves, as it is often cited as a benefit of doing participatory research.
Limitations and reflections on evaluation
We faced challenges with the aspects of our evaluation reported here. Although we encouraged, rather than obliged, people to complete the pre- and post-survey, this may have set up a power dynamic in which participants were feeling “researched,” although the evaluators stressed that they were not part of the core support team. As the project was parent-led, perhaps the evaluation should have been participatory, too, and future projects may wish to do this using tools such as the Science Products Inventory (Edson et al 2024) or the WeObserve Co-Evaluation tool (WeObserve, undated). There were some logistical issues with the survey, namely that at the time, Facebook did not have the functionality to alert everyone to the existence of the survey; rather, they had to visit the group and see the pinned post at the top of the page, or happen to visit at a time when a reminder post had been issued. This likely contributed to the drop off in respondents for the post-survey (272) compared with the number of initial respondents (719) (with only 149 completing both), and may have biased the responses to those who were still actively engaging with the group. The drop off may also have been due to people feeling over-researched, as alongside the surveys, we also conducted polls and interviews, and did some evaluation exercises on the few in-person meet ups (see Parenting Science Gang 2019a).
The post-survey was particularly challenging because many participants from the pre-survey had already left the group or turned off Facebook notifications, and nearly half of those completing the post-survey did not complete the first, either because they had joined the group later on so missed the requests to complete it, or missed the “pinned post” asking them to complete the pre-survey when they joined. The length of the survey may have been off-putting to many respondents.
In the future, projects could just ask for participants to complete one survey at the end of the project, including questions reflecting on their levels of knowledge, confidence, etc., at its inception. Although this may cause issues if people are unable to remember the beginning of the project, it may be a better option than requiring people to complete two surveys. Another issue with surveying people at the beginning of the project, as also identified by Robinson et al. 2024, is that the pre-survey is issued before trusting relationships have been established and therefore may exacerbate power dynamics.
Although we were not able to detect many changes in our participants as a whole through statistical tests, the free-text responses were very important for capturing evidence of change. They were also useful for understanding why people had responded the way they did, for example, that they felt their confidence around science was high at project inception so it didn’t need boosting. These free-text comments, combined with the other formative evaluation methods reported elsewhere (Parenting Science Gang 2019a), gave us a rich picture of how the project was working and what amendments needed to be made.
Future projects may wish to consider involving participants more in the evaluation process, as Kieslinger et al. (2022) argue that participatory approaches require participatory methods for evaluation. This could involve talking to participants more at the beginning about what they wanted out of the project and then designing an evaluation process to measure these goals, which may increase levels of engagement with the evaluation.
Conclusion
Detecting significant changes in citizen science participants’ knowledge, confidence, attitudes, etc., in projects such as Parenting Science Gang is challenging because of biases in who completes surveys, the number of drop-outs over time, and often small sample sizes. However, we have shown here that combining quantitative measures in surveys with free-text boxes can provide details on how projects have impacted, or not, participants. Such text boxes, if combined with other evaluation approaches such as interviews can provide rich insights. Project initiators should think carefully about both summative and formative evaluation. Summative evaluation can be used to capture any changes in participants over time, but pre- and post-surveys may not be the most useful method due to participant drop-outs, particularly if projects are allowing flexible participation. Formative evaluation is essential for getting ongoing feedback about projects, as amendments can then be made in order to maximise the chances of having positive outcomes for participants.
Parenting Science Gang demonstrated that this “radical” form of citizen science, led by people who are not researchers, can lead to scientific outputs and outcomes (see Collins et al. 2020) and have transformative effects on some participants (e.g., in terms of their confidence). We recommend all citizen science practitioners to consider how they can be more participant-led in their work. Such projects require excellent facilitation and project management, and significant amounts of signposting and scaffolding to ensure participants know where in the research process they are, what the next steps are, and how they can get involved. Running such projects on existing platforms (in this case, Facebook) is a good way of recruiting large numbers of participants who are able to contribute at times that suit them, but the downside of this is that those who are less involved may miss the signposting and so feel lost.
Supplementary Files
Supplementary File 1
Table 1: The questions asked in the Parenting Science Gang survey(s). DOI: https://doi.org/10.5334/cstp.972.s1
Supplementary File 2
Statistics: Statistical test results for all tests mentioned in this paper. DOI: https://doi.org/10.5334/cstp.972.s2
Supplementary File 3
Figures 1 and 2: Responses to the question about trust in sources of advice about parenting. DOI: https://doi.org/10.5334/cstp.972.s3
Data Accessibility Statement
Anonymised data are available on request from the lead author (SW).
Ethics and Consent
Ethical approval was provided by University of York Department of Environment and Geography Ethics Committee on 24/03/2017 (no number provided), with numerous subsequent amendments as parents decided what research to conduct. Consent was given at the time of completing the survey to use anonymised results.
Acknowledgements
The team would like to thank Karen Bultitude for her work as an external evaluator for the project, and Jean McKendree for closing the post-survey whilst SW was on maternity leave.
Author Contributions
SC conceptualisation (lead), funding acquisition, methodology, investigation; TGG and RB conceptualisation, methodology, investigation, analysis; AS conceptualisation, project administration, analysis; RA analysis, visualisation, writing – original draft; SW conceptualisation, evaluation, analysis, writing – original draft, review, and editing. All authors consented to the submission of the manuscript.