Characteristics of a Citizen Science Approach to Mental Health Research: International Delphi Consultation

Joy Llewellyn-Beardsley; Stefan Rennick-Egglestone; Liz Dowthwaite; Olamide Todowede; Chris Greenhalgh; Doreen Boyd; Stuart Moran; Colleen Ewart; Corrine Hendy; Mike Slade

doi:10.5334/cstp.971

Characteristics of a Citizen Science Approach to Mental Health Research: International Delphi Consultation

Volume 11 (2026): Issue 1

By: Joy Llewellyn-Beardsley , Stefan Rennick-Egglestone , Liz Dowthwaite , Olamide Todowede , Chris Greenhalgh , Doreen Boyd , Stuart Moran , Colleen Ewart , Corrine Hendy and Mike Slade

Open Access

|Jul 2026

Abstract

Citizen science, in which members of the public contribute to scientific research, has been adopted as a participatory mental health research methodology. An expert consultation was undertaken to identify consensus on (1) distinguishing characteristics of mental health citizen science (MHCS), and (2) approaches to improving the quality and sustainability of the experience for mental health citizen scientists.

A two-round online Delphi consultation was conducted, recruiting project leads and public contributors. Round 1 (R1) comprised 59 questions, addressing distinguishing features of MHCS projects; relevance of the European Citizen Science Association (ECSA) principles; differences between MHCS and other methodologies; and creating high-quality and sustainable participation. Round 2 (R2) repeated questions where consensus had not been reached, and asked new questions based on R1 feedback.

R1 included 34 experts (27 project leads and seven public contributors from 10 countries), with consensus on 18 key MHCS features. R2 included 15 experts (six countries), with consensus on a further 12 items. Key findings on distinctiveness include consensus that public contributors should be involved in co-creation of MHCS at all stages. Adhering to scientific methods, addressing safeguarding and accessibility/inclusion needs, and providing support for distress are also considered definitive. Consensus on high-quality and sustainable citizen experiences included access to learning resources, increasing involvement over time, and co-operation between MHCS projects.

The thirty features on which expert consensus was established can be used as a basis for designing context-sensitive MHCS. Experts concur that coproduction should be a defining feature of MHCS rather than one possible category.

References

Aceves-Bueno, E., Adeleye, A.S., Feraud, M., Huang, Y., Tao, M., Yang, Y., and Anderson, S.E. (2017) The accuracy of citizen science data: a quantitative review. Bulletin of the Ecological Society of America, 98(4), pp. 278–290. https://www.jstor.org/stable/90013289
Search in Google Scholar Back to article
Alon, I., Haidar, H., Haidar, A., and Guimon, J. (2025) The future of artificial intelligence: Insights from recent Delphi studies. Futures, 165, p.103514. DOI: 10.1016/j.futures.2024.103514
Open DOI Search in Google Scholar Back to article
Ball, M.P., Bobe, J.R., Chou, M.F., Clegg, T., Estep, P.W., Lunshof, J.E., Vandewege, W. et al. (2014) Harvard Personal Genome Project: lessons from participatory public research. Genome Medicine 6(2), pp.1–7. 10.1186/gm527
Open DOI Search in Google Scholar Back to article
Batty, R., Florescu, A., Gulliksen, E., Wong, A., Kamvar, R., Pinfold, V., Rennick-Egglestone, S. et al. (2024) The potential of citizen mental health science. London: Wellcome Trust. Available from: 10.21955/wellcomeopenres.1115398.1
Open DOI Search in Google Scholar Back to article
Beange, I. and Collins, S. (2022) Depression Detectives (Citizen Science). Edinburgh Open Research Conference Proceedings, Edinburgh, UK, on 27 May 2022, Vol 1. DOI: 10.2218/eorc.2022.7114
Open DOI Search in Google Scholar Back to article
Blacker, S., Kimura, A.H., and Kinchy, A. (2021) When citizen science is public relations. Social studies of science, 51, pp. 780–796. DOI: 10.1177/03063127211027662
Open DOI Search in Google Scholar Back to article
Bonhoure, I., Cigarini, A., Vicens, J., and Perelló, J. (2019) Citizen social science in practice: a critical analysis Of A Mental Health Community-Based Project. Palgrave Communications, 6. DOI: 10.31235/osf.io/63aj7
Open DOI Search in Google Scholar Back to article
Cipriani, A., Seedat, S., Milligan, L., Salanti, G., Macleod, M., Hastings, J., Thomas, J., et al. (2023) New living evidence resource of human and non-human studies for early intervention and research prioritisation in anxiety, depression and psychosis. BMJ Mental Health, 26(1). DOI: 10.1136/bmjment-2023-300759
Open DOI Search in Google Scholar Back to article
Colder Carras, M., Machin, K., Brown, M., Marttinen, T.-L., Maxwell, C., Frampton, B., Jackman, M., et al. (2023) Strengthening review and publication of participatory mental health research to promote empowerment and prevent co-optation. Psychiatric Services, 74(2), pp.166–172. DOI: 10.1176/appi.ps.20220085
Open DOI Search in Google Scholar Back to article
Collins, S.E., Clifasefi, S.L., Stanton, J., Straits, K.J., Gil-Kashiwabara, E., Rodriguez Espinosa, P., Nicasio, A. V., et al. (2018) Community-based participatory research (CBPR): Towards equitable involvement of community in psychology research. American Psychologist, 73, 884. DOI: 10.1037/amp0000167
Open DOI Search in Google Scholar Back to article
Corrigan, P.W. and Twiss, M., (2026). What are the implications of investigator positionality for mental health services research?. The British Journal of Psychiatry, 228(2), pp.95–97.DOI: 10.1192/bjp.2025.114
Open DOI Search in Google Scholar Back to article
den Broeder, L., Devilee, J., van Oers, H., Schuit, A. J. and Wagemakers, A. (2018) Citizen Science for public health. Health Promotion International, 33(3), pp. 505–514. DOI: 10.1093/heapro/daw086
Open DOI Search in Google Scholar Back to article
Fontúrbel, F.E. (2023) Predatory Citizen Science? Bulletin of the Ecological Society of America, 104(3), p.e2075. DOI: 10.1002/bes2.2075
Open DOI Search in Google Scholar Back to article
Güell, E., Benito-Amat, C. and Molas-Gallart, J. (2023) Priority setting in mental health research: A scoping review of participatory methods. Mental Health & Prevention, 30, p.e200279 DOI: 10.1016/j.mhp.2023.200279
Open DOI Search in Google Scholar Back to article
Haklay, M., Dörler, D., Heigl, F., Manzoni, M., Hecker, S., Vohland, K. et al. (2021a) What is citizen science? The challenges of definition. In: K. Vohland, A. Land-Zandstra, L. Ceccaroni, R. Lemmens, J. Perelló, M. Ponti, R. Samson et al (eds.), The science of citizen science (pp.13–33). London: Springer. DOI: 10.1007/978-3-030-58278-4_2
Open DOI Search in Google Scholar Back to article
Haklay, M., Fraisl, D., Greshake Tzovaras, B., Hecker, S., Gold, M., Hager, G., Ceccaroni, L. et al. (2021b). Contours of citizen science: a vignette study. Royal Society Open Science, 8(8). DOI: 10.1098/rsos.202108
Open DOI Search in Google Scholar Back to article
Hawke, L. D., Dada-Phillips, W., Seiyad, H., Orson, J., Goldsmith, L., Conway, S., Jordan, A. et al. (2025) Best practices guidelines for the engagement of people with lived experience and family members in mental health and substance use health research: a modified Delphi consensus study. Health Expectations, 28, e70152. DOI: 10.1111/hex.70152
Open DOI Search in Google Scholar Back to article
Hawke, L. D., Sheikhan, N. Y., Jones, N., Slade, M., Soklaridis, S., Wells, S. and Castle, D. (2022) Embedding lived experience into mental health academic research organizations: critical reflections. Health Expectations, 25(5), pp.2299–2305. DOI: 10.1111/hex.13586
Open DOI Search in Google Scholar Back to article
Israel, B. A., Schulz, A. J., Becker, A. B., Reyes, C. L., Parker, E. A. and Reyes, A. G. (2026). Community-based participatory research: evolution and significant developments. Annual Review Of Public Health, 47, 135–157. DOI: 10.1146/annurev-publhealth-082224-022223
Open DOI Search in Google Scholar Back to article
Jorm, A. F. 2015. Using the Delphi expert consensus method in mental health research. Australian & New Zealand Journal of Psychiatry, 49(10), pp.887–897. DOI: 10.1177/0004867415600891
Open DOI Search in Google Scholar Back to article
Mah, H., Dobson, R. and Thomson, A. (2025). The Importance Of Lived Experience: A Scoping Review On The Value Of Patient And Public Involvement In Health Research. Health Expectations, 28, e70205. DOI: 10.1111/hex.70205
Open DOI Search in Google Scholar Back to article
Niederberger, M. and Spranger, J. (2020). Delphi technique in health sciences: a map. Frontiers in Public Health, 8, p.457. DOI: 10.3389/fpubh.2020.00457
Open DOI Search in Google Scholar Back to article
Pain, R., Whitman, G. and Milledge, D. (2022). Participatory action research toolkit: An introduction to using PAR as an approach to learning, research and action. Practice Guide, Durham Univeristy. Avaiable from: https://eprints.icstudies.org.uk/id/eprint/293
Search in Google Scholar Back to article
Pinfold, V., Burgess, A., Couperthwaite, L., Garavini, E., Gibson, J., Kamvar, R., Kenny, A. et al. (2025). Emerging practice in mental health patient and public involvement research advisory groups: a narrative review. BMC Psychiatry, 25(1), p. 722. DOI: 10.1186/s12888-025-07120-8
Open DOI Search in Google Scholar Back to article
Pykett, J., Chrisinger, B., Kyriakou, K., Osborne, T., Resch, B., Stathi, A., Toth, E. et al. (2020) Developing a Citizen Social Science approach to understand urban stress and promote wellbeing in urban communities. Humanities & Social Sciences Communications, 6 (85). DOI: 10.1057/s41599-020-0460-1
Open DOI Search in Google Scholar Back to article
Remmers, G., Albert, A., Laer, J. V., Wildevuur, S., Groot, M. D., Imre, B. and Covernton, E. (2024). Citizen Science For Health: An International Survey On Its Characteristics And Enabling Factors. Citizen Science: Theory and Practice, 9(1). DOI: 10.5334/cstp.693
Open DOI Search in Google Scholar Back to article
Richmond, L., Caton, N., Downs, J., Newton, A., Devereux-Fitzgerald, A. and Brooks, H. (2023). Creating positive experiences of involvement in mental health research. The Lancet Psychiatry, 10(3), pp.220–227. DOI: 10.1016/S2215-0366(22)00430-8
Open DOI Search in Google Scholar Back to article
Robinson, L. D., Cawthray, J. L., West, S. E., Bonn, A., and Ansine, J. (2018) Ten principles of citizen science. In: S. Hecker, M. Haklay, A. Bowser, Z. Makuch, J. Vogel, & A. Bonn (eds.) Citizen Science: Innovation in Open Science, Society and Policy (pp. 27–40). London: UCL Press. 10.14324/111.9781787352339
Open DOI Search in Google Scholar Back to article
Robinson, J. A., Kocman, D., Speyer, O. and Gerasopoulos, E. (2021) Meeting volunteer expectations—a review of volunteer motivations in citizen science and best practices for their retention through implementation of functional features in CS tools. Journal of Environmental Planning and Management, 64(12), pp. 2089–2113. DOI: 10.1080/09640568.2020.1853507
Open DOI Search in Google Scholar Back to article
Shang, Z (2023) Use of Delphi in health sciences research: a narrative review. Medicine, 102(7), p. e32829. DOI: 10.1097/MD.0000000000032829
Open DOI Search in Google Scholar Back to article
Silverio, S.A., Sheen, K.S., Bramante, A., Knighting, K., Koops, T.U., Montgomery, E., November, L. et al. (2022). Sensitive, challenging, and difficult topics: Experiences and practical considerations for qualitative researchers. International Journal of Qualitative Methods, 21, p.16094069221124739. DOI: 10.1177/160940692211247
Open DOI Search in Google Scholar Back to article
Smith, E., Bélisle-Pipon, J.-C. and Resnik, D. (2019) Patients as research partners; how to value their perceptions, contribution and labor? Citizen Science: Theory and Practice, 4(1). DOI: 10.5334/cstp.184
Open DOI Search in Google Scholar Back to article
Staniszewska, S., Brett, J., Simera, I., Seers, K., Mockford, C., Goodlad, S., Altman, D.G. et al. (2017). GRIPP2 reporting checklists: tools to improve reporting of patient and public involvement in research. BMH, 358. DOI: 10.1136/bmj.j3453
Open DOI Search in Google Scholar Back to article
Todowede, O., Lewandowski, F., Kotera, Y., Ashmore, A., Rennick-Egglestone, S., Boyd, D., Moran, S. et al. (2023) Best practice guidelines for citizen science in mental health research: systematic review and evidence synthesis. Frontiers in Psychiatry, 14, p.e1175311. DOI: 10.3389/fpsyt.2023.1175311
Open DOI Search in Google Scholar Back to article
Todowede, O., Rennick-Egglestone, S., Boyd, D., Moran, S., Bell, A., Sweeney, A., Hart, A. et al. (2025). How Can Citizen Science Enhance Mental Health Research Quality: Theory Of Change Development. BMJ Open, 15, e091007. DOI: 10.1136/bmjopen-2024-091007
Open DOI Search in Google Scholar Back to article
Tøttrup, A. P., Lillemark, M. R., Hansen, J., Svendsen, S. H., Hickinbotham, S., Toftdal, M., Juhl, H. A. et al. (2025) A Concept for Co-Creation in Participatory Science: Insights From Developing the Archaeological Next Generation Lab. Citizen Science: Theory and Practice, 10(1) p.22. DOI: 10.5334/cstp.832
Open DOI Search in Google Scholar Back to article
Van Vaerenbergh, Y. and Thomas, T.D., (2013). Response styles in survey research: A literature review of antecedents, consequences, and remedies. International journal of public opinion research, 25(2), pp.195–217. DOI: 10.1093/ijpor/eds021
Open DOI Search in Google Scholar Back to article
Veldmeijer, L., Terlouw, G., Van Os, J., Van Dijk, O., Van’t Veer, J. and Boonstra, N. (2023). The involvement of service users and people with lived experience in mental health care innovation through design: systematic review. JMIR Mental Health, 10, p.e46590. DOI: 10.2196/46590
Open DOI Search in Google Scholar Back to article
Weirauch, V., Mainz, A., Nitsche, J., Busse, T.S. and Meister, S., (2026). Consensus-based reporting guidelines for participatory development and evaluation of digital health interventions. NPJ Digital Medicine 9,(169). DOI: 10.1038/s41746-026-02355-5
Open DOI Search in Google Scholar Back to article
Wiggins, A. and Wilbanks, J. (2019) The rise of citizen science in health and biomedical research. The American Journal of Bioethics, 19(8), pp.3–14. DOI: 10.1080/15265161.2019.1619859
Open DOI Search in Google Scholar Back to article

Articles in this issue

DOI: https://doi.org/10.5334/cstp.971 | Journal eISSN: 2057-4991

Journal RSS Feed

Language: English

Page range: 16 - 16

Submitted on: Dec 18, 2025

Accepted on: Jun 12, 2026

Published on: Jul 22, 2026

Published by: Ubiquity Press

In partnership with: Paradigm Publishing Services

Keywords:

Citizen science,

community science,

mental health,

participatory methods,

mental health citizen science,

Delphi study

© 2026 Joy Llewellyn-Beardsley, Stefan Rennick-Egglestone, Liz Dowthwaite, Olamide Todowede, Chris Greenhalgh, Doreen Boyd, Stuart Moran, Colleen Ewart, Corrine Hendy, Mike Slade, published by Ubiquity Press
This work is licensed under the Creative Commons Attribution 4.0 License.

Volume 11 (2026): Issue 1