Skip to main content
Have a personal or library account? Click to login
Evaluating Impact on Co-Researchers: Instruments for Participatory Projects and Insights from Practice with Citizens, Artists, and Artificial Intelligence Experts Cover

Evaluating Impact on Co-Researchers: Instruments for Participatory Projects and Insights from Practice with Citizens, Artists, and Artificial Intelligence Experts

Open Access
|Nov 2026
Read online

References

  1. Almatrafi, O., Johri, A. and Lee, H. (2024) A systematic review of AI literacy conceptualization, constructs, and implementation and assessment efforts (2019–2023). Computers and Education Open, 6. DOI: 10.1016/j.caeo.2024.100173
  2. Bauer, L. and Meyer, K. (2025) How to explain AI: Impact & Engagement Case for the Participatory and Collaborative Development of an Artistic Intervention to Promote AI Literacy. DOI: 10.5281/zenodo.16753066
  3. Beck, S., Bergenholtz, C., Bogers, M., Brasseur, T.-M., Conradsen, M. L., Di Marco, D. and Distel, A. P., et al. (2022) The Open Innovation in Science research field: a collaborative conceptualisation approach. Industry and Innovation, 29(2), pp. 136–185. DOI: 10.1080/13662716.2020.1792274
  4. Bornmann, L. (2013) What is societal impact of research and how can it be assessed? a literature survey. Journal of the American Society for Information Science and Technology, 64(2), pp. 217–233. DOI: 10.1002/asi.22803
  5. Breuer, E., Lee, L., de Silva, M. and Lund, C. (2016) Using theory of change to design and evaluate public health interventions: a systematic review. Implementation science: IS, 11(63). DOI: 10.1186/s13012-016-0422-6
  6. Donovan, C. (2011) State of the art in assessing research impact: introduction to a special issue. Research Evaluation, 20(3), pp. 175–179. DOI: 10.3152/095820211X13118583635918
  7. ECSA (European Citizen Science Association) (2015) Ten Principles of Citizen Science. DOI: 10.17605/OSF.IO/XPR2N
  8. Gönner, J. von, Herrmann, T. M., Bruckermann, T., Eichinger, M., Hecker, S., Klan, F. and Lorke, J., et al. (2023) Citizen science’s transformative impact on science, citizen empowerment and socio-political processes. Socio-Ecological Practice Research, 5(1), pp. 11–33. DOI: 10.1007/s42532-022-00136-4
  9. Haklay, M., Motion, A., Balázs, B., Kieslinger, B., Greshake Tzovaras, B., Nold, C., Dörler, D., et al. (2020) ECSA’s Characteristics of Citizen Science. Zenodo. DOI: 10.5281/zenodo.3758668
  10. Kieslinger, B., Schäfer, T., Heigl, F., Dörler, D., Richter, A., and Bonn, A. (2018) Evaluating citizen science: Towards an open framework. In: Bonn A, S. Hecker S, Haklay M, Bowser A, Makuch Z, and Vogel J (eds.), Citizen Science. UCL Press. pp. 81–96. https://www.jstor.org/stable/j.ctv550cf2.13
  11. LBG OIS Center (2021) The OIS Impact Model. Ludwig Boltzmann Gesellschaft. DOI: 10.5281/zenodo.10078636
  12. LBG OIS Center (2022) The OIS Reflection Instruments. Ludwig Boltzmann Gesellschaft. DOI: 10.5281/zenodo.10079448
  13. LBG OIS Center (2023) The Logic Model Template. Ludwig Boltzmann Gesellschaft. DOI: 10.5281/zenodo.15025982
  14. Long, D. and Magerko, B. (2020) What is AI Literacy? Competencies and Design Considerations. In: Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems (CHI ’20). ACM, pp. 1–16. DOI: 10.1145/3313831.3376727
  15. Mahve-Beydokhti, M. and Wailzer, M. (2026) Designing Co-Creative Theory of Change Processes in Research. LBG Open Innovation in Science Center. DOI: 10.5281/zenodo.21372127
  16. Meyer, K., Meneweger, T., Bauer, L. and Mara, M. (2023) How to explain AI in an understandable way? Citizen scientists, AI experts and artists find answers together. In: Proceedings of Austrian Citizen Science Conference 2023 — PoS (ACSC2023). Sissa Medialab. DOI: 10.22323/1.442.0010
  17. Meyer, K., Meneweger, T. and Mara, M. (2024) “A song all about AI”: a musical exhibition booth to foster artificial intelligence literacy. In: Proceedings of ECSA2024 and Austrian Citizen Science Conference 2024. ARPHA Proceedings 6. DOI: 10.3897/ap.e126144
  18. Newcomer, K. E., Hatry, H. P. and Wholey, J. S. (2015) Handbook of Practical Program Evaluation. Wiley. DOI: 10.1002/9781119171386
  19. Passani, A., Janssen, A., Hölscher, K. and Di Lisio, G. (2022) A participatory, multidimensional and modular impact assessment methodology for citizen science projects. fteval Journal for Research and Technology Policy Evaluation, vol. 54, pp. 33–42. DOI: 10.22163/fteval.2022.569
  20. Pedersen, D. B., Grønvad, J. F. and Hvidtfeldt, R. (2020) Methods for mapping the impact of social sciences and humanities—A literature review. Research Evaluation, 29(1), pp. 4–21. DOI: 10.1093/reseval/rvz033
  21. Reale, E., Avramov, D., Canhial, K., Donovan, C., Flecha, R., Holm, P. and Larkin, C., et al. (2018) A review of literature on evaluating the scientific, social and political impact of social sciences and humanities research. Research Evaluation, 27(4), pp. 298–308. DOI: 10.1093/reseval/rvx025
  22. Reed, M.S. (2025) The Research Impact Handbook. London, UK: Routledge. DOI: 10.4324/9781003615019
  23. Reed, M.S., Bryce, R. and Machen, R. (2018) Pathways to policy impact: a new approach for planning and evidencing research impact. Evidence & Policy, 14(3), pp. 431–458. DOI: 10.1332/174426418X15326967547242
  24. Reed, M.S., Ferré, M., Martin-Ortega, J., Blanche, R., Lawford-Rolfe, R., Dallimer, M. and Holden, J. (2021) Evaluating impact from research: A methodological framework. Research Policy, 50(4). DOI: 10.1016/j.respol.2020.104147
  25. Robopsychology Lab (2023) How to Explain AI. Available at https://www.jku.at/en/lit-robopsychology-lab/research/projects/how-to-explain-ai [Last accessed 10 August, 2026].
  26. Schäfer, M., Bührer-Topcu, S., Ehnert, F., Graf, V., Haus, J., Höhener, O., Holtmann, F., et al. (2025) Societal Impact of Transdisciplinary and Participatory research: A Guide for Assessment (GTPF Working Paper No. 2, English Translation). Zenodo. https://zenodo.org/records/17937353
  27. Schrögel, P. (2025) Ein Leitfaden für Partizipation in der Forschung. PartWiss. DOI: 10.5281/zenodo.14786253
  28. Shirk, J., Ballard, H., Wilderman, C., Phillips, T., Wiggins, A., Jordan, R., McCallie, E., et al. (2012) Public Participation in Scientific Research: a Framework for Deliberate Design. Ecology and Society, 17(2). DOI: 10.5751/ES-04705-170229
  29. Smith, K.E., Bandola-Gill, J., Meer, N., Stewart, E. and Watermeyer, R. (2020) The Impact Agenda: Controversies, consequences and challenges, Bristol University Press. DOI: 10.46692/9781447339861
  30. Staley, K. (2015) ‘Is it worth doing?’ Measuring the impact of patient and public involvement in research. Research involvement and engagement, 1, p. 6. DOI: 10.1186/s40900-015-0008-5
  31. Vaughn, L.M. and Jacquez, F. (2020) Participatory research methods – choice points in the research process. Journal of Participatory Research Methods, 1(1). DOI: 10.35844/001c.13244
  32. Wailzer, M. and Soyer, L. (2022) Co-Developing an impact model for evaluating the societal impact of participatory research approaches. fteval Journal for Research and Technology Policy Evaluation, 53, pp. 107–117. DOI: 10.22163/fteval.2022.547
  33. Wehn, U., Gharesifard, M., Ceccaroni, L., Joyce, H., Ajates, R., Woods, S., Bilbao, A., et al. (2021) Impact assessment of citizen science: state of the art and guiding principles for a consolidated approach. Sustainability Science, 16, pp. 1683–1699. DOI: 10.1007/s11625-021-00959-2
  34. Wiek, A., Talwar, S., O’Shea, M. and Robinson, J. (2014) Toward a methodological scheme for capturing societal effects of participatory sustainability research. Research Evaluation, 23(2), pp. 117–132. DOI: 10.1093/reseval/rvt031
DOI: https://doi.org/10.5334/cstp.983 | Journal eISSN: 2057-4991
Language: English
Page range: 43 - 43
Submitted on: Feb 13, 2026
Accepted on: Sep 28, 2026
Published on: Nov 12, 2026
Published by: Ubiquity Press
In partnership with: Paradigm Publishing Services

© 2026 Kathrin Meyer, Lara Bauer, Mathieu Mahve-Beydokhti, Martina Mara, published by Ubiquity Press
This work is licensed under the Creative Commons Attribution 4.0 License.