Mental Health Service Preparedness during a Pandemic: A Delphi Study in Slovenia
By: Špela Selak, Nuša Crnkovič, Janja Horvat and Irena Makivić
References
- Bower M, Smout S, Donohoe-Bales A, O'Dean S, Teesson L, Boyle J, et al. A hidden pandemic? An umbrella review of global evidence on mental health in the time of COVID-19. Front Psychiatry. 2023;14:1107560. doi: 10.3389/fpsyt.2023.1107560.
- Duden GS, Gersdorf S, Stengler K. Global impact of the COVID-19 pandemic on mental health services: A systematic review. J Psychiatr Res. 2022;154:354–377. doi: 10.1016/j.jpsychires.2022.08.013.
- So S, Wang TQ, Yu BE, Malvankar-Mehta MS. The psychological impact of the COVID-19 pandemic on frontline healthcare workers. A systematic review and a meta-analysis. EJMH. 2023;18:1–22. doi: 10.5708/EJMH.18.2023.0003.
- Vinko M, Mikolič P, Roškar S, Jeriček Klanšček H. Positive mental health in Slovenia before and during the COVID-19 pandemic. Front Public Health. 2022;10:963545. doi: 10.3389/fpubh.2022.963545.
- Matić T, Pregelj P, Sadikov A, Rus Prelog P. Depression, anxiety, stress, and suicidality levels in young adults increased two years into the COVID-19 pandemic. IJERPH. 2022;20(1):339. doi: 10.3390/ijerph20010339.
- Gabrovec B, Selak Š, Crnkovič N, Cesar K, Šorgo A. Raziskava o doživljanju epidemije Covid-19 med študenti: Poročilo o opravljenjeni raziskavi [Internet]. Ljubljana: Nacionalni inštitut za javno zdravje; 2021 [cited 2025 Jan 17]. Available at:
https://nijz.si/wp-content/uploads/2021/04/splosno_porocilo_o_opravljeni_raziskavi_-_raziskava_o_dozivljanju_epidemije_covid-19_med_studenti.pdf - World Health Organization. COVID-19 strategic preparedness and response plan - operational planning guideline [Internet]. Geneva: World Health Organization; 2021. [cited 2025 Jan 17]. Available at:
https://www.who.int/publications/m/item/covid-19-strategic-preparedness-and-response-plan-operational-planning-guideline - World Health Organization. Mental health impacts of COVID-19 across the WHO European region and associated opportunities for action: Briefing [Internet]. Geneva: World Health Organization; 2022 [cited 2025 Jan 17]. Available at:
https://www.who.int/europe/publications/i/item/WHO-EURO-2022-6108-45873-66068 - World Health Organization, Regional Office for Europe. Strengthening the health systems response to COVID-19: Adapting primary health care services to more effectively address COVID-19 [Internet]. Copenhagen: World Health Organization, Regional Office for Europe; 2020 [cited 2025 Jan 17]. Available at:
https://apps.who.int/iris/bitstream/handle/10665/332783/WHO-EURO-2020-727-40462-54321-eng.pdf - World Health Organization. World mental health report: Transforming mental health for all [Internet]. Geneva: World Health Organization; 2022 [cited 2025 Jan 17]. Available at:
https://www.who.int/publications/i/item/9789240049338 - World Health Organization, Regional Office for Europe. WHO European framework for action on mental health 2021–2025. Copenhagen: World Health Organization, Regional Office for Europe; 2022.
- The Organization for Economic Cooperation and Development. Tackling the mental health impact of the COVID-19 crisis: An integrated, whole-of-society response [Internet]. The Organization for Economic Cooperation and Development; 2021 [cited 2025 Jan 17]. Available at:
https://campusmentalhealth.ca/wp-content/uploads/2021/06/Tackling-the-mental-health-impact-of-the-COVID-19-crisis-An-integrated-whole-of-society-response.pdf - Rojnic Kuzman M, Vahip S, Fiorillo A, Beezhold J, Pinto da Costa M, Skugarevsky O, et al. Mental health services during the first wave of the COVID-19 pandemic in Europe: Results from the EPA Ambassadors Survey and implications for clinical practice. Eur Psychiatry. 2021;64(1):e41. doi: 10.1192/j.eurpsy.2021.2215.
- European Public Health Alliance. Mental health in the eye of the COVID-19 hurricane [Internet]. 2020 [cited 2025 Jan 17]. Available at:
https://www.epr.eu/mental-health-in-the-eye-of-the-covid-19-hurricane/ - Moreno C, Wykes T, Galderisi S, Nordentoft M, Crossley N, Jones N, et al. How mental health care should change as a consequence of the COVID-19 pandemic. Lancet Psychiatry. 2020;7(9):813–824. doi: 10.1016/S2215-0366(20)30307-2.
- Pearson CM, Mitroff II. From crisis prone to crisis prepared: A framework for crisis management. AMP. 1993;7(1):48–59. doi: 10.5465/ame.1993.9409142058.
- Mitroff II, Shrivastava P, Udwadia FE. Effective crisis management. AMP. 1987;1(4):283–292. doi: 10.5465/ame.1987.4275639.
- World Health Organization. Building health system resilience to public health challenges: Guidence for implementation in countries [Internet]. Geneva: World Health Organization; 2024 [cited 2025 Jan 17]. Available at:
https://iris.who.int/server/api/core/bitstreams/70b880e5-ccf1-4b59-a6cd-57d7ecfe017b/content - Vinko M, Roškar S, Novak Šubara T, Tančič Grum T. Local mental health support in Slovenia during COVID-19: Setting up primary health care helplines [Internet]. Behaviour & Addiction, Childhood & Adolescence. 2021;15 [cited 2025 Jan 17]. Available at:
https://eurohealthnet-magazine.eu/local-mental-health-support-in-slovenia-during-covid-19-setting-up-primary-health-care-helplines/ - National Institute of Public Health. Operativna skupina za izvajanje psihološke pomoči v razmerah epidemije oziroma pandemije: Končno poročilo o izvedenih aktivnostih, povezanih s covidom-19 [Internet]. Ljubljana: National Institute of Public Health; 2021 [cited 2025 Jan 17]. Available at:
https://nijz.si/zivljenjski-slog/dusevno-zdravje/koncnoporocilo-operativne-skupine-za-izvajanje-psiholoske-pomoci-medepidemijo-covida-19/ - Pujolar G, Oliver-Anglès A, Vargas I, Vázquez ML. Changes in access to health services during the COVID-19 pandemic: A scoping review. IJERPH. 2022;19(3):1749. doi: 10.3390/ijerph19031749.
- Odredba o začasnih ukrepih na področju organizacije zdravstvene dejavnosti zaradi obvladovanja nalezljive bolezni COVID-19 [Internet]. Uradni List RS. 164/20, 168/20, 171/20, 173/20, 204/20, 8/21. 2020 [cited 2025 Jan 17]. Available at:
http://www.pisrs.si/Pis.web/pregledPredpisa?id=ODRE2605 - Trevelyan EG, Robinson PN. Delphi methodology in health research: How to do it? Eur J Integrat Med. 2015;7(4):423–428. doi: 10.1016/j.eujim.2015.07.002.
- Niederberger M, Spranger J. Delphi technique in health sciences: A map. Front Public Health. 2020;8:457. doi: 10.3389/fpubh.2020.00457.
- Adler M, ed. Gazing into the Oracle: The Delphi method and its application to social policy and public health. Kingsley; 1996.
- Hallowell MR, Gambatese JA. Qualitative research: Application of the Delphi method to CEM research. J Constr Eng Manage. 2010;136(1):99–107. doi: 10.1061/(ASCE)CO.1943-7862.0000137.
- Akins RB, Tolson H, Cole BR. Stability of response characteristics of a Delphi panel: Application of bootstrap data expansion. BMC Med Res Methodol. 2005;5(1):37. doi: 10.1186/1471-2288-5-37.
- Makivić I, Švab V, Selak Š. Mental health needs assessment during the COVID-19 pandemic: Consensus based on Delphi study. Front Public Health. 2021;9:732539. doi: 10.3389/fpubh.2021.732539.
- Selak Š, Gabrovec B, Cesar K, Horvat J, Crnkovič N. Organizacija in delovanje služb s področja duševnega zdravja v okviru zdravstvenega varstva v Sloveniji v času epidemije covida-19: Znanstvena monografija [Internet]. Ljubljana: Nacionalni inštitut za javno zdravje; 2023 [cited 2025 Jan 17]. Available at:
https://nijz.si/wp-content/uploads/2024/02/organizacija-in-delovanje-sluzb-s-podrocjadusevnega-zdravja-v-okviru-zdravstvenega-varstva-v-sloveniji-v-casucovida-19.pdf - Diamond IR, Grant RC, Feldman BM, Pencharz PB, Ling SC, Moore AM, et al. Defining consensus: A systematic review recommends methodologic criteria for reporting of Delphi studies. J Clin Epidemiol. 2014;67(4):401–409. doi: 10.1016/j.jclinepi.2013.12.002.
- Hasson F, Keeney S, McKenna H. Research guidelines for the Delphi survey technique. J Adv Nurs. 2000;32(4):1008–1015. doi: 10.1046/j.1365-2648.2000.t01-1-01567.x.
- World Health Organization, Regional Office for Europe. Looking back, looking forward: Rapid assessment of the mental health system in Slovenia [Internet]. Copenhagen: World Health Organization, Regional Office for Europe; 2020 [cited 2025 Jan 17]. Available at:
https://www.zadusevnozdravje.si/wp-content/uploads/2021/04/WHO-mission-report-on-mental-health-in-Slovenia-2020-1.pdf - Ghahramani S, Kasraei H, Hayati R, Tabrizi R, Marzaleh MA. Health care workers' mental health in the face of COVID-19: A systematic review and meta-analysis. Int J Psychiatry Clin Pract. 2023;27(2):208–217. doi: 10.1080/13651501.2022.2101927.
- Umbetkulova S, Kanderzhanova A, Foster F, Stolyarova V, Cobb-Zygadlo D. Mental health changes in healthcare workers during COVID-19 pandemic: A systematic review of longitudinal studies. Eval Health Prof. 2024;47(1):11–20. doi: 10.1177/01632787231165076.
- Witteveen AB, Young S, Cuijpers P, Ayuso-Mateos JL, Barbui C, Bertolini F, et al. Remote mental health care interventions during the COVID-19 pandemic: An umbrella review. Behav Res Ther. 2022;159:104226. doi: 10.1016/j.brat.2022.104226.
- OMRA. Kaj je OMRA? [Internet]. [cited 2025 Jan 17]. Available at:
https://www.omra.si/kaj-je-omra/o-programu/ - Torous J, Jän Myrick K, Rauseo-Ricupero N, Firth J. Digital mental health and COVID-19: Using technology today to accelerate the curve on access and quality tomorrow. JMIR Ment Health. 2020;7(3):e18848. doi: 10.2196/18848.
- World Health Organization. Global strategy on digital health 2020–2025. 1st ed. Geneva: World Health Organization; 2021.
- Spanakis P, Peckham E, Mathers A, Shiers D, Gilbody S. The digital divide: Amplifying health inequalities for people with severe mental illness in the time of COVID-19. Br J Psychiatry. 2021;219(4):529–531. doi: 10.1192/bjp.2021.56.
- Agumba JN, Musonda I. Experience inusing the Delphi method in construction healthand safety research. In: 2013.
- Nasa P, Jain R, Juneja D. Delphi methodology in healthcare research: How to decide its appropriateness. WJM. 2021;11(4):116–129. doi: 10.5662/wjm.v11.i4.116.
- Shang Z. Use of Delphi in health sciences research: A narrative review. Medicine. 2023;102(7):e32829. doi: 10.1097/MD.0000000000032829.
- D'Agostino A, Demartini B, Cavallotti S, Gambini O. Mental health services in Italy during the COVID-19 outbreak. The Lancet Psychiatry. 2020;7(5):385–387. doi: 10.1016/S2215-0366(20)30133-4.
- Montes JM, Hernández-Huerta D. Impact of the COVID-19 pandemic on acute inpatient psychiatric units in Spain. Psychiatry Res. 2021;304:114136. doi: 10.1016/j.psychres.2021.114136.
Language: English
Page range: 83 - 93
Submitted on: Jun 12, 2025
Accepted on: Apr 10, 2026
Published on: Jun 1, 2026
Published by: National Institute of Public Health, Slovenia
In partnership with: Paradigm Publishing Services
Publication frequency: 4 issues per year
Related subjects:
© 2026 Špela Selak, Nuša Crnkovič, Janja Horvat, Irena Makivić, published by National Institute of Public Health, Slovenia
This work is licensed under the Creative Commons Attribution 4.0 License.