Mental Health Service Preparedness during a Pandemic: A Delphi Study in Slovenia

This study utilised the Delphi method, an iterative process of anonymous expert consultation using a structured set of questions and controlled feedback to achieve consensus (23). The method is commonly used in contexts characterised by insufficient knowledge, uncertainty, or a lack of evidence, particularly in healthcare (24, 25). A modified online Delphi method, combining qualitative and quantitative approaches, was employed (hereafter referred to as the Delphi study).

Participants were selected using non-random purposive sampling based on predefined criteria related to professional roles in mental health, including professional background, level of care, mental health service type, and target population. The Delphi panel was designed to ensure a consistent level of expertise while capturing heterogeneity across key dimensions of the Slovenian mental health system.

All invited participants were experienced professionals involved in decision-making or service provision, and all agreed to participate. The final sample was considered sufficient to capture perspectives across levels of care and service types. Regional diversity was considered, while specific locations were concealed to ensure anonymity.

Eleven experts (6 men and 5 women) participated, consistent with recommendations for Delphi panels of 8–12 participants (26). Given the highly specialised nature of the expert field, a smaller sample was considered appropriate to provide reliable and in-depth expert input (27). Participant characteristics are presented in Table 1. The response rate across all rounds was 100%.

Participation in the Delphi study was voluntary, and all participants were informed about the purpose and procedures of the study before taking part. Responses were collected anonymously and handled confidentially. Informed consent was implied through participation and completion of all Delphi rounds.

The proposed activities were developed based on: 1) a previous Slovene Delphi study on mental health needs assessment during the COVID-19 pandemic (28), 2) analyses of established national and international activities during the pandemic (29), 3) an analysis of national directions for mental health service organisation during the pandemic, and 4) research on mental health service functioning in Slovenia during the pandemic (29).

A total of 54 activities were identified across 7 measures or thematic categories: systemic, general, promotion and prevention, treatment, work organisation, education, and telemedicine. Activities were structured according to 2 complementary dimensions: implementation timing (before onset, at onset, and during a pandemic) and thematic categories. These distinguished between system-strengthening measures, which aim to improve baseline system capacity and resilience, and crisis-specific measures, which are activated in response to emergency conditions.

This Delphi study on mental health services during a pandemic was conducted via e-mail and consisted of 4 rounds (Figure 1). In the first round, experts reviewed, added, and modified the proposed activities, and identified vulnerable groups and key stakeholders in the mental health services network to provide contextual mapping of relevant actors and population groups. Responses were analysed using qualitative content analysis, and activities were refined and structured for the second round.

Figure 1.

Overview of the Delphi study procedure.

In the second round, participants rated the importance of each activity on a 5-point Likert scale (1 – entirely unimportant, 2 – unimportant, 3 – neither important nor unimportant, 4 – important, 5 – very important). Quantitative data were analysed using descriptive statistics. For each activity, mode and mean scores were calculated, and the level of agreement was expressed as the percentage of participants assigning higher importance ratings. Consensus was defined as at least 70% of panel members rating an item as important (score 4 or 5), corresponding to agreement by at least 8 of 11 experts, a threshold commonly used in Delphi research (30, 31). Sensitivity analysis was conducted using higher thresholds (75% and 80%). Activities meeting consensus threshold were retained.

In the third round, activities that had not reached consensus were reassessed. Participants were provided with their previous ratings and the group modal values from the second round. Consensus was achieved in this round, and activities that did not reach the 70% consensus threshold were excluded. Inter-round stability of ratings was assessed using the interquartile range (IQR) across round two and round three, with smaller values indicating greater convergence of responses.

In the fourth round, activities were classified according to implementation timing using a 3-point scale (1 – should be implemented immediately – before onset; 2 – should be implemented at the onset of a pandemic; 3 – should be implemented during a pandemic according to identified needs). Final categorisation was based on modal responses, appropriate for ordinal data given the prior convergence of ratings.

The final results are presented as a list of activities structured across 3 implementation phases and ranked according to modal expert ratings of importance.

In this study, the term “recommendations” refers to expert-derived proposals for strengthening mental health service preparedness and response, rather than formal national clinical guidelines. Recommendations were conceptually grouped into policy- and system-level measures, organisational procedures, and frontline service activities, and presented according to thematic categories and implementation phases (before onset, at onset, and during a pandemic).

The study aimed to develop expert-based recommendations for mental health services in the context of a pandemic by translating established resilience and crisis management frameworks into context-specific operational activities through a structured Delphi process. Although findings reflect the Slovenian context, the approach may inform preparedness planning in comparable healthcare systems by helping to identify system-specific gaps and priorities. Experts identified 47 activities for implementation. The results indicated that almost half of the proposed activities should be implemented before a pandemic and a similar proportion at its onset. This distribution highlights the importance of proactive system strengthening alongside crisis response, consistent with WHO's emphasis on anticipatory resilience-building (18).

Activities identified for immediate implementation were grouped into systemic, general, promotion and prevention, education, treatment, and telemedicine measures. Systemic measures reflect pre-existing structural barriers affecting the efficiency and accessibility of mental health services (28, 32), and align with WHO guidance (7, 8) supporting long-term system strengthening. The shift towards strengthening mental health within primary care is also consistent with international recommendations (9, 15) and supports a more sustainable system. Importantly, these measures reflect structural capacity-building rather than temporary crisis responses. General, education, and telemedicine measures are also consistent with these recommendations (8, 13).

Activities proposed for implementation before pandemic onset align with the initial phases of WHO's resilience framework (18) and Mitroff's crisis management model (16), emphasising prioritisation, preparedness, and early response capacity. Measures such as professional training and the development of clinical guidelines and protocols enhance organisational preparedness and adaptive capacity. These activities may therefore be understood as mechanisms for strengthening adaptive capacity within the mental health system. This is further supported by promotion, prevention, and treatment measures, including continuity of outpatient and community-based care, psychoeducational activities for vulnerable groups, and improved coordination across service levels.

At pandemic onset, systemic and general measures reflect the need for strategic planning and resource mobilisation, consistent with WHO's resilience framework (18) and previous findings of insufficient preparedness (2, 21). Work organisation measures emphasise the protection of healthcare workers, whose increased psychological burden during the COVID-19 pandemic has been well documented (3, 33, 34), highlighting the interdependence between workforce wellbeing and service continuity, a core component of system resilience recognised by WHO (18). Treatment measures focus on maintaining continuity of care, in line with recommendations made by the WHO Technical Advisory Group (8), as service disruptions have been associated with worsening mental health outcomes (2, 35). Participating experts emphasised promotion, prevention, and psychoeducation for the general population, particularly for vulnerable groups identified in this study. In the Slovenia context, existing programmes such as the OMRA programme (36) support baseline health literacy, with additional activities further strengthening awareness and resilience. The emphasis on educational activities for healthcare workers reflects their increased psychosocial burden, particularly among frontline staff. However, they were not explicitly identified as a distinct vulnerable group in the Delphi consensus process, suggesting potential underrepresentation despite strong evidence from COVID-19 literature. These measures aim to contain the impact of the crisis, aligning with crisis management models and supporting system resilience (16, 18), and reinforce preparedness as a continuous, multi-phase process.

The remaining activities focus on accessibility and optimisation of mental health services during a pandemic, including the gradual transition to hybrid and telemedicine delivery of preventive and treatment services, supporting resilience and mitigating the impact of the crisis (16, 18). Among the identified recommendation domains, telemedicine represents a particularly transformative component due to its implications for service delivery and equity. While it supports continuity of care during public health emergencies, its implementation must consider the potential impact of the digital divide (37, 38). Older adults, rural populations, people with low socioeconomic status, and people with severe mental illness may face barriers related to digital literacy, access to devices, internet connectivity, or cognitive and functional limitations (37,38,39). Without targeted support, telepsychiatry and remote services may exacerbate inequalities in access to mental health care (37, 39). These findings highlight that digital innovation should be embedded within equity-oriented health policies rather than implemented as a purely technological solution. Preparedness planning should therefore include strategies to address digital accessibility and literacy before future crises, in line with the WHO Global Strategy on Digital Health (38). This may include digital literacy programmes, alternative access pathways, and monitoring of equity-related indicators in telehealth use.

Although the proposed measures reflect expert consensus and are aligned with established crisis management and health system resilience frameworks, their effectiveness has not been empirically tested. The recommendations should therefore be interpreted as consensus-based guidance that requires further validation through implementation and evaluation.

In the Slovenian context, surge mental health workforce capacity could be mobilised through temporary redeployment from community mental health centres, health promotion centres, screening programmes, developmental outpatient services, rehabilitation institutes, and other temporarily suspended non-acute services, as well as through coordination with private or contracted providers, non-governmental organisations, and reallocation across different levels of care. Such mobilisation requires predefined protocols, rapid training, and safeguards to maintain continuity of essential services. Stakeholder mapping from the Delphi process clarifies roles and responsibilities across governance levels and supports coordinated implementation of preparedness activities, in line with WHO guidance emphasising clearly defined governance structures, inter-institutional coordination mechanisms, and predefined chains of responsibility at national and regional levels (18). The identification of vulnerable groups and stakeholders should therefore be understood as a contextual mapping exercise rather than as an indicator of differential weighting or causal influence on specific activities.

Although the Delphi process focused on preparedness and response phases, resilience frameworks also emphasise the importance of post-crisis evaluation and learning. These aspects were not explicitly addressed through the expert consensus process and therefore represent an area for further development. Future work should incorporate a post-pandemic evaluation and learning phase, supported by predefined indicators to monitor system performance and inform system-level learning, such as service utilisation, continuity of care, workforce capacity, telemedicine uptake, and equity of access across regions and population groups. Embedding such indicators into routine monitoring could support a shift from reactive crisis response to adaptive learning systems.

The Delphi process primarily engaged mental health professionals; however, the proposed measures reflect a two-pronged approach to pandemic preparedness by addressing both the safety and working conditions of staff and the access to, and the safety of, people receiving care. Measures such as protective equipment, safe working conditions, and telemedicine support infection control while maintaining continuity of care, highlighting their interdependence within resilient health systems.

The study did not include a formal ranking or weighting of activities beyond the consensus threshold, as the aim was to identify core preparedness domains and structure them according to implementation timing rather than to establish a fixed hierarchy of importance. Prioritisation may therefore need to be adapted to national contexts and resource availability. Future research could incorporate structured prioritisation or cost-effectiveness approaches.

Although the Delphi panel was relatively small, it included experts from diverse service settings and levels of care. In the Slovenian context, where the pool of national-level mental health experts is limited, full participation (100% response rate) across all rounds strengthens the robustness of the findings. While the small sample may have limited the range of perspectives captured, the panel size is consistent with Delphi literature (40, 41), and the high level of participant expertise further supports the reliability of the consensus (42).

Several limitations should be acknowledged. The study does not explicitly address post-crisis evaluation and learning, as emphasised in crisis management (16, 17) and resilience frameworks (18). In addition, mental health service users were not involved, and the findings are therefore based solely on expert perspectives, which may introduce bias, as people receiving care could provide additional insights into service organisation and delivery.

Delphi methodology is based on expert consensus, reflecting shared professional judgment rather than proof of effectiveness. The proposed recommendations should therefore be interpreted as informed, expert-based, guidance requiring further validation through real-world implementation and evaluation studies. The proposed recommendations have not yet been empirically implemented or formally evaluated, and their real-world effectiveness and feasibility therefore remain unknown. Future research and practical implementation should incorporate systematic evaluation and feedback mechanisms to support continuous improvement in the organisation of mental health services during public health emergencies.