Introduction
Over the last decades, the mental health of employees has been a growing concern due to increasing numbers of absenteeism or reduced work performance in the Western world (OECD 2015). Research shows overall moderate, yet positive evidence for the value of workplace interventions to prevent or reduce negative work outcomes among employees with common mental health problems (CMHPs) (Pomaki et al. 2012; Wagner et al. 2016). However, workplace stakeholders, especially supervisors or line managers, feel unsure about how, when, why and through what strategies to promote work participation (e.g., staying at work, work performance) (Janssens et al. 2021; Wagner et al. 2016). To illustrate, 40% of a representative panel of Dutch employers reported not knowing how to support employees with CMHP to continue working (Janssens et al. 2021). This is particularly challenging in prevention, where practical guidelines on increasing skills to promote work participation are scarce (Jetha et al. 2021; Nexo et al. 2018). Furthermore, in case of CMHPs, such as depression, anxiety or stress, symptoms often develop slowly and saliently, and employees fear disclosure due to stigma (Brouwers et al. 2020; van Beukering et al. 2021). This adds to the complexity of early detection of problems and offers adequate solutions to promote work participation outcomes such as stay at work and work performance.
Supervisors have shown to be key workplace stakeholders to signal and address (the onset of) behavioural changes in the workplace due to mental health problems (Nielsen 2013; Oomens et al. 2010; Sorensen et al. 2020). Psychosocial work exposures, including (lack of) supervisor support, are often considered as (one of) the causes of CMHP, and at the same time the solution to enhance mental health and work participation (Niedhammer et al. 2021). A supportive supervisor can facilitate employees with CMHP, for example, by offering job accommodations or time off work for psychological treatment (Haveraaen et al. 2015; Joosen et al. 2021).
Supervisor’s supportive behaviour (promoting work participation) as a concept
We consider the supportive behaviour of supervisors as a concept, enhancing employees with CMHP to stay and thrive at work. Supportive behaviour of supervisors to promote work participation for employees with mental health problems is a rather new concept in this field. Learning from return to work practices, effective communication, ability to manage privacy and disclose reasons for sickness absence to colleagues, ability to establish trust, being understanding and approachable, and ability to develop work adjustments were important during phases of return to work for employees with CMHP (Nielsen & Yarker 2023). In another study on work participation with CMHP it was found that showing trustful, low-key availability and empathy by supervisors facilitates staying at work, as well as offering adequate, timely support and job accommodations (van Hees et al. 2022b). Considering the behavioural traits given here, to increase supportive behaviour, we base this on a behavioural theory, the Theory of Planned behaviour (Fishbein & Ajzen 2010). This theory places intention as the most proximal predictor of behavioural performance, predicted by attitude, social influence and self-efficacy, emphasising the motivational processes underlying supportive behaviours. Self-efficacy, also conceptualized by Bandura’s social cognitive theory (Bandura 1997) refers to people’s confidence in their ability to perform specific activities, influencing how people think, feel, and act (Liu et al. 2014). Reflecting on the results of the needs assessment used to develop this intervention (van Hees et al. 2022c), those behavioural determinants match well. It seems important to have positive attitudes and social influences towards mental health at the workplace and that supervisors believe in themselves (self-efficacy) to signal and take preventive actions (Hultqvist et al. 2023). Besides, supervisors may need to increase skills on how to deal with problems at work due to CMHP. As such, both motivational processes and skills development should be addressed as mechanisms of change in preventive interventions to increase supervisor support (Aust et al. 2023; Hultqvist et al. 2023; Liu et al. 2014; van Hees et al. 2022c).
The role of context in realising intervention outcomes
In practise, contexts are diverse and challenging to ‘capture’, while implementing organisational interventions (Nielsen & Noblet 2018; Pawson 2013). At the same time, contextual factors can hinder and facilitate change, both within the person and in the organisation. In research, it often remains unclear how environmental factors, such as the learning climate or social safety were targeted or evaluated in interventions. As a result, it is still challenging to establish general patterns of how combinations of organisational interventions and contexts produce outcomes (Abildgaard et al. 2020). However, in realist terms, the pre-existing context within which organizational interventions are implemented shapes which mechanisms are triggered and should therefore be considered (Dalkin et al. 2015; Greenhalgh & Manzano 2022). Differences in the context and implementation, for example, in a health promotion initiative, may lead to markedly different effects in the targeted workplaces (Nielsen et al. 2006). During the development study of this intervention, organizational contextual factors such as organizational support, a safe culture and knowing the OHP from previous (RTW) cases, deemed important factors influencing participation to our intervention (van Hees et al. 2022c). Also, enabling circumstances, such as some level of interest by supervisors in mental well-being, having had training on CMHP and having more work experience leads to more managerial preventive actions for those employees (van de Voort et al. 2019). The programme context should allow tailored implementation, where implementors can adjust materials to the needs of supervisors in their organisation (Noar et al. 2007).
The impact of interventions on supervisor support
Interventions to strengthen supervisory capacity and supportive behaviour may have a positive impact on work outcomes. However, there is still limited evidence on effective supervisory interventions and how they bring about positive or increased work outcomes (Wagner et al. 2015). Although workplace interventions have been shown promising in preventing mental health problems for employees (Bourbonnais et al. 2011; Gilbert-Ouimet et al. 2015), it is yet unknown what the effects of such interventions are on actual supervisors’ supportive behaviour (Perry et al. 2020). Preventive interventions that target supervisors’ behaviour consist of elements such as a behaviour oriented approach (Christensen et al. 2019; Hammer et al. 2019; Perry et al. 2020) and a participative problem solving approach (Bourbonnais et al. 2006). Therefore, it may be worth it to intervene on supervisor’s behaviour, being the outcome of this workplace intervention.
Research aim and questions
The overall aim of the intervention was to strengthen supportive behaviour by using knowledge and practical expertise from occupational health professionals (OHPs), experts in return-to-work trajectories, in a preventive phase while employees are still at work. The aim of this study is therefore to gain insight into the impact of a workplace intervention to strengthen supervisor’s supportive behaviour, and to unravel and understand how and why it works, for whom and under which circumstances. The intervention ‘Stay at Work-Supervisor Guideline (SAW-SG)’ consists of an online guideline and coaching sessions provided by OHPs on work participation for employees with CMHP. The following research questions were investigated:
Does this intervention work? What are the changes over time, regarding supportive behaviour of supervisors between before (t0) and post-intervention after three months (t1) and at follow-up after six months (t2), participating to the SAW-SG intervention?
How, why, for whom and under what circumstances does this intervention work? Investigating factors in the personal, organisational or intervention context and behavioural determinants as mechanisms of change (self-efficacy, social influence, attitude, intention and skills).
Methods
Mixed-methods realist evaluation
In this study a novel intervention for supervisors to promote work participation of employees with CMHP is evaluated using a mixed-methods realist design. To better understand the working mechanisms of workplace interventions, researchers call to further explore causal relations between intervention components, organizational factors, mechanisms and outcomes, in addition to measuring effects by randomised and quasi-experimental, controlled designs (Nielsen & Miraglia 2017). As the work environment and capacity of all stakeholders is highly context-specific, it is hard to ‘control’ for implementation quality and contextual factors (von Thiele Schwarz et al. 2017). In response to those challenges, we applied a realist evaluation approach to explore how mechanisms bring about the resulting outcomes in context-specific ways (Pawson et al. 2005). Box 1 provides the definition of realist terms. We used an explorative mixed-methods design, analysing qualitative data to explain and nuance the quantitative findings. A mixed-methods realist design was chosen, in order to triangulate results and provide a more in-depth evaluation of both the retrieved mechanisms (intervention resources and reasonings) and context (pre-existing personal and organisational context). Whether the intervention worked or not was first evaluated by quantitatively assessing changes over time. On how the intervention worked (mechanisms) and under what circumstances and for whom (context), quantitative factors were assessed first. Thereafter, context-mechanism, outcome (CMO) configurations were compared to results of the statistical analyses, to determine points of convergence, divergence or complementariness. This led to the refined programme theory.
Box 1. Definition of realist terms
Initial programme theory: the working hypotheses from the researchers who design and evaluate an intervention (Emmel et al. 2018)
Context: context refers to ‘something that enables or disables the current mechanism of interest’ (Emmel et al. 2018, p. 55). It often refers to the ‘setting’ or ‘pre-existing factors’ of programmes and research. As conditions change over time, the context may also reflect aspects of those changes while the programme is implemented.
Mechanisms: mechanisms are underlying, latent entities, processes or structures that lead to influence the outcome (Dalkin et al. 2015). This can refer to processes within the participant of an intervention or exposure (resources), their cognitive and emotional responses (reasonings), typically related to the intervention or exposure being offered.
Outcome: an outcome is what can be measured in terms of impact across the target population, using measurable or measured indicators. Outcomes can be considered as quantitative or qualitative, and intended or unintended (Jagosh et al. 2011).
CMO configuration: describes the causal links between context, mechanisms and outcome considered as causative explanations pertaining to the evidence on the topic of interest (Emmel et al. 2018).
Setting and selection of study participants
In the Netherlands, according to the Dutch Gatekeeper Improvement Act (Weel et al. 1999), the employer is responsible for the return to work trajectory of sick-listed workers during the first two years of sickness absence. During this time, employers are obliged to contract and consult occupational health services to support the return to work process, however not prior to sickness absence. OHPs who are educated as ‘labour experts’ in the Dutch social security system provide those services and could also provide preventive interventions. Labour experts act independently and are expert in matching the employee’s work capacities with the work environment. In our study, 23 OHPs were recruited by social media announcement and on the website of the national association of labour experts and large occupational health service practices. The OHP selected the participating employer organisations (van Hees et al. 2022c). Selection criteria for OHPs to be included in our study were to have 2 to 6 supervisors available to participate in the intervention. Most Dutch employers have contracted independently operating OHPs or contracted occupational health services. In our study, contracted OHPs, independently or through the occupational health service were included, or OHPs who were employed (and internal) by the participating organisation. Participation for supervisors contained attending the intervention activities, free access to the guideline and data collection activities, implying three questionnaires of 15 minutes (baseline, post-intervention and post-follow-up) and an optional interview of 30 min. Each participant in this manuscript has given written informed consent to participate to the study and to publish these case details. This study has been approved by the Ethical Review Board of Tilburg University, The Netherlands (RP423). This study is reported in accordance with the Rameses II reporting standards for realist evaluations (Wong et al. 2016).
Procedures
We developed the initial programme theory, referring to Table 1 (Jagosh 2020), using both deduction based on a theoretical model as middle range theory (Fishbein & Ajzen 2010) and induction based on results from previous studies and the pre-test (van Hees et al. 2022a, 2022b, 2022c). The formulation of programme theories about the intervention and its implementation strategies helped to ensure that those strategies have a greater likelihood of bringing about the intended outcomes (Nielsen 2022). At baseline, we validated the initial programme theory by interviewing supervisors, who participated in this study (n = 15). This helped us to specify contextual factors that (may) trigger mechanisms supposed to be necessary for the intervention to have impact. At post-follow-up, we again collected qualitative data by interviewing supervisors (n = 14). Questionnaires were distributed at baseline (t0), directly after the intervention at 3 months (t1) and post-follow-up, at 6 months (t2). The data of the study were collected in 2021.
Table 1
Initial programme theory.
| CONTEXT | MECHANISMS | OUTCOME |
|---|---|---|
| Motivation Supervisors are likely to reflect on their own approach and attitudes, skills and pitfalls/shortcomings through self-assessment, leading to increased motivation (awareness, motivation) And gain insights into their own performance that gives self-confidence or area for improvement (self-efficacy) | Improving their supportive behaviour, through supportive actions |
| Skills They reach out to the OHP expert directly, especially when they had positive experience with the OHP in the past and when they experience a trustful relationship with OHP They feel supported not being alone in this, sharing experiences with the expert and co-supervisors (social influence), that gives room for discussion but with a systematic, evidence-based tool, so they know the given actions on how to support from the guideline (practical skills) | Improving their supportive behaviour, through supportive actions |
[i] General circumstances: The intervention (guideline and coaching sessions) is implemented in circumstances where there is/are: (1) A safe culture for sparring with OHP, (2) not being assessed on sick leave numbers in job performance as supervisor, and/or (3) facilitated by organisation to invest in mental health of team members.
The SAW-SG intervention consisted of (1) the online, theory and practise-based guideline and (2) four coaching sessions to increase supervisor supportive behaviour (Blijven werken met psychische klachten (han.nl)) (van Hees et al. 2022c). The guideline provided the supervisor with suggested actions presented as five step-wise themes on how to promote work participation: (1) signal CMHP that affect the employee’s behaviour or work timely, (2) talk about impact of CMHP at work, (3) stimulate employee’s autonomy and sense of responsibility, (4) explore, facilitate and evaluate job accommodations to match work with employee’s needs and abilities, and (5) consultation of occupational health support to select tailored interventions. Additionally, the guideline included two main categories of basic environmental conditions for supervisors: (1) ways to know the employees well in their regular working behaviour and (2) ways to strengthen the team and organisation. The intervention was implemented by OHPs through coaching sessions with supervisors, either individually or in small groups (2–6 participants). These sessions focussed on the use of the online guideline, using current cases or dilemmas brought up by supervisors. Beforehand, OHPs were trained on the content of the guideline and implementation through coaching sessions. A training protocol and training materials were used to ensure implementation quality. The intervention took three months. Between 3 and 6 months there was a ‘follow-up period’ with optional additional coaching about this topic, with OHP being available for consultation on request of the supervisor.
Measures
Outcome
The outcome is considered self-reported supportive supervisor behaviour. This concept is operationalized as self-reported behaviour using an adapted version of related questionnaires (Corbiere et al. 2014; Ketelaar et al. 2017). Items were modified to the five themes in the guideline to fit the context of this study. An example is (theme 1): ‘Over the last three months, I timely signal psychological complaints’. Six items (Cronbach’s α: 0.81) were used, ratings were provided on a five-point Likert scale from totally agree to totally disagree. An average score ranging from 1 to 5 was calculated.
Behavioural determinants as mechanisms of change (quantitative)
Behavioural determinants were self-efficacy, intention, skills, attitude and social influence ranging from 1 to 5. Self-efficacy of the supervisor on how to support was measured using a scale with six items (Cronbach’s α: 0.84) adjusted from Ketelaar et al. (Ketelaar et al. 2017); intention to support employees with CMHP was measured by two items, using an adjusted version of Hendriksen et al. (2016; Cronbach’s α: 0.72); and skills items were based on the themes in the guideline adjusted from Ketelaar et al. (2017; four items, Cronbach’s α: 0.78). Reliability of the attitude scale and social influence scale were relatively low (respectively Cronbach’s α of 0.37 and 0.60) and single items scored very high at baseline. Therefore, we did not include these scales and single items in the further analyses.
Personal factors
Several factors were assessed to explore for whom and under what circumstances each mechanism is triggered, leading to the outcome of supportive behaviour, based on the initial programme theory. For personal factors, we measured the supervisor’s age, gender, sector of organisation, size of organisation, years of supervisory experience, educational level of employees in team, being assessed on sick leave number by their own manager, dealt before with CMHP in personal life or at work, and whether the supervisor followed a course on a related topic in the last 5 years.
Organizational factors
Organizational factors included the following standardised measurement tools: organizational support, assessed by six items measuring the experienced organizational support (Cronbach’s α 0.74; Hendriksen et al. 2016), learning climate as in being facilitated by the organisation (Cronbach’s α 0.83) and error avoidance by supervisors (Cronbach’s α 0.67; Nikolova et al. 2014).
Intervention factors
To unravel how the intervention worked and how it was implemented, items regarding the intervention factors were included in the questionnaire. The items were based on the initial programme theory and the findings of the pre-test of the intervention (van Hees et al. 2022c). It contains questions regarding the implementation process, previous collaboration between OHP and the supervisor, frequency of sparring with an OHP or colleague by the supervisor and the way coaching sessions were organised in groups or individually.
Statistical analysis
First, descriptive statistics were used to summarise baseline characteristics of enrolled participants. Since all questions were categorised and compulsory, there are no missing and no outliers. Then, we tested difference scores of outcomes between baseline and post-intervention (t0–t1), and between baseline and post-follow-up (t0–t2), using paired sample-t tests (two tailed, including bootstrap). Correlations between the outcome and all covariates were computed, as well as attrition analyses (see Appendix 2). Covariates for the final regression model were selected in two steps. First, Pearson correlation coefficients were inspected in relation to the outcome. Second, we searched for mechanisms of change and triggering circumstances (personal, organizational, interventional factors) that may contribute to the changes over time, based on the initial programme theory and correlation coefficients. We conducted a multivariate backward regression analyses for the outcome (behaviour difference t0–t1 and t0–t2), to calculate the explained variance of the model and the significance of included predictors. Variables were assessed for collinearity and interaction. We choose the model with the best fit based on the highest explained variance, a significant F-ratio and significant coefficients. The statistical significance level was set at 0.05. All analyses were performed using SPSS version 27 (IBM Corp, Released 2011, IBM SPSS Statistics for Windows, Version 20.0. Armonk, NY). See Appendix 1 for the code book (right column) and Appendix 3 for the used syntax.
Qualitative data collection and analysis
Participating supervisors (n = 15) were selected at random before the start of the intervention and the same, but one, were interviewed at post-follow-up (n = 14). At baseline, interviews focussed on interviewees’ experience with workplace interventions and strategies how such an intervention could work, with the purpose to ‘refine’ the initial programme theory (Manzano 2016). Post-follow-up, interviews focussed on the experience with the SAW-SG intervention, and what in themselves (reasonings), in the intervention (intervention resources) and in their organisation or other factors (circumstances) lead to the outcomes. Two members of the research team analysed transcripts of baseline interviews by searching and refining CMO-configurations. Each element was inserted as ‘codes’ in the data analysis software (Dalkin et al. 2021). Post-follow-up, we analysed transcripts based on the encoded CMO-configurations, derived from the validated programme theory. By using the transcripts we confirmed or recoded CMO-configurations, using code groups and adjusted code names (Dalkin et al. 2021). ATLAS.ti was used for analysis.
Results
Study population
For baseline characteristics, see Table 2. Participating supervisors mainly worked at large organisations in health care or public administration sector. 62% were female supervisors, and most supervisors were between 40 and 59 years old. The total sample consisted of 99 supervisors, of which 92 supervisors completed the baseline questionnaire (t0, response rate 93%). After the intervention, 65 supervisors completed the questionnaire (t1: 66%, 27 missing) and post-follow-up 56 supervisors completed the third questionnaire (t2: 57%, 36 missing).
Table 2
Baseline characteristics of study population (n = 92).
| PERSONAL CHARACTERISTICS OF SUPERVISOR | N | % |
|---|---|---|
| Gender | ||
| Female | 57 | 62.0 |
| Male | 35 | 38.0 |
| Age | ||
| 18–29 years | 5 | 5.4 |
| 30–39 years | 20 | 21.7 |
| 40–49 years | 27 | 29.3 |
| 50–59 years | 34 | 37.0 |
| 60–67 years | 6 | 6.5 |
| Sector | ||
| Health care | 37 | 40.2 |
| Public administration | 21 | 22.8 |
| Business services | 13 | 14.1 |
| Other* | 21 | 22.9 |
| Size of organisation | ||
| Small (10–49) | 2 | 2.2 |
| Medium (50–249) | 16 | 17.4 |
| Large (250 and more) | 73 | 79.3 |
| Supervisory experience, in years | ||
| 0–1 | 7 | 7.6 |
| 2–5 | 26 | 28.3 |
| 6–10 | 18 | 19.6 |
| >10 | 41 | 44.6 |
| Number of supervised employees (M (range), SD) | 66 (4–700) | 93 |
| Educational level of supervised employees | ||
| Low | 9 | 9.8 |
| Medium | 42 | 45.7 |
| High | 41 | 44.6 |
| Dealt with CMHP personally | ||
| Not (rarely, never) | 46 | 50.0 |
| Yes (often, regularly, sometimes) | 46 | 50.0 |
| Dealt with CMHP at work | ||
| Not (sometimes, rarely, never) | 39 | 42.4 |
| Yes (often, regularly) | 53 | 57.6 |
| Being assessed on sick leave number | ||
| No | 40 | 43.5 |
| Yes | 52 | 56.5 |
| Followed course in last 5 year about this topic (at t1) | ||
| No | 23 | 25.0 |
| Yes | 42 | 45.7 |
| Organizational characteristics | ||
| Organizational support (neutral = 1 – totally agree = 3) (M, SD) | 2.18 | 0.59 |
| Learning climate: organisation facilitates learning (at t1, n = 65, M, SD) | 4.11 | 0.81 |
| Learning climate: error avoidance (at t1, n = 65, M, SD) | 2.63 | 0.76 |
| Intervention characteristic | ||
| Previous collaboration between OHP and supervisor | ||
| No | 50 | 54.3 |
| Yes | 42 | 45.7 |
| Frequency sparring with an OHP | ||
| Never/seldom | 10 | 10.9 |
| Regularly | 42 | 45.7 |
| Often/always | 40 | 43.5 |
| Coaching received in group/individually | ||
| Group | 39 | 42.4 |
| Individually | 24 | 26.1 |
| Both | 29 | 31.5 |
[i] *Agriculture, industry, construction, trade, transport, hospitality, IT, education, culture sports, and recreation.
How was this intervention implemented?
During this intervention study, 92 supervisors were coached by 23 OHPs in 2021. OHPs used video calling most of the time (71.4%), as the COVID-19 pandemic regulations prohibited face-to-face meetings in some conditions. Most of them held 2 to 3 sessions with the participating supervisor. Sessions were held either in small groups (31%), individually (26%) or both (35%). During the sessions, the OHPs asked supervisors to discuss cases of employees with CMHP who were still active in work, emphasising the importance of prevention and work accommodations rather than solely focussing on those on sick leave. Based on the online, evidence-based guideline, they encouraged proactive measures to address signals before they escalate. Strategies used during the coaching sessions were interactive (giving feedback, discussion, customized advice and guided practise on adjusted work by relating cases to the given suggestions in the guideline), as well as informative (providing written and verbal information, creating sense of urgency and success stories about mental health and organizational safety).
Does the SAW-SG intervention work?
Figure 1 shows the changes over time, using comparison of mean values for the programme outcome (behaviour), and behavioural determinants (self-efficacy, skills, intention) at baseline, post-intervention and post-follow-up. There was a significant increase of supportive behaviour between baseline and post-follow-up (6 months; p < 0.00) and not significant between baseline and post-intervention (3 months; p = 0.08). Regarding the behavioural determinants, there was a significant difference in mean scores of supervisor’s skills and self-efficacy over time between baseline and post-intervention (p < 0.00). Self-efficacy and skills showed also significant increases between baseline and post-follow-up (p < 0.00). Differences for intention were small and not statistically significant both post-intervention (p = 0.88) and post-follow-up (p = 0.19). Intention scored high at baseline as shown in Figure 1.
Figure 1
Changes over time, based on mean scores.
How, for whom and under what circumstances does this intervention work?
Quantitative results
From various covariates regarding personal factors, organizational factors, intervention aspects and behavioural determinants, only increased self-efficacy and previous collaboration between OHP and supervisor correlate significantly with the supportive behaviour of the supervisor (see Table 3). The overall multivariate regression model for changes over time directly after the intervention was statistically significant (R2 = 0.29, F(6.56) = 3.80, p = 0.00), explaining 28.9% of the variation in behaviour change. Supervisors being assessed on sick leave number (b-coefficient –0.33), previous collaboration between OHP and supervisor (b-coefficient –0.26) and with more increase in self-efficacy between baseline and post-intervention (b-coefficient 0.33) significantly predicted supportive behaviour (see Table 4). For post-follow-up, the overall regression model was also statistically significant (R2 = 0.27, F(6.41) = 2.57, p = 0.03), explaining 27.3% of the variation in change between baseline and post-follow-up. More increase in self-efficacy between baseline and post-follow-up (b-coefficient 0.29) and having followed a course on this topic (b-coefficient –0.38) as well as having dealt with CMHP personally (b-coefficient 0.34) were significant covariates predicting behavioural change at post-follow-up.
Table 3
Means of difference scores (standard deviations, confidence intervals t-value) and correlations post-intervention (t1, n = 65) and post-follow-up period (t2, n = 56) compared to baseline (t0).
| Mb | SD | CI | t | BEHAVIOUR t0–t1c | BEHAVIOUR t0–t2 | ||
|---|---|---|---|---|---|---|---|
| LOWER | UPPER | ||||||
| Outcomes | |||||||
| Behaviour t0–t1 | 0.15 | 0.68 | –0.02 | 0.32 | 1.77 | 1 | |
| Behaviour t0–t2 | 0.33** | 0.63 | 0.17 | 0.50 | 3.98 | 0.68** | 1 |
| Behavioural determinants | |||||||
| Self-efficacy t0–t1 | 0.36** | 0.50 | 0.23 | 0.49 | 5.73 | 0.27* | 0.20 |
| Self-efficacy t0–t2 | 0.42** | 0.54 | 0.28 | 0.56 | 5.84 | 0.25 | 0.34* |
| Skills t0–t1 | 0.29** | 0.43 | 0.18 | 0.40 | 5.37 | 0.19 | 0.13 |
| Skills t0–t2 | 0.41** | 0.48 | 0.28 | 0.54 | 6.33 | 0.20 | 0.22 |
| Intention t0–t1 | 0.01 | 0.42 | –0.10 | 0.11 | 0.15 | 0.14 | 0.10 |
| Intention t0–t2 | 0.07 | 0.40 | –0.04 | 0.18 | 1.34 | 0.24 | 0.09 |
| Personal characteristics | |||||||
| Dealt with CMHP personally | 0.15 | 0.13 | |||||
| Dealt with CMHP at work | –0.18 | –0.25 | |||||
| Being assessed on sick leave number | –0.22 | –0.14 | |||||
| Followed a course on the topic | 0.01 | –0.25 | |||||
| Organizational characteristics | |||||||
| Learning climate: individual error avoidance | 0.10 | 0.05 | |||||
| Organisational support | –0.22 | –0.17 | |||||
| Intervention characteristics | |||||||
| Previous collaboration between OHP and supervisor | 0.33** | 0.15 | |||||
[i] *Significant at the 0.05 level (two-tailed). **Significant at the 0.01 level (two-tailed).
aCompared difference with t0, bM, mean difference, cPearson Correlation, SD, standard deviation, CI, 95% confidence interval of the difference.
Correlations for all covariates included in the study can be found in Appendix 2.
Table 4
Results of stepwise backward regression analysis of supervisor characteristics and circumstances on change in supportive behaviour (n = 56).
| VARIABLE | BEHAVIOUR POST-INTERVENTION (t0–t1) | BEHAVIOUR POST-FOLLOW-UP (t0–t2) | ||||
|---|---|---|---|---|---|---|
| b-COEFFICIENT | t | STANDARDISED COEFFICIENTS (β) | b-COEFFICIENT | t | STANDARDISED COEFFICIENTS (β) | |
| (Constant)/intercept | 0.13 | 0.34 | 0.12 | 0.28 | ||
| Learning climate: error avoidance | 0.19 | 1.74 | 0.22 | 0.25 | 1.91 | 0.29 |
| Previous collaboration between OHP and supervisor | –0.35* | –2.26 | –0.26 | –0.20 | –1.12 | –0.15 |
| Self-efficacy t0–t1 difference | 0.43* | 2.71 | 0.32 | 0.38* | 2.09 | 0.29 |
| Followed course in last 5 years on topic | –0.04 | –0.19 | –0.02 | –0.53* | –2.40 | –0.38 |
| Being assessed on sick leave number | –0.44* | –2.68 | –0.33 | –0.28 | –1.34 | –0.20 |
| Dealt with CMHP personally | 0.32 | 1.97 | 0.24 | 0.45* | 2.26 | 0.34 |
[i] *Significant at p = 0.05.
Qualitative results
Derived from the qualitative data, we retrieved five mechanisms with their specific enabling circumstances that deemed evident for strengthening supervisory support according to the participants. We explain these mechanisms and circumstances that led to the outcomes of the SAW-SG intervention, operated on interpersonal and individual supervisor level. Each mechanism presents the intervention resource leading to the reasonings of participants. We also present two overall enabling conditional circumstances to the intervention. The presented mechanisms (intervention resources and reasonings), triggered by specific and overall conditional circumstances are depicted in Figure 2.
Figure 2
Enabling circumstances, triggering mechanisms (intervention resources and reasonings), leading to outcomes during the SAW-SG Intervention.
Mechanisms on the interpersonal level
Tailored and accessible expertise, leading to trust in OHP
Supervisors reported that because of the OHP’s tailored expertise (e.g., on mental health and ways to stay at work), supervisors received useful advice on their cases, increasing their skills and confidence in their role and insights into interventions to offer the employee. They appreciated that OHPs acted empathic and independent, using a non-judgemental approach during the coaching sessions. It was appreciated when the OHP tailored the (amount of) information from the guideline to current cases. Supervisors who previously collaborated with the OHP, and supervisors who received tailored advice to their specific work context had deeper discussions with OHP on their supportive behaviour, because they could trust the OHP and better understand circumstances. Also, supervisors appreciated the opportunity to participate individually or in small groups and being given allocated time to sit and reflect during the intervention. Because of the low-key accessibility of the OHP, initiated by this intervention set-up, interaction between supervisors and OHPs was intensified. For the supervisors who felt this low key access to external occupational health services or had the OHP internally available, it was easier to ask for support or advice.
I realized that when I need help on a difficult case, the OHP is the expert to advise me accordingly, while before I would ask HR. (Supervisor 2)
Coaching skills by the OHP, that is being independent, empathic and also critical, was more important than presenting the content of guideline. This I can study myself. (Supervisor 11)
Group learning leading to collective efficacy with co-supervisors
Supervisors were very motivated to discuss about Staying at work despite CMHP with the OHP as well as with co-supervisors in small group sessions. They experienced group learning while sharing information on the guideline and applying this to cases. They realised during the coaching sessions to not be the only one facing dilemma’s with employees with CMHP. This led to the unintended outcome of intensified interaction among them. Supervisors felt collective efficacy among co-supervisors (a group’s shared belief in its conjoint capabilities to organise and execute actions (Bandura 1997), but only if there was no hierarchy among the members in the group sessions, and if they had the time and regular scheduled moments to reflect on cases and their own supportive behaviour and activities.
Discussion with the OHP and other supervisors who know me quite well made me more aware of my blind spots. (Supervisor 6)
I feel that after the project, we should repeat what we discussed during this intervention in regular management meetings. (Supervisor 13)
Mechanisms on the individual level
Self-reflection: leading to awareness and willingness to improve
Supervisors mentioned that they became more aware and Willing to improve because they reflected upon themselves in this theme because the intervention gave them insights on urgency and their role. To feel this motivation, they needed at least some interest in mental well-being. For those supervisors who dealt with CMHP before (this could be in their personal life, education or work as supervisor) or for those who were yet aware of the urgency and the importance of their role, it was easier to reflect on their own behaviour. This seemed to lead to increased motivation, to support those employees, through the coaching or guideline.
This intervention made me more motivated in my role as supervisor, I enjoy guiding and supporting my subordinates even more. (Supervisor 4)
Getting confirmation by guidelines leading to self-efficacy
Supervisors reported the importance of getting confirmation on their performance, by applying the guideline on previous cases or current cases. Also, by previous success stories with employees who continued to work despite mental health issues, they felt confident (Self-efficacy) in themselves to be able to promote work participation through their support. In this, they reported it helped to know their own strengths and weaknesses and their used leadership style, for example, from courses in the past on this topic or through this intervention. During the coaching, supervisors self-assessed their practise with the given actions in the guideline and shared their vision with the OHP or other supervisors. Participating in the intervention also increased self-efficacy on how to converse with employees, especially on less visible struggles or problems at work due to CMHP.
Now I can trust that I can handle these difficult conversations because I saw the benefits of using the guideline, in myself and in the response of employees. (Supervisor 12)
Practicing given actions, leading to skills to act
Lastly, many participants addressed that they gained more skills on how to support employees, because this intervention gave them the skills to act, by practicing their conversing skills knowing what to say or not to say. Namely, by preparing or reflecting on their activities and by actively considering concrete alternatives throughout the process, from the guidelines or tailored by the OHP. Especially if supervisors could directly apply what they learnt during the coaching session to a current case, and if felt the responsibility to improve their role or change their managerial actions, it increased supportive behaviour. This was easier when not being occupied by a high caseload or span of control.
I scanned the guideline before I shared my concerns with the employee, to see what I could do or say differently. (Supervisor 14)
After the intervention, I more frequently have ‘coffee’ with my subordinates, just checking how things are going and asking also about the challenges, because I noticed it saves time and effort to invest more beforehand. (Supervisor 3)
Conditional circumstances
Overall, in order to bring about the reported changes in the supportive behaviour of supervisors, a safe learning climate and a facilitating organisation were essential conditional circumstances. Regarding a safe learning climate, participants expressed the importance that the organisation promoted trial and error of new actions managing the employee with CMHP, without assessment or judgement on (operational) performance indicators of supervisors or sick leave numbers. Also, the organisation should provide autonomy for supervisors to directly intervene. Concerning a facilitating organisation, participants reported that they need to be supported by senior management in terms of time and access to occupational health expertise. Also, if the intervention was tailored to their own organisation, it was easier to apply the suggested actions and learnings during the intervention. It was mentioned that the organisation should have interventions in place to offer to employees, that promote mental health.
Refined programme theory
We suppose the following refined programme theory after validation and evaluating the intervention.
On interpersonal level
If supervisors collaborated with the OHP in the past, have low key access to an empathic yet independent OHP, and they participate to the intervention by (small group) scheduled coaching sessions with OHP and they experience a safe learning climate without hierarchy (contextual factors); then this accessible, tailored expertise, as well as group learning with co-supervisors can trigger sharing cases out of trust to freely speak and gaining advice from the OHP, and supervisors shared experiencing with co-supervisors creating collective efficacy when realising not being alone in this (mechanisms), leading to increased supportive behaviour and intensified interaction with OHP and co-supervisors (outcomes).
On individual level
If supervisors are aware of their role and the urgency regarding preventive mental health management, and they are interested and somehow experienced in dealing with CMHP (in their personal life or having had a course on this topic), and they reflect and discuss using the guideline, and generally they are facilitated by the organisation and experience autonomy to intervene in a safe learning climate (contextual factors); then this intervention provides self-reflection by discussion about the guideline, getting confirmation and practise of given actions and considering alternatives, which can trigger supervisors’ awareness and willingness to improve, their self-efficacy on how to converse with employees who struggle at work and improved skills to act (mechanisms), strengthening their supportive behaviour to support employees with mental health problems (outcomes).
Discussion
We discuss the most important findings, followed by implications for future research and practice and the strengths and limitations of this study.
The impact of the intervention
Our first research question addressed whether the intervention works, in order to strengthen supervisor’s supportive behaviour. We found that for participants in this study, supervisors’ supportive behaviour to support employees with CMHP increased over time compared to baseline. Participants were deemed quite experienced, since single items for intention, attitude and social influence were already high at baseline. However, participating supervisors were motivated to develop themselves on this topic, reflected also by the finding from the qualitative interviews that willingness to improve was found an important mechanism. This may be because supervisors had voluntarily decided to participate in this intervention, as found in a similar study on facilitating employees’ return to work (Negrini et al. 2018). The results of the study indicate that supportive behaviour only increased significantly post-follow-up, after 6 months compared to baseline, and not directly after the intervention (at 3 months). Whereas skills and self-efficacy, framed as mechanisms of change, showed significant increases directly after the intervention. A possible explanation of such a ‘delayed’ behavioural change, based on the qualitative data, may be that new behaviour needs more time to consolidate (Davis et al. 2015). Supervisors expressed to appreciate the monthly sessions in which the intervention offered practical managerial actions that helped them out in current cases of team members struggling with CMHP, but they could reflect on previous cases also. However, they preferred a longer coaching period to consolidate new approaches.
Interpersonal mechanisms triggering behavioural change
Our study confirms that mechanisms are not only on an individual level but also on interpersonal and collective levels related to managerial behaviour and organisational culture (Marchal et al. 2012). This multi-level interpretation of findings is supported by previous realist research in the way that mechanisms, circumstances and outcomes operate at different levels (Punton et al. 2016) and interact with each other (Lacouture et al. 2015).
Interpersonal dynamics seem crucial in the way supervisory support is experienced. This applies to the dynamics between the employees with CMHP and their supervisors, but also to the supervisors and their interaction with other stakeholders in the organisation. Unsurprisingly, our intervention led to intensified interactions, where trust in the OHP’s expertise was especially triggered if supervisors collaborated with this OHP in the past. Establishing and maintaining a trusting relationship between the supervisor and OHP has also shown to be important for successful return-to-work trajectories (Corbiere et al. 2014). This seems challenging in practice, where often high caseloads and regular rotation of professionals lead to a lack of continuity of occupational health services in organisations (Saunders 2015; van Hees et al. 2022a). The SAW-SG intervention seems to nurture relationships among workplace stakeholders, facilitating personalized and continuous assistance that might be necessary in prevention as well (Quick et al. 2013). Moreover, this intervention showed how learning in small groups led to a group learning process among co-supervisors, in a way that supervisors supported each other to deal with employees with CMHP. This concept, also known as collective efficacy (Walumbwa et al. 2004), could be triggered because supervisors from the same organisation had the chance to define their shared beliefs and capabilities to organise and execute the courses of action, based on the guidelines (Salanova et al. 2003). While a previous study showed similar results for employees’ work outcomes, our findings suggest that through developing collective efficacy among supervisors, positive employee work outcomes (i.e., staying at work) can be promoted by providing co-supervisors with emotional support (Walumbwa et al. 2004).
Individual mechanisms triggering behavioural change
On the individual level, we identified the following mechanisms triggering a change in supervisors’ supportive behaviour: willing to improve, self-efficacy and skills to act. Self-efficacy is extensively proposed as a mechanism of change in the literature, being an important predictor of behaviour in various behavioural models. Also in our data, self-efficacy showed the largest change over time among all behavioural determinants. This suggests that the supervisor’s confidence in conversing about mental health issues is positively correlated with their supportive behaviour (Bryan et al. 2018; Ketelaar et al. 2017; Weston et al. 2019), that is in line with the self-efficacy theory (Bandura 1997). We also suggest that the increase in self-efficacy may be attributable to the multifaceted design of the intervention, in which supervisors could learn through feedback loops and by directly applying their action plans (Ketelaar et al. 2017). It seems likely that the increase in self-efficacy was achieved in our intervention by going beyond knowledge-based training to supervisors. The coaching sessions led to practical skills in having difficult conversations and allowing them space and time to practise these techniques with the right expert (OHP), using concrete action plans (Bryan et al. 2018; Negrini et al. 2018). These managerial actions could be tailored to fit the organisational context by the OHP, as an important middle range theory for organisational interventions (Roodbari et al. 2021).
Reflecting on the sequence of mechanisms, we saw that the mechanisms on interpersonal level lead also to changes in mechanisms on the individual level. To better understand how this intervention worked, we observed the mechanisms accrued in stages. Outcomes of one stage of self-efficacy and skills development informed or transformed the context for subsequent stages of behavioural change, referring to the ‘ripple effect’ (Jagosh et al. 2015). We theorize that self-efficacy and skills are foundational elements to a supervisor’s supportive behaviour, triggered by interpersonal mechanisms such as collective efficacy and trustful collaborations with professionals.
The importance of the organizational context
We particularly addressed the context by investigating personal factors and organizational factors, gaining insights under which circumstances this intervention works. As supervisors’ supportive behaviour does not occur in a vacuum, the organizational context plays an important role in implementation and evaluation of workplace interventions (Lundmark et al. 2020; Nielsen & Noblet 2018). Our results provide guidance to which contextual and personal factors to focus on in such interventions. In the qualitative results, with regard to various mechanisms, the enabling conditions of having a facilitating organisation and operating in a safe learning climate are deemed crucial to strengthen the supportive behaviour of supervisors (Holwerda et al. 2016). These ‘pre-existing’ contextual factors have been theorized as the omnibus contextual factors that influence readiness for change, (e.g., working conditions) and multi-level management support (e.g., allocating resources; Roodbari et al. 2023) and seem crucial in implementation of workplace mental health interventions (Paterson et al. 2024). Theory on leadership explains also how such contextual factors influence leadership processes and outcomes, such as leader behaviour, perceived support and trust in a leader (Oc 2018). Therefore, these findings underscore that it is not only the employee who needs to be supported by the organisation, to stay at work, but the supervisors themselves also need support from their organisation to increase their capacity on providing the accurate managerial actions needed by employees who struggle at work.
Our intervention addressed the work context as defined for organizational intervention processes, using four key principles (Nielsen & Noblet 2018). Regarding the first principle, tailoring the intervention to the organizational context, we provided OHPs with a basic, relatively flexible, implementation protocol and training. Flexibility allowed them to implement the intervention in a way that it would fit in the organisation (e.g., individually or in a group, or the number of supervisors being coached; Roodbari et al. 2021). In line with two other principles (participation and communication), we developed the implementation and evaluation activities through a participatory approach and offered materials for OHPs to communicate actively about those activities. This has been found to be successful in other studies (Abildgaard et al. 2020; Schelvis et al. 2016, van Hees et al. 2022c). Lastly, regarding the fourth principle of managerial support, supervisors need to feel and be supported by their own (staff) managers. This was ensured in our intervention by the fact that OHPs selected organisations that had support from senior management and in which supervisors were given time and autonomy to adopt and implement this intervention, in line with a recent review on the implementation of workplace mental health interventions (Paterson et al. 2024). This reflects the above mentioned interpersonal dynamics in the workplace, where supervisors also find themselves in a leader-worker relationship, where they should experience autonomy to increase supportive behaviour (Slemp et al. 2018).
Implications for future research
In general, our study findings respond to the call to further develop and refine methods to assess the causal explanations between contexts, mechanisms and outcomes in workplace interventions (Jagosh 2020; Nielsen & Miraglia 2017). Regarding the quantitative data, only a few covariates were significantly correlated and contributed to the outcome, while we explored thoroughly from the start of the intervention how and under which circumstances such intervention could work. We found that the working mechanisms leading to change may be either latent or unobservable and therefore hard to measure quantitatively. Therefore, our study demonstrates that evaluating what makes an intervention work using qualitative methods leads to more realistic insights from what actually happened during intervention implementation. More qualitative research is needed to unpack mechanisms and circumstances that lead to the outcomes of workplace interventions. An alternative to identify particularly salient implementation factors and mechanisms is coincident analyses (Roczniewska et al. 2024).
In the present study, we choose to evaluate an outcome that was proximal to the adherence of the intervention, namely supportive behaviour and its behavioural determinants, self-reported by supervisors. A recent review study on psychosocial work environment, health, and retention of workers, suggested that it might be easier to reach the more proximal effects of the interventions (e.g., in the psychosocial work environment) than it is to reach the more distal effects (health and retention; Aust et al. 2023). It would be highly interesting to gain insights on the impact of this intervention on more distal outcomes, such as individual outcomes (i.e., the employee’s experience of supervisory support and work outcomes such as sick leave numbers) and organisational outcomes (i.e., on mental health literacy of senior and middle management or organizational climate).
Our findings expose that the same intervention might work through a number of different mechanisms at the same time, each in specific circumstances (Abildgaard et al. 2020; Holman & Axtell 2016). We refined the initial programme theory using empirical data in one study. However, we suggest to test this refined programme theory in the next realist evaluation cycle in the same organisations until the observed patterns of context, mechanisms and outcomes are fully explained or can be used as an initial programme theory for other interventions in other organisations (Roodbari et al. 2023).
Implications for practise
This study demonstrated that through a multifaceted intervention, comprising of an online guideline and coaching sessions, supportive behaviour by supervisors can be strengthened. From an organizational perspective, the SAW-SG intervention can be easily implemented on a small scale, under the condition that supervisors are facilitated by senior management, in an organisation where they feel safe. As our study was conducted during the COVID-19 pandemic, sessions were held partly online. An advantage was to easily plan the coaching sessions, especially with a small group of supervisors in one organisation from different departments or locations. A disadvantage was that coaching seemed challenging when not yet knowing the supervisor or not being able to ‘read’ all non-verbal signals due to the virtual environment. We found that the intervention can be implemented face-to-face as well as online, which seems beneficial to its implementation.
Additionally, our study showed that it is valuable to create a low-key route for supervisors to consult an OHP, also in the phase of prevention. Supervisors valued the expertise and action-oriented approach of OHPs regarding challenging cases the supervisor was dealing with at that moment. This increased the likelihood to signal and act early and to discuss job accommodations or other interventions with the employee. OHPs had different conversations than they usually have in return-to-work trajectories. As a result, OHPs gained new insights into what is going on in the particular team or organisation before the employee reports ill. This may also benefit return-to-work trajectories since OHPs are better informed about the work context, to match employee’s capabilities with work and the work environment. Key to such participatory interventions is an open learning culture, in which supervisors and OHPs are co-learners in an empowerment process (Egan et al. 2009; Murta et al. 2007).
Although the intervention aimed to change the behaviour of individual supervisors, interventions as such may also promote the inclusivity of employees with CMHP in workplaces on a larger organizational level (Shahwan et al. 2022). In our observations, this occurred by introducing such intervention to HRM and senior management that increased awareness on the topic. Likewise, supervisors and OHPs acted as project champions, making plans for the sustainability and scalability of this intervention, for example, by alerting other supervisors and teams about the guideline or providing internal presentations and webinars (Ipsen et al. 2015; Paterson, et al. 2024; von Thiele Schwarz et al. 2017).
Strengths and limitations
First, a major strength of this study is the high level of fidelity to the intervention. All OHPs completed the intervention. For the supervisors, the response rates of the questionnaire were 71% after the intervention and 60% at follow-up. Second, in line with the realist approach, we allowed for a variety of settings and implementation strategies. In this way, OHPs could tailor the novel intervention to their own organisation and setting. The mixed-methods realist design provided detailed insights into which circumstances triggered certain mechanisms, in line with the recently recommended realist evaluation cycle in organisational interventions that improves the understanding and thereby the likelihood of intervention success (Roodbari et al. 2023). Third, the quantitative data were supplemented by interviews with more than one-fourth of the study population who completed the intervention (15/56 participants), which provided a comprehensive picture of the impact of the intervention.
A number of limitations need to be mentioned. First, our method of recruiting supervisors (OHPs who selected and invited the organisation and its supervisors) may have led to selection bias. Because of the voluntary participation we could explain the relatively high baseline scores, especially on supervisors’ exposure to CMHP and positive attitude and intention regarding the topic of this intervention. This was also found in a previous study, where the influence of transformative leadership was mediated through the supervisors’ positive attitudes and actions towards the intervention (Lundmark et al. 2017). Second, the questionnaires addressed items regarding the outcomes that were in line with the guideline’s themes. This may have led to socially desirable answers in the questionnaires throughout the intervention period. Also, it may be that supervisors reported certain behaviours due to their awareness of being observed, referring to the Hawthorn effect (McCambridge et al. 2014). However, we did not observe this during the interviews held on the impact of this intervention. Alternative ways of data collection could be explored, that is, the use of journaling or observations. Also, considering the self-serving bias of participants, the next evaluation of this intervention should include the employee’s experience of supervisory support. Lastly, the study population reflected mostly the public sector and a high proportion of supervisors working in the health care sector.
Conclusions
Using a mixed-methods realist approach, this study shows that after participating in a workplace intervention, supervisors’ self-reported behaviour, to support employees with CMHP increased compared to baseline. On the individual level, self-efficacy seemed an important mechanism, besides willingness to improve and skills to act. On the interpersonal level, the intervention intensified interactions between supervisors and OHPs besides increased supportive behaviour, due to collective efficacy among supervisors and trust in the OHP’s advice. To strengthen supervisory support, conditional circumstances triggering those mechanisms were allocated time and structural, low-key access to expertise from an OHP and operating in a safe learning climate. These findings may encourage employers to invest in the supportive behaviour of supervisors, who have an important role to promote work participation for employees struggling with CMHP. Preventive workplace interventions that are multifaceted and action-oriented towards the capacity of supervisors seem a way to enable employees with CMHP to (partly) keep working and perform well in their jobs. These findings may prove useful for researchers, policymakers and practitioners who must weigh the costs and benefits of preventive interventions targeted at supervisors, in order to create mentally healthy workplaces.
Additional Files
The additional files for this article can be found as follows:
Appendix S1
Appendix Outcomes and covariate measures of the questionnaire. DOI: https://doi.org/10.16993/sjwop.249.s1
Appendix S2
Appendix Additional results from quantitative data. DOI: https://doi.org/10.16993/sjwop.249.s2
Acknowledgements
We would like to thank all the participating OHPs and supervisors for their participation in the intervention.
Funding Information
The GAK Institute supported this work under grant 2017-928.
Competing Interests
The authors have no competing interests to declare.