Introduction
Leadership is a central and critical aspect of any organized activity. Many organizations implement leadership training programs on the basis that leadership can be developed and that leaders can learn to become more functional leaders (Lacerenza et al. 2017). However, such programs often have an inadequate scientific basis. Fundamental problems are that knowledge about what constitutes effective leadership training is relatively scarce (Lacerenza et al. 2017; Martin et al. 2021) and that a large proportion of the skills acquired in leadership training are not transferred to the work environment (Baldwin & Ford 1988; Ford, Baldwin & Prasad 2018).
Effective leadership training must be integrated into the organizational context in which managers operate. Training sessions often take place outside the workplace, and the new skills need to be translated or transferred into organizational practice. Positive transfer of training is defined as the degree to which trainees effectively apply the behaviors learned in a training context to the job (Baldwin & Ford 1988). Leadership training can thus be seen as an intervention at both the individual level and the organizational level. Both levels are involved in the change process and cannot be viewed in isolation from each other. Intervention researchers have pointed out the importance of going beyond effect studies by broadening the scope of intervention research to examine the context and mechanism of the interventions being studied (Nielsen & Miraglia 2017; Nielsen & Randall 2013). This means that interventions must be evaluated to an extent that covers more than the average effect sizes. Nielsen and Randall (2013) proposed a framework of three themes to examine interventions in more detail: the context, the mental models of the participants, and the design and implementation of the intervention. In the training transfer research, three themes of importance for training transfer have been identified: work environment (cf., the context of the intervention), trainee characteristics (cf., mental models of the participants), and training design (cf., the design and implementation of the intervention) (Baldwin & Ford 1988). Transfer research with more extensive and distributed training, delayed measures of training outcomes, independent raters of performance improvement (i.e., not self-ratings), independent methods for measuring the performance improvement and the predictors for the performance improvement, and measures of effectiveness instead of use are warranted (Baldwin & Ford 1988; Blume et al. 2010). The present study adheres to these five recommendations.
The purpose of the current study was to examine variation in outcomes among managers undergoing leadership training and to analyze this variation inspired by training transfer research and the organizational intervention framework proposed by Nielsen and Randall (2013). In this study, we consider leader behavioral change as a preceding step to performance development. Thus, leader performance is about translating trained leader behaviors adaptively into interactions with subordinates. This distinction corresponds to Blume et al.’s (2010) notion that using trained behaviors precedes effective application of the trained behaviors. We are interested in how the variation in the development of leader performance can be explained. The first aim was to investigate which contextual circumstances facilitate or hinder the development of leadership performance. The second aim was to investigate the importance of managers’ mental models in the development of leadership performance. The third aim was to identify important intervention activities for the development of leadership performance.
This study intends to contribute to the field by placing leadership training in a larger organizational context and to integrate training transfer theory and organizational development evaluation theory. The study does so by simultaneously testing the effect of several theoretically driven variables at multiple levels, where the effect of transfer is measured through independent raters of performance improvement (i.e., subordinates’ assessment of change in leaders’ performance).
Contextual circumstances
Contextual circumstances refer to how organizational factors and the working environment of the trained managers may either facilitate the transfer of leadership training or hinder it from being effective. Context may thus moderate and mediate the link between the intervention exposure and its outcomes (Nielsen & Randall 2013). The ultimate goal of leadership training is to help leaders develop skills and abilities that can be applied to their daily work in their organization (Lacerenza et al. 2017). Leaders need to develop functional leadership behaviors in order to contribute to a more effective and efficient organization. This suggests that the transfer of behaviors from the training to the organization is of key importance. Here, “transfer” refers to the extent to which trainees utilize the skills and abilities taught during training on the job (Lacerenza et al. 2017). However, different conditions in the leader’s work context may hinder or facilitate such transfer (Baldwin & Ford 1988; Ford, Baldwin & Prasad 2018).
Managerial level and experience
The manager’s organizational level and experience may affect the training results. First line leaders with little experience may enter leadership training with a great deal of room for improvement, so it may be easier for them to achieve good development results. On the other hand, high-level leaders often have extensive experience and thus may have less room for improvement as they approach a ceiling effect. In their meta-analytic review, Avolio et al. (2009) found that low-level leaders exhibited a greater overall effect from leadership training than middle-level and high-level leaders. In contrast, Lacerenza et al. (2017) found that low-, middle-, and high-level leaders experienced similar learning gains and improvements in their results, but that high-level leaders showed weaker transfer effects. These researchers suggest that high-level managers, through their longer experience, may have developed entrenched behaviors that are difficult to change. Thus, high-level leaders may have more difficulty translating the knowledge and skills acquired in leadership training into their managerial job.
Hypothesis 1: High-level managers are expected to have more difficulty translating acquired knowledge and skills into their managerial jobs and thus show less improvement in leadership performance.
Hypothesis 2: Managers with brief managerial experience are expected to benefit more from leadership training and thus to show greater improvement in leadership performance.
Span of control
The span of control (SOC) refers to the number of employees a manager is responsible for (Meier & Bohte 2000; Meier & Bohte 2003). It is a structural condition that affects the managers’ relationships with their subordinates (Schyns, Maslyn & Weibler 2010). Furthermore, a large SOC unfavorably increases managers’ job demands (Wallin, Pousette & Dellve 2014). It can be argued that a larger SOC makes it more difficult to assimilate training, because the manager’s operational everyday life is difficult to plan and is filled with urgent situations that must be handled.
Hypothesis 3: Managers with many subordinates are expected to experience more obstacles in assimilating leadership training and thus to show less improvement in leadership performance.
Working conditions of managers: job demands and resources
The job demand–resource theory (JD-R) (Bakker & Demerouti 2007; Bakker et al. 2003) provides a general framework for how working conditions may affect various organizational outcomes. The theory describes two general categories of working conditions: job demands and job resources. Job demands (i.e., stressors) are ‘the physical, psychological, social, or organizational aspects of the job that require sustained physical and/or psychological (i.e., cognitive and emotional) effort and are therefore associated with certain physiological and/or psychological costs’ (Bakker & Demerouti, 2007, p. 312). Job resources are the organizational aspects of the job that are stimulating and that reduce physiological and psychological costs. The JD-R theory proposes two main processes. One is driven by high demands and leads to job strain and health impairment. In the other process, job resources lead to high work engagement and performance. The JD-R theory has been applied to managerial work (Eklöf et al. 2010), and managers’ demands have been found to stem from the organization (organizational stressors), the role itself (role stressors), and interaction with subordinates (subordinate stressors). Resources for managers have been found to consist of support from superior managers (superior support) and from other managers at the same level (peer support) (Eklöf et al. 2010). Support from supervisors and peers are central work environment factors for training transfer (Baldwin & Ford 1988), in particular for open skills such as leadership performance (Blume et al. 2010). However, transfer research has provided inconsistent results related to support, and differentiating between superior and peer support is warranted (Richter et al. 2020; Tafvelin et al. 2021).
Hypothesis 4: Higher demands in terms of a) organizational stressors, b) role stressors, and c) subordinate stressors are expected to make it difficult to assimilate leadership training and thus result in less improvement in leadership performance.
Hypothesis 5: A larger amount of available resources in terms of a) superior support and b) peer support are expected to make it easier to assimilate leadership training and thus result in greater improvement in leadership performance.
Mental models of the participants
Mental models impact how participants relate to an intervention and the activities provided (Nielsen & Randall 2013). A fundamental mental model is the participant’s readiness for change. Participants’ readiness for change is important for the success of organizational interventions (Weiner, Amick & Lee 2008). Organizational readiness for change refers to organizational members’ change commitment and change efficacy in implementing organizational change (Weiner 2009). Thus, being ready for change means that a) the participant perceives that there is a relevant need for change that must be addressed (i.e., change commitment), and b) that the intervention offered is believed to have the capacity to respond to this need for change (i.e., change efficacy). When organizational readiness for change is high, organizational members are more likely to initiate change, exert greater effort in support of change, and exhibit greater persistence in the face of obstacles or setbacks during implementation (Nytrø et al. 2000; Weiner 2009). Similarly, trainee motivation has been suggested to be a critical component of trainee characteristics for successful training transfer (Baldwin & Ford 1988; Blume et al. 2010; Ford, Baldwin & Prasad 2018; Tafvelin & Stenling 2021). We propose that managers with high readiness for change can be expected to put more effort, initiative, and persistence into training activities and homework assignments, which would make them more successful in developing their leadership.
Hypothesis 6: Managers with high readiness for change to develop their leadership are expected to show greater improvement in leadership performance.
Intervention design and implementation
The implementation of an intervention involves the mechanism by which it achieves its outcomes (Nielsen & Miraglia 2017). In training transfer research, this aspect is categorized under training design (Baldwin & Ford 1988). Baldwin & Ford (1988) conclude that training transfer research should integrate findings from counseling and psychotherapy for application to organizational settings to maximize training transfer. Three core components of cognitive behavior therapy that can be applied to leadership training and may constitute important intervention activities are behavior analysis (Haynes & O’Brien 1990), behavior training (Segrin 2008), and homework assignments to facilitate contextual adaptation (Robinson 2008).
Behaviour analysis
Lacarenza et al. (Lacerenza et al. 2017) concluded that needs analysis is an important activity in leadership training. Behavior analysis is a procedure for need analysis that involves identifying behavior deficits while considering important, controllable, causal relationships between the behavior deficits and the environmental antecedents to, and consequences of, the behavior (Greer 2020; Haynes & O’Brien 1990). Von Thiele Schwarz et al. (2016) and Gravina et al. (2019) have outlined how behavior analysis can be used to individualize leadership training to fit the needs of the manager. Grounding the training in a needs analysis increases the job relevance of the training content, which potentially improves training transfer (Baldwin & Ford 1988). Behavior analysis also includes general principles—that is, providing managers with a theoretical understanding of basic mechanisms in behavioral learning—another potentially critical design element for training transfer (Baldwin & Ford 1988; Blume et al. 2010).
Hypothesis 7: Managers whose training is adapted to fit the individual manager’s needs through behavior analysis are expected to show greater improvement in leadership performance.
In-session behaviour training
Lacarenza et al. (2017) identified practice as a critical activity in leadership training for achieving transfer and results (i.e., influence on organizational and subordinate outcomes). Behavior rehearsal during training sessions enables managers to practice the desired behaviors in a supportive environment and receive reinforcing feedback before performing them in an actual target situation at work (Segrin 2008; Sundel & Sundel 2017).
Hypothesis 8: Training new behaviors during training sessions increases the conditions for developing leadership; thus, leaders under such training conditions are expected to show greater improvement in leadership performance.
Homework assignments: plan, implement, evaluate
Lacarenza et al. (2017) identified the training location as a potent moderator; the influence on organizational and subordinate outcomes was found to be considerably higher for on-site (d = 1.12) than off-site training (d = 0.37). Here, ‘on-site’ implies an environment that was similar, if not identical, to the managers’ work environment. Homework provides the managers with identical stimulus and response elements in the training and transfer settings and increases the opportunity to use the trained behaviors, two suggested success factors for training transfer (Baldwin & Ford 1988). In addition, designing homework assignments in which the managers practice their trained leadership behavior at work can facilitate contextual adaptation (Robinson 2008) through instrumental (Greer 2020) and social-positive (Beavers, Iwata & Lerman 2013) reinforcement.
Hypothesis 9: Managers whose training includes planning, implementation, and evaluation of homework assignments are expected to show greater improvement in leadership performance.
In summary, nine hypotheses were formulated to test the effect on the transfer of leadership training, based on the organizational intervention framework and training transfer theory; see Figure 1.
Figure 1
The present study hypotheses mapped on an integration of organizational intervention framework (Nielsen & Randall 2013) and training transfer theory (Baldwin & Ford 1988).
Materials and methods
Participants and procedures
This study investigates variation in the outcomes of a managerial training intervention based on behavioral principles (Björnsdotter & Grill 2021) known as managerial behavior training (MBT) (Grill, Pousette & Björnsdotter 2023). The managers were recruited and trained in a cross-municipal unit for the recruitment and development of municipal managers for local governmental municipalities in Western Sweden. Informed consent was obtained from all participants. The study was part of a larger research project, which has been approved by the Swedish Ethical Review Authority (reg. no. 1060-18/2019-00590). Other results from this project are reported elsewhere (Grill et al. 2023).
The training was distributed (Baldwin & Ford 1988) and comprised six one-to-one sessions 2 weeks apart over a period of three months, and each session lasted approximately 90 min. All trainers were licensed psychologists. Each session was designed to include behavior analysis (Haynes & O’Brien 1990), in-session behavior training (Segrin 2008), and homework assignments (Robinson 2008). Based on the manual, the actual content for each session was adapted to suit the development needs of each manager.
The training included behavior analysis, goal setting, instructions, practice with performance feedback, and homework. Behavior analysis involved assessing the manager’s pre-intervention levels of goal setting, performance feedback, value-based feedback, and listening. The manager and trainer conducted a behavioral interview to identify and analyze stimuli related to the manager’s leadership behaviours. Managers were trained to conduct functional analyses of their own and their employees’ behaviors, identifying key behavioral deficits. Goals were set to target the deficits identified in the behavior analysis. One to three behavioral and participative goals were set and documented. Managers received spoken and written instructions on conducting behavior analysis, setting goals, giving feedback, and using listening behaviors as reinforcement. Practice with performance feedback consisted of behavioral training and rehearsal to shape leadership behaviors, with exercises selected from 11 modules, including listening, dealing with criticism, positive reinforcement, and time management. Homework assignments were given to practice trained behaviors at work, followed up in subsequent sessions with performance feedback on progress towards goals.
In total, 64 managers participated in the training. Of these, 59 managers completed at least five sessions and were included in this study. The training was run in two subsequent rounds, with 25 managers in the first round (September to November 2019) and 34 managers in the second round (January to April 2020). In the first round, the manager and trainer met in person. In the second round, due to the COVID-19 pandemic, societal restrictions were implemented, which had a profound influence on the training and data collection.1
Data collection
This study used data from multiple sources collected before, during, and after the intervention period. The participating managers reported on contextual information and their mental models before the training started by means of an online questionnaire, with a response rate of 100%. The proportion of higher-level managers (who had managers as subordinates) was 34%, while 66% were first-line managers. The average number of subordinates was 14.7 (SD = 10.4). The mean age of the managers was 47.9 years (SD = 6.0), the mean managerial experience was 10.2 years (SD = 6.7), 61% were women, and 98% had a university education. Data on the implementation was collected at the end of each of the six training sessions; both the participating managers and the trainers independently assessed the implementation of the session. After completion, the forms were put in sealed envelopes and distributed to the first author. A total of 347 forms from participating managers and 351 forms from trainers were returned, with respective response rates of 98.6% and 99.7%. Data on the dependent variables was collected from the participating managers’ employees via online questionnaires, with the employees rating managerial performance before and after the intervention period. A total of 406 questionnaires were received pre-intervention (173 in round one and 233 in round two, with a response rate of 71.6%) and 377 post-intervention (157 in round one and 220 in round two, with a response rate of 69.4%). Each manager was rated by an average of 6.9 (SD = 3.1) employees pre-intervention and by an average of 6.4 (SD = 3.2) employees post-intervention. The mean age of the responding employees pre-intervention was 45.1 years (SD = 10.3), 62.0% were women, and 78.6% had a university education.
Measurements
In this study, the predictor variables were categorized according to contextual circumstances, the mental models of the participants, and the intervention implementation. The dependent variable was leadership performance, assessed by the employees of the participating managers.
Contextual circumstances
Five types of contextual circumstances were measured: managerial level, managerial experience, span of control, managers’ work demands (i.e., stressors), and managers’ work resources (i.e., supports).
The dichotomous variable managerial level was coded as high managerial level (i.e., a strategic manager) if at least one of the subordinates was a manager; otherwise, the manager was coded as low managerial level (i.e., an operative manager). Managerial experience referred to the manager’s number of years in a managerial position. Span of control referred to the manager’s number of direct subordinate permanent employees. Managers’ work demands and work resources were measured using questions from the Gothenburg Manager Stress Inventory (GMSI) (Eklöf et al. 2010). Gadolin et al. (2022) reported the factor structure of these scales. The manager assessed the occurrence of demanding and supporting circumstances in her or his work situation during the past 6 months. Three types of work demands were included: organizational stressors (three questions, α = 0.65); role stressors (three questions, α = 0.56); and subordinate stressors (six questions, α = 0.85). The questions were answered with five response options, ranging from never/almost never (1) to always/almost always (5). Two types of resources were included: support from senior management (superior support) (four questions, α = 0.88) and support from managerial colleagues (peer support) (two questions, α = 0.89). The questions had five response options, ranging from very poorly (1) to very well (5). All scales are reported in Appendix 1.
Mental models of the participants
Readiness for change. Three scale questions from the framework of motivational interviewing (Rollnick & Miller 1995) were adapted for the purpose of assessing readiness for leadership change. The questions represented two facets of readiness for change (i.e., change commitment and change efficacy) and their theoretical outcome (i.e., change effort) (Weiner 2009). The questions had 11 response options, ranging from not at all (0) to very much (10). The Cronbach’s alpha was α = 0.76. Items are reported in Appendix 1.
Intervention design and implementation
Behavior analysis was assessed by two questions after sessions one to five. The ratings for all sessions were averaged. The Cronbach’s alpha was 0.87 for the managers and 0.67 for the trainers. In-session behavior training was assessed by three questions after sessions two to five. The ratings for all sessions were averaged. The Cronbach’s alpha was 0.97 for the managers and 0.72 for the trainers. The homework assignments consisted of three components: planning, implementation, and evaluation. Homework planning was assessed by three questions after sessions one to five. The ratings for all sessions were averaged. The Cronbach’s alpha was 0.61 for the managers and 0.76 for the trainers. Homework implementation was assessed by a single question after sessions two through six. The ratings for all sessions were averaged. Homework evaluation was assessed by a single question after sessions two through six. The ratings for all sessions were averaged. The questions had six response options, ranging from not at all true (1) to completely true (6). All scales are reported in Appendix 1.
Dependent variable
Leadership performance was assessed by the employees of the participating managers.
Leader performance was assessed with nine items adapted from the multi-factor leadership questionnaire (Avolio & Bass 2004) encompassing three facets of leadership performance: leadership effectiveness (four items), employee engagement (three items), and employee satisfaction with manager (two items). All items were rated on a Likert scale ranging from 1 (never) to 7 (always). The composite reliability was α = 0.95. Items are reported in Appendix 2.
Data analysis
To evaluate the variation in the effect of leadership training on performance, multilevel modeling (MLM) was performed using the mixed model procedure in IBM SPSS Statistics version 28. When participants are nested in groups (i.e., employees belonging to the same manager), MLM accounts for this dependence in data (Snijders & Bosker 2011). A base model consisting of a main fixed effect for measurement time (pre- and post-intervention) was estimated. At level 1, repeated observations from the employees were estimated using an unrestricted variance structure (allowing the variance to take different values at pre- and post-intervention). At level 2, the variation between managers was accounted for by a random intercept and a random slope. The significance of slope variance was tested by comparing model fit through a difference in the log-likelihood between models with and without a random slope (Snijders and Bosker 2011). To estimate variance components and model comparisons, the restricted maximum likelihood (REML) estimation method was used.
To assess the potential impact of the COVID-19 pandemic on round two, the differences between the two rounds were tested by extending the base model with a dichotomous explanatory variable (i.e., round one and round two). The main effect and the moderation effect (interaction between time and round) were estimated. Furthermore, independent t-tests were performed for all predictor variables to test for differences due to the round.
For each hypothesis testing of the context variables and mental model, the base model was extended with one explanatory variable. The main effect (effect of the predictor variable without regard to time) and the moderation effect (interaction between time and the predictor variable) were estimated. The hypothesis was supported if a significant moderation effect was found in the expected direction, since such a moderation effect implies that the change in leadership performance from pre- to post-intervention is moderated by (i.e., dependent on) the level of the predictor variable.
For the intervention activities variables, the procedure was the same, but two predictor variables (assessments by the manager and the trainer) were included simultaneously in the models. Maximum likelihood (ML) was used for the estimation of all models for hypothesis testing.
Results
Table 1 provides the descriptive statistics and correlation between the predictor variables measuring the context of the intervention, the mental models of the participants, and the implementation of the intervention.
Table 1
Descriptive statistics and correlation between the predictor variables measuring the context of the intervention, the mental models of the participants, and the implementation of the intervention (n = 59).
| 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | 9 | 10 | 11 | 12 | 13 | 14 | 15 | 16 | 17 | 18 | MIN | MAX | M | SD | ||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 1 | Managerial level | 1.00 | 2.00 | 1.66 | 0.48 | ||||||||||||||||||
| 2 | Managerial experience | –.33* | 1.00 | 25.00 | 10.24 | 6.67 | |||||||||||||||||
| 3 | Span of control | .43** | –.23 | 2.00 | 49.00 | 14.73 | 10.42 | ||||||||||||||||
| 4 | Readiness for change | .31* | –.14 | .21 | 5.67 | 10.00 | 8.81 | 1.08 | |||||||||||||||
| 5 | Organizational stressors | –.02 | –.14 | –.12 | .17 | 1.33 | 5.00 | 3.29 | 0.78 | ||||||||||||||
| 6 | Role stressors | .03 | .03 | .04 | –.10 | .50** | 1.00 | 5.00 | 2.93 | 0.75 | |||||||||||||
| 7 | Subordinate stressors | –.10 | .18 | –.20 | –.08 | .34** | .15 | 1.00 | 5.00 | 3.79 | 0.65 | ||||||||||||
| 8 | Superior support | .00 | .06 | –.28* | –.23 | –.42** | –.51** | –.12 | 1.00 | 5.00 | 2.16 | 0.90 | |||||||||||
| 9 | Peer support | –.02 | .12 | –.25 | –.34** | –.44** | –.46** | –.13 | .65** | 1.00 | 4.00 | 2.09 | 1.01 | ||||||||||
| 10 | Behaviour analysis (manager) | –.02 | .03 | –.27* | –.02 | –.02 | .02 | .16 | –.04 | –.06 | 3.70 | 6.00 | 5.29 | 0.50 | |||||||||
| 11 | Behaviour analysis (trainer) | –.06 | .10 | –.14 | –.18 | –.13 | –.10 | .07 | .19 | .25 | .17 | 3.50 | 5.63 | 4.79 | 0.54 | ||||||||
| 12 | In-session behavior training (manager) | .06 | –.19 | –.07 | –.07 | –.05 | –.04 | .06 | .09 | –.04 | .59** | .17 | 1.00 | 6.00 | 4.83 | 1.13 | |||||||
| 13 | In-session behavior training (trainer) | –.03 | .02 | –.15 | –.18 | –.17 | –.24 | .14 | .27* | .32* | .09 | .80** | .24 | 3.08 | 6.00 | 4.79 | 0.61 | ||||||
| 14 | Homework planning (manager) | –.13 | –.08 | –.16 | .12 | .04 | .02 | .07 | –.06 | –.34** | .57** | .04 | .61** | .06 | 3.50 | 6.00 | 5.32 | 0.56 | |||||
| 15 | Homework planning (trainer) | .00 | –.07 | –.19 | –.11 | –.16 | –.19 | .02 | .31* | .19 | .12 | .68** | .18 | .66** | .25 | 3.60 | 6.00 | 5.06 | 0.58 | ||||
| 16 | Homework implementation (manager) | .14 | .08 | –.23 | .13 | .01 | .17 | .09 | –.12 | –.11 | .54** | .06 | .14 | –.04 | .38** | .11 | 2.75 | 6.00 | 4.61 | 0.67 | |||
| 17 | Homework implementation (trainer) | .00 | .05 | –.31* | –.07 | .09 | –.01 | .05 | .07 | .24 | .21 | .52** | .00 | .45** | .13 | .50** | .53** | 2.00 | 6.00 | 4.44 | 0.83 | ||
| 18 | Homework evaluation (manager) | –.08 | –.03 | –.16 | .22 | .16 | .07 | .15 | –.14 | –.20 | .72** | .04 | .60** | .06 | .63** | .05 | .40** | .09 | 4.20 | 6.00 | 5.32 | 0.51 | |
| 19 | Homework evaluation (trainer) | .00 | –.09 | –.08 | –.07 | –.10 | –.05 | –.01 | .07 | .19 | .14 | .67** | .30* | .67** | .28* | .64** | .22 | .67** | .24 | 2.20 | 5.60 | 4.63 | 0.68 |
[i] Note: *p < .05, **p < .01.
The stability coefficient of the dependent variable—that is, the correlation between leader performance at T1 (M = 5.322, SD = 1.179) and at T2 (M = 5.328, SD = 1.184)—was r(327) = .817, p < .001.
The base model for leader performance did not show a significant effect from the time (B = –0.006 s.e. = 0.043 t(42.3) = 0.130, p = 0.897). Table 2 shows parameter estimates for the random components in the base model. The amount of variation between the managers, calculated as the random intercept divided by the total variance, was approximately 25.1%. The amount of change at the managerial level, calculated as the slope variance divided by the total variance, was approximately 1.4%. Model comparisons between the base model (with a random slope) and a model without a random slope showed that the random slope was not significant (diff log-likelihood (1) = 1.001, p > .05).
Table 2
Parameter estimates for the random components in the base models for leader performance.
| ESTIMATE | STD. ERROR | |
|---|---|---|
| Repeated measures (employees) | ||
| Variance T1 | 1.044 | 0.079 |
| Covariance T1–T2 | 0.805 | 0.072 |
| Variance T2 | 1.056 | 0.082 |
| Managerial level | ||
| Intercept variance | 0.359 | 0.098 |
| Slope variance | 0.020 | 0.022 |
Introducing the round as a predictor variable in the model showed no main effect of the round but did show a significant moderation effect (B = 0.215 s.e. = 0.083 t(41.6) = 2.602, p = 0.013). This means that the effect of the intervention was greater in the first round than in the second round. However, T-tests for predictor variables at T1 showed no difference between the rounds. Hence, round was not controlled for in the subsequent analyses.
Contextual circumstances
For each contextual variable, a regression model was estimated; see Table 3.
Table 3
Effect for leader performance of contextual variables. The testing of hypothesis was based on the moderation effects; a significant moderation effect indicates that the change in leadership performance from pre- to post-interventions was dependent on the level of the predictor variable.
| PREDICTOR VARIABLE | MAIN EFFECT | MODERATION EFFECT | ||||||||
|---|---|---|---|---|---|---|---|---|---|---|
| ESTIMATE | STD. ERROR | df | t | p | ESTIMATE | STD. ERROR | df | t | p | |
| Managerial level | 0.149 | 0.206 | 66.545 | 0.720 | 0.474 | –0.053 | 0.095 | 47.569 | –0.557 | 0.580 |
| Managerial experience | –0.011 | 0.015 | 64.369 | –0.748 | 0.457 | 0.004 | 0.007 | 48.448 | 0.602 | 0.550 |
| Span of control | 0.007 | 0.009 | 61.298 | 0.728 | 0.470 | –0.005 | 0.004 | 60.683 | –1.139 | 0.259 |
| Readiness for change | 0.054 | 0.089 | 59.289 | 0.605 | 0.548 | 0.078 | 0.038 | 43.105 | 2.041 | 0.047 |
| Organizational stressors | 0.142 | 0.120 | 57.694 | 1.180 | 0.243 | –0.028 | 0.057 | 43.497 | –0.504 | 0.617 |
| Role stressors | 0.074 | 0.127 | 59.072 | 0.582 | 0.563 | –0.154 | 0.060 | 42.901 | –2.584 | 0.013 |
| Subordinate stressors | 0.440 | 0.146 | 73.421 | 3.013 | 0.004 | 0.024 | 0.078 | 51.722 | 0.315 | 0.754 |
| Superior support | –0.132 | 0.105 | 61.104 | –1.253 | 0.215 | 0.117 | 0.054 | 59.628 | 2.177 | 0.033 |
| Peer support | –0.137 | 0.094 | 61.006 | –1.460 | 0.149 | 0.085 | 0.043 | 46.389 | 1.974 | 0.054 |
Managerial level: hypothesis 1
High-level managers were expected to have more difficulty translating acquired knowledge and skills into their managerial job; thus, a negative moderation effect was expected. However, no moderating effect of managerial level on leader performance was found. Hence, hypothesis 1 was not supported.
Managerial experience: hypothesis 2
Managers with brief managerial experience were hypothesized to benefit more from the training; thus, for longer experience, a negative moderation effect was expected. However, no moderating effect of managerial experience on leader performance was found. Hence, hypothesis 2 was not supported.
Managers’ span of control: hypothesis 3
Managers with many subordinates were hypothesized to experience more obstacles in assimilating the training; thus, a negative moderation effect was expected. However, no moderating effect of SOC on leader performance was found. Hence, hypothesis 3 was not supported.
Managers’ job demands: hypothesis 4
Higher job demands were expected to make it more difficult for managers to assimilate the training; thus, negative moderation effects were expected for the three stress variables of organizational stressors, role stressors, and subordinate stressors.
Organizational stressors. No moderating effect of organizational stressors on leader performance was found. Hence, hypothesis 4a was not supported.
Role stressors. A moderating effect of role stressors was found for leader performance (B = –0.154, p = .013), implying that managers with higher role stress did not develop their leadership performance as much as managers with lower role stress. Hence, hypothesis 4b was supported.
Subordinate stressors. No moderating effect of subordinate stressors on leader performance was found. Hence, hypothesis 4c was not supported. However, a main effect of subordinate stressors was found (B = 0.440, p = .004), implying a higher frequency of leader performance among managers that experience more subordinate stressors.
Managers’ resources: hypothesis 5
More resources were expected to make it easier for a manager to assimilate the training; thus, positive moderation effects were expected for the two resource variables of superior support and peer support.
Superior support. A positive moderation effect of superior support was found for leader performance (B = 0.117, p = .033). Hence, hypothesis 5a was supported. This implies that managers with higher superior support developed their leadership more than managers with lower superior support.
Peer support. A borderline significant moderation effect of peer support was found for leader performance (B = 0.085, p = .054). Hence, hypothesis 5b was partly supported. This finding implies a tendency for managers with high peer support to improve their performance more than managers with low peer support.
Mental models of the participants
Readiness for change: hypothesis 6
Managers with high readiness for change in developing their leadership were expected to put more effort, initiative, and persistence into their training and homework, which would make them more successful in developing their leadership; thus, a positive moderation effect was expected. A positive moderation effect of motivation was found for leader performance (B = 0.078, p = .047) implying that managers with a higher readiness for change developed more than managers with lower readiness for change. Hence, hypothesis 6 was supported.
Intervention design and implementation
Table 1 provides the descriptive statistics and the correlation between managers’ and trainers’ ratings of the five variables assessing the intervention implementation. The correlations between the managers’ and trainers’ ratings of the variables were between 0.17 and 0.53. Only the correlation for the variable of homework implementation was statistically significant. Thus, with the exception of homework implementation, the agreement between managers and trainers was low. The ratings of the variables were generally high. All measures showed mean values in the higher end of the six-point response format (M = 4.61–5.32 for the managers and M = 4.44–5.06 for the trainers). The managers rated three variables significantly higher than the trainers: behavior analysis (managers: M = 5.29, SD = 0.50; trainers: M = 4.79, SD = 0.54, t(58) = 5.65, p < .001); homework planning (managers: M = 5.32, SD = 0.56; trainers: M = 5.06, SD = 0.58, t(58) = 2.93, p < .01); and homework evaluation (managers: M = 5.32, SD = 0.51; trainers: M = 4.63, SD = 0.68, t(58) = 7.12, p < .001).
For the testing of the hypothesis, one regression model was estimated for each variable; see Table 4.
Table 4
Effect for leader performance of intervention activities. The testing of hypothesis was based on the moderation effects; a significant moderation effect indicates that the change in leadership performance from pre- to post-interventions was dependent on the level of the predictor variable.
| PARAMETER | MAIN EFFECT | MODERATION EFFECT | ||||||||
|---|---|---|---|---|---|---|---|---|---|---|
| ESTIMATE | STD. ERROR | df | t | p | ESTIMATE | STD. ERROR | df | t | p | |
| Behaviour analysis (manager) | 0.167 | 0.184 | 58.099 | 0.908 | 0.368 | 0.132 | 0.084 | 43.275 | 1.571 | 0.124 |
| Behaviour analysis (trainer) | 0.212 | 0.171 | 57.834 | 1.240 | 0.220 | 0.042 | 0.078 | 42.302 | 0.534 | 0.596 |
| In-session behavior training (manager) | 0.097 | 0.088 | 67.058 | 1.098 | 0.276 | 0.044 | 0.041 | 56.242 | 1.065 | 0.291 |
| In-session behavior training (trainer) | 0.067 | 0.157 | 59.841 | 0.423 | 0.674 | 0.049 | 0.072 | 41.996 | 0.687 | 0.496 |
| Homework planning (manager) | 0.216 | 0.170 | 59.535 | 1.266 | 0.210 | 0.034 | 0.078 | 41.080 | 0.433 | 0.667 |
| Homework planning (trainer) | 0.159 | 0.168 | 60.449 | 0.946 | 0.348 | –0.018 | 0.078 | 42.195 | –0.234 | 0.816 |
| Homework implementation (manager) | –0.002 | 0.168 | 59.513 | –0.014 | 0.988 | –0.037 | 0.076 | 44.837 | –0.489 | 0.627 |
| Homework implementation (trainer) | 0.030 | 0.132 | 54.606 | 0.224 | 0.823 | –0.022 | 0.058 | 34.924 | –0.384 | 0.704 |
| Homework evaluation (manager) | 0.270 | 0.183 | 55.567 | 1.476 | 0.146 | 0.116 | 0.085 | 44.513 | 1.372 | 0.177 |
| Homework evaluation (trainer) | 0.019 | 0.137 | 54.719 | 0.138 | 0.891 | –0.047 | 0.063 | 41.749 | –0.739 | 0.464 |
Behaviour analysis: hypothesis 7
Adapting training to fit the individual managers needs through behavior analysis was expected to result in larger improvements in leadership performance; thus, a positive moderation effect was expected. However, no moderating effect of managers’ or trainers’ rating of behavior analysis on leader performance was found. Hence, hypothesis 7 was not supported.
In-session behaviour training: hypothesis 8
In-session behavioral training of new behaviors was expected to result in larger improvements in leadership performance; thus, a positive moderation effect was expected. However, no moderating effect of managers’ or trainers’ rating of in-session behavior training on leader performance was found. Hence, hypothesis 8 was not supported.
Homework assignment: hypothesis 9
Planning, implementation, and evaluation of homework assignments were expected to result in larger improvements in leadership performance; thus, positive moderation effects were expected. However, no moderation effects for any of these variables on leader performance were detected. Hence, hypothesis 9 was not supported.
Discussion
Leadership training always takes place in an organizational context. It is therefore important to examine the conditions for leadership training at both the individual and organizational levels. The purpose of the current study was to examine variation in improvement of leadership performance among managers in managerial training and to analyze this variation inspired by the framework proposed by Nielsen and Randall (2013) and transfer research (Baldwin & Ford 1988). We investigated the roles of context, mental models, and intervention design and implementation on the effect of a leadership training intervention based on behavioral principles. The results showed that several context variables and the manager’s readiness for change were important for the improvement of leadership performance, but we could not find evidence of an effect of specific intervention activities.
The first specific aim was to investigate which contextual circumstances facilitate or hinder the development of leadership performance. The work situation of managers in terms of demands and resources was found to have several effects on the outcome of training. Superior support facilitated the improvement of leadership performance. Another potentially important source of support was managerial colleagues, exhibiting a borderline facilitating effect on the development of leadership performance. However, role stressors exhibited a hindering effect. In sum, the results indicate that the transfer of new skills and their translation into effective leadership can be difficult for managers in unfavorable work situations. Berntson, Wallin & Härenstam (2012) have shown the work environment of managers to be associated with managerial performance. Our results show that the work environment is also of importance for developing as a manager. If demands and resources are not balanced in the organization, the effect of leadership training is likely to be hampered. To get the most out of leadership training interventions, the managers’ work situation must be appropriate.
Our findings on the importance of social support for the transfer of training are in line with those of the meta-study by Blume et al. (2010). There, they found consistent support for social support, especially from supervisors, being related to the transfer of training.
However, role stressor is a previously not specifically identified predictor for training transfer. Blume identified organizational constraints in the work environment (which could include role stress), but this category had a negligible relationship with transfer. In our operationalization, role stressors include having to work overtime or take work home; tension between administrative work, business development, and contact with employees; and having unclear or undefined responsibilities and goals. Role stress was in fact the strongest predictor for training transfer in our study, implying that future leadership training practice should address role stress as part of their training interventions. Future research on how managers role stress can be addressed before or during training interventions is warranted. Interventions could include reducing workload during training and/or clarifying and defining the manager’s responsibilities and goals prior to initiating a training intervention.
The second aim was to investigate the importance of the managers’ mental models for the development of leadership performance. Participants’ readiness for change has been highlighted as a key mental model (Weiner 2009, Nytrø et al. 2000). Our results showed that managers’ pre-intervention readiness for change had a facilitating effect on the development of leadership performance, thus corroborating research on motivation as a predictor for training transfer (Baldwin & Ford 1988; Blume et al. 2010). Readiness for change thus seems to impact the manager’s ability to translate newly acquired knowledge and skills into a leadership that delivers results in the eyes of employees.
The third aim was to identify important intervention activities for the development of leadership performance. Behavior analysis, behavior training, and homework assignment have been identified as important elements in leadership training (Grill et al. 2023). We found no facilitating or hindering effects from any of the intervention activities we studied. Should this be understood to mean that variation in the implementation of the intervention does not matter for the effects of leadership training? We think not. Participants’ and trainers’ assessments of the intervention activities showed that the training worked very well throughout; that is, the average ratings of the intervention activities variables were very high. Thus, the opportunity to detect any impact of specific intervention activities was restricted. However, it can be concluded that the intervention activities we identified as critical and necessary for change to occur were consistently in place.
There were differences between the rounds in the effect of the intervention, with the first round showing a greater effect than the second. However, there was no difference in the predictor variables between the rounds. Thus, it seems that conditions other than those we measured were important in the difference in the intervention effect between the rounds. One major difference was the COVID-19 pandemic, which started during round two. At the onset of the pandemic, interactions between managers and employees decreased dramatically. Many managers and employees were forced to work from home, which severely limited their opportunities for interaction, probably making it more difficult for managers to transfer new behaviors into daily managerial practice. Early in the pandemic, technical platforms did not work optimally, which further hampered communication. It is possible that employees did not notice any change in leadership performance because they did not meet their managers in person. The results point to the importance of interaction with employees in translating knowledge and skills into developed and functional management behaviors.
Managers with more stress from subordinates showed higher rates of leader performance. This was an unexpected finding for which we cannot provide an adequate explanation. Future research needs to show whether this was a sample-specific random association or whether there may be an underlying third variable that explains the association.
Methodological discussion
This study was conducted on municipal managers in a few municipalities in Sweden, which may restrict the generalizability of the results. Moreover, the participating managers were highly motivated to participate, which may also restrict generalizability. There were several conditions that may have limited this study’s ability to detect the effects of context and intervention activities on intervention outcomes. First, the multilevel analysis of random effects showed small random slopes. This means that there was little variation in the changes to explain at the managerial level, which may have limited our ability to detect causes of differences in change between managers. Second, there were relatively large numbers of employees but relatively few managers, resulting in low power in the moderation analyses, which, in turn, reduced our ability to detect effects concerning variation in the development of leadership performance. Third, low reliability in some of the context predictor variables (organizational stressors and role stressors) may have attenuated the effects in the moderation analyses. Fourth, the intervention activities variables had high mean values, which limited variation; this may have made it difficult to detect effects. Fifth, the low consistency between the raters of the intervention activities (i.e., managers’ and trainers’ ratings) may indicate the low reliability of the session ratings. Overall, this may have reduced this study’s ability to detect the effects of intervention activities. Despite these restrictions, this study has identified several conditions that facilitate or impede training.
A strength of this study is that the outcome measures of the training did not come from the managers themselves but from another data source consisting of assessments made by the managers’ subordinates. The subordinates can thus be viewed as observers of the managers’ performance. This reduces threats to validity from common method variance (Podsakoff et al. 2003).
Conclusions
In this study, we showed the importance of working conditions in giving leaders good opportunities to develop their leadership. Organizations wishing to increase the competence of their leaders by offering leadership training need to be aware of what is required to achieve favorable results. Senior managers who provide good and genuine support, for example, by taking an interest in the work of subordinate managers and facilitating a work situation that allows time for development, are one such prerequisite. Managers who are going to train their own leadership abilities must also have reasonable working conditions without too much stress; moreover, they require a willingness to enter into a process of change. Leaders at all levels can support such readiness for change by being present and supportive in their leadership. Overall, organizations need to create favorable conditions for their leaders’ change and development.
Data availability statement
The data that support the findings of this study are available on reasonable request from the corresponding author [AP]. The data are not publicly available due to privacy or ethical restrictions.
Appendices
Appendix
Appendix 1
Questionnaire items for managers’ demands and resources, managers’ readiness for change, and evaluation of intervention activities assessed by managers and trainers.
| VARIABLE | ITEM | RESPONSE RANGE |
|---|---|---|
| Managerial job demands | ||
| Organizational stressors | Your operation does not have the resources to cope with peak loads | 1–5 |
| You are unable to work according to your own judgement because of decisions made by superiors, politicians, or authorities | 1–5 | |
| You need to explain and justify ‘bad’ or ‘negative’ decisions made by superiors | 1–5 | |
| Role stressors | Having to work overtime or take work home | 1–5 |
| Tension between administrative work, business development, and contact with employees | 1–5 | |
| My responsibilities and mission as a manager are clear and defined (Reversed scored) | 1–5 | |
| Subordinate stressors | There are cooperation problems or conflicts between subordinates | 1–5 |
| Problems with security and mutual trust within the team | 1–5 | |
| Subordinates not working in the way you have agreed | 1–5 | |
| You feel that subordinates are working against you | 1–5 | |
| Subordinates have insufficient structure in their work | 1–5 | |
| Insufficient professional competence of subordinates | 1–5 | |
| Managerial resources | ||
| Superior support | Superiors show real interest in what I do and the problems I have as a manager | 1–5 |
| I trust superiors to help me, if necessary, to find a work situation that I can cope with | 1–5 | |
| Your superior manager’s requirements and expectations of you are unclear (Reversed scored) | 1–5 | |
| Your boss ignores your views and overrules you (Reversed scored) | 1–5 | |
| Peer Support | I can get good support from manager colleagues when needed | 1–5 |
| I have good opportunities to discuss and reason with manager colleagues about the operation | 1–5 | |
| Readiness for change | ||
| Change commitment | How important is it for you to develop as a manager? | 0–10 |
| Change efficacy | How much do you think you can succeed in developing as a manager within the framework of this leadership training? | 0–10 |
| Change effort | How willing are you to invest in this right now? | 0–10 |
| Assessment of intervention activities by managers | ||
| Behavior analysis | I have been supported in analyzing how I act (behave) in my leadership | 1–6 |
| I have been supported in examining how my leadership behaviors can change | 1–6 | |
| In-session behavior training | I have been supported in practicing behavioral changes through exercises | 1–6 |
| Any exercises during the session had a clear purpose | 1–6 | |
| These exercises have been relevant for developing my functional leadership behaviours | 1–6 | |
| Homework planning | I have been supported in concretizing one or more homework assignments that I will carry out before the next session | 1–6 |
| The homework assignment is clearly formulated in writing | 1–6 | |
| The homework assignment is relevant to my leadership | 1–6 | |
| Homework implementation | I have completed the homework as planned | 1–6 |
| Homework evaluation | I have been supported in evaluating how the homework functioned | 1–6 |
| Assessment of intervention activities by trainers | ||
| Behavioral analysis | I as a trainer have supported the client in analysing how he/she acts (behaves) in his/her leadership | 1–6 |
| I as a trainer have supported the client in examining how his/her leadership behavior can be changed | 1–6 | |
| In-session behavioral training | I as a trainer have supported the client in practicing behavioural changes through exercises | 1–6 |
| Any exercises during the session had a clear purpose | 1–6 | |
| These exercises have been relevant to developing the client’s functional leadership behaviors | 1–6 | |
| Homework planning | I as a trainer have supported the client in concretizing one or more homework assignments to be completed before the next counseling session | 1–6 |
| The homework assignment is clearly formulated in writing | 1–6 | |
| The homework assignment is relevant to the client’s leadership | 1–6 | |
| Homework implementation | The client has completed the homework assignment as planned | 1–6 |
| Homework evaluation | I as a trainer have supported the client in evaluating how the homework functioned | 1–6 |
Appendix 2
Questionnaire items for leadership performance, assessed by the employees of the participating managers.
| FACET | ITEM | RESPONSE RANGE |
|---|---|---|
| Leadership effectiveness | Is effective in representing our organization | 1–7 |
| Is effective in doing his/her job | 1–7 | |
| Is effective in making employees perform efficiently | 1–7 | |
| Is effective in leading employees | 1–7 | |
| Employee engagement | Makes employees exceed expectations | 1–7 |
| Makes employees want to succeed in my work | 1–7 | |
| Makes employees want to perform efficiently | 1–7 | |
| Employee satisfaction | Leads the activities for which he/she is responsible in a satisfactory way | 1–7 |
| Works with employees in a satisfactory way | 1–7 |
[i] Rate how frequently your manager accomplishes the following outcomes.
1 = never, 2 = almost never, 3 = occasionally, 4 = sometimes, 5 = often, 6 = almost always, and 7 = always.
Notes
[3] During the second half of round two, and at the time of the post-measurement for round two, several managers were not in physical contact with their subordinates. Entire organizations had to change and seek ways of working through online communication, which also necessitated online solutions for the leadership training sessions. Managers became preoccupied with problem-solving in a new reality, and crisis meetings to solve new situations were common. The situation resulted in a shift in focus, and leadership development was downgraded while urgent issues were given priority. There was also sizeable variation in how the pandemic affected the managers and their organizations.
Competing Interests
The authors have no competing interests to declare.