Table 1
Measuring Impact in Selected Public Health Training Programs.
| TITLE (YEAR) | EVALUATION STRATEGIES |
|---|---|
| Levels of Evaluation: Beyond Kirkpatrick (1994) [6] | Expands the Kirkpatrick Model by adding a fifth level concerned with societal impact and by slightly redefining some of the levels to apply to human performance interventions in general. |
| The Bass handbook of leadership: Theory, research, and managerial applications, 4th ed.(2004) [7] | The foundational approach to measuring training programs, Kirkpatrick’s Four-Level Model, first introduced in 1959. The four levels of the model include 1) reaction, 2) learning, 3) behavior, and 4) results. |
| The Results of an Evaluation Scan of 55 Leadership Development Programs (2004) [8] | Most assessments do not move past the first two levels, reaction and learning, stopping at program satisfaction and knowledge gained. |
| Management Matters: A Leverage Point for Health System Strengthening in Global Health (2015) [9] | An overview of about 2 dozen studies examining the link between management and health system performance. Results showed that training interventions can have influence, but none showed causal relationship between training and health outcomes. This review is limited in “size, scope and rigor” with a focus on process indicators, levels of satisfaction, and knowledge gained on self-reported set of specific competencies. |
| Evaluating the Impact of Leadership Development (2017) [10] | Evaluation of leadership interventions is evolving: a growing body of work that sees leadership as a “networked process.” A shift from competency-based developed to vertical development that supports thinking in more complex, systematic, and strategic ways. It emphasizes cultural responsiveness. |
| Using Social Network Analysis in Evaluation (2013) [11] | Social Network Analysis in evaluation is useful when the leadership initiative is expected to lead to observable changes in a network structure. This tool can help understand the network embedded within a program or initiative, in terms of its density, connectedness, balance, and/or centralization. |
| Measuring Leadership development: Quantify your program’s impact and ROI on organizational performance (2012) [12] | Return on investment (ROI) approaches to evaluating leadership development connect leadership development strategy and activities to a specific mission. The ROI approach focuses first on measuring individual reactions, learning and behaviors, then seeks to connect those changes directly to specific, measurable objectives, such as measures of improved patient or community health outcomes. |
Figure 1
Examples of program logic models: Panel A—Afya Bora Consortium program logic model and Panel B—The WomenLift Health Initiative program logic model.
Figure 2
Sample Theory of Change framework for Leadership Program Evaluation.
Table 2
Short-, medium- and long-term indicators and measurement strategies.
| INDICATORS | EVALUATION METHODS |
|---|---|
| Short-term (1–2 years) | |
|
|
| Medium-term (3–5 years) | |
|
|
| Long-term (5+ years) | |
|
|