Abstract
Health Promotion and Improvement (HP&I) programmes are delivered at regional level through the public healthcare system in the Republic of Ireland. The breadth of work targets health inequalities and prevention of health conditions for local at-risk populations, and is sometimes delivered in collaboration with other health and social care professionals or community partners. Programmes are generally evidence-based, but not subject to mandatory systematic evaluation post-implementation.
HP&I practitioners often have diverse professional backgrounds, with varying levels of experience in evaluation. It is vital that research/evaluation remain core aspects of HP&I professional practice¹. This ensures that programme delivery is effective within local contexts and that practice-based evidence is produced for improving population health.
We commissioned the development of an evaluation framework to build capacity for designing evaluation across HP&I programmes within our region. We convened a steering group representing stakeholders across care groups, organisations and sectors. An external consultancy (S3 Solutions) was selected to undertake the work, and consultations were undertaken with local HP&I personnel. Two dimensions of an Evaluability Assessment² were used, which involved developing logic models for each category of work and prioritising indicators by value and accessibility.
We also undertook a baseline survey with HP&I personnel to assess change-readiness for evaluation within the team, and research self-efficacy³. Further, public input was sought at draft stage, to ensure that the framework considered their perspective. This was achieved through two informal consultations with six service-users (stop smoking service and a community wellbeing programme).
The evaluability assessment led to a final set of indicators for each work category. An overarching framework was conceived, drawing from existing frameworks⁴ ⁵, but bespoke to our work. These provides a template for future evaluation design, with five domains: Reach, Experience, Impact, Maintenance and Value.
Median scores were high for most aspects of change-readiness within the team with respect to implementing an evaluation framework, but scores were low-medium for many aspects of research self-efficacy, particularly for evaluation design.
Service-users highlighted important additional considerations around data sensitivity, modes of data collection, and person-centred indicators.
Implementation of this work will commence shortly, highlighting the framework’s usability and allowing standardisation of our evaluation process. This will create an efficient evaluation process, and inform decision-making and resource prioritisation. Further, scaling up will enhance our ability to coordinate the effective delivery of programmes that service the community through multiple collaborations.
1.Barry MM, Battel-Kirk B, Dempsey C. The CompHP core competencies framework for health promotion in Europe. Health Education & Behavior. 2012 Dec;39(6):648-62.
2.Davies R. Planning evaluability assessments: A synthesis of the literature with recommendations. Report of a study commissioned by the Department for International Development. 2013 Sep 16.
3.Holden G, Barker K, Meenaghan T, Rosenberg G. Research self-efficacy: A new possibility for educational outcomes assessment. Journal of social work education. 1999 Oct 1;35(3):463-76.
4.Michie S, Van Stralen MM, West R. The behaviour change wheel:a new method for characterising and designing behaviour change interventions. Implementation science. 2011 Dec;6(1):1-2.
5.Re-Aim (2023) What is PRISM? [Accessed 22.11.23] Availa