Advancing knowledge for health and care integration- a kaleidoscopic evaluation approach

Background: The inherent complexity of health and care integration, in the context of dynamic social environments, is mirrored by the challenges associated with effective evaluation of these initiatives. How can independent evaluation studies capture and make sense of a range of diverse perspectives, in different organisational and social contexts over time, and use this knowledge to advance effective integration?

 

Approach: A metaphor often heard to depict the challenges of successful integration is that it is ‘fixing the plane whilst flying’. Appreciation of this complexity has shaped our integrated kaleidoscopic approach to the independent evaluation of the implementation of the Welsh Government’s Health and Social Care Regional Integration Fund (RIF), 2022-2027. RIF is a strategic lever for change, based on a vision to support health and social care integration to meet diverse needs across the life course (for individuals, families and communities). This evaluation has stitched together three evaluation approaches, one which considers principles (or guidance) and whether they are being adopted and working or not- ‘Principles-Focused Evaluation’ (Patton, 2018). The second through realistic evaluation approaches, uses mixed methods and ‘digs for nuggets of evidence’ which enables the evaluator to explain underlying causal factors. The third includes an economic evaluation, Social Return on Investment (SROI), with nested Social Cost-Benefit Analysis (SCBA).

 

Results: In this paper we outline the various components of the integrated evaluation approach, how they are stitched together and discuss the benefits and challenges of this process. As a team, we have used each approach independently and also synthesised insights from across the separate sub studies. The overall evaluation methodology is informed by Patton’s Principles-Focused Evaluation’ (Patton, 2018) congruent with the centrality of principles within the RIF. To date 5 evaluation reports have been published about various aspects of the effectiveness of the RIF.

 

 

 

Implications: Health and care integration agendas are being implemented in policy and practice contexts comprised of many different actors with different perspectives and values about the vision, pathways and outcomes for integration. This complexity has implications for the evaluation methods used to make sense of what happens in practice and why, and the results or outcomes. We argue an integrated evaluation approach is well suited to this task. However, this is not without its challenges given each method is rooted in different conceptions of what is knowledge and how ‘we know what we know’. It has required an openness amongst the evaluation team to talk and reflect about evaluation methods, structured facilitated workshops to synthesise evaluation findings, and methods to engage evaluation users along the way (e.g., stakeholder workshops).

Reference: Patton, M. (2018) Principles-Focused Evaluation: The Guide. The Guilford Press.