Have a personal or library account? Click to login
Implementation, Mechanisms of Effect and Context of an Integrated Care Intervention for Vulnerable Families in Central Sydney Australia: A Research and Evaluation Protocol Cover

Implementation, Mechanisms of Effect and Context of an Integrated Care Intervention for Vulnerable Families in Central Sydney Australia: A Research and Evaluation Protocol

Open Access
|Jul 2019

Figures & Tables

ijic-19-3-4217-g1.png
Figure 1

Theory of Change – HHAN Early Intervention and Clinical Elements.

Table 1

NSW Health Monitoring and Evaluation Approach [1].

Key stepsOutputs
Formative evaluation components
Discovery and planning through program logic
  • Detailed program overview of activities and expected outcomes

  • Key assumptions about how change will occur

  • Anticipated outputs and outcomes

Development of relevant indicators
  • Process indicators and metrics recognising that both local and state-wide indicators exist

  • Progressively develop new data collection mechanisms

Development of purposeful road maps
  • Develop road map milestones based on key evaluation questions emerging from the logic maps

  • Develop milestones that reflect indicators, both qualitative and quantitative, that allow assessment of actual outcomes relative to expected outcomes

Definition of key functional components of integrated care
  • Common framework of functional components to facilitate the development and capture of core indicators

Summative evaluation components
Design of reporting approach
  • Quarterly output/outcomes reports for discussion at local health district (LHD) performance meetings and integrated care governance committees

  • Annual outcome evaluation reports

Identification of data sources
  • Identify all monitoring and evaluation data sources

  • Use routine data collection wherever possible

The philosophy of continuous improvement
  • Continuous improvement strategy based on Plan-Do-Study-Act (PDSA) cycle approach

  • Changes to program direction or arrangements based on reflection on monitoring results and outcome reports – what is working and what is not

Detailed design and execution of evaluation approach
  • Appropriateness, effectiveness, efficiency assessed at different stages of the program to determine immediate, intermediate and longer-term outcomes

ijic-19-3-4217-g2.png
Figure 2

Key elements of the development and evaluation process [7].

ijic-19-3-4217-g3.png
Figure 3

Intervention and Program Theory.

ijic-19-3-4217-g4.png
Figure 4

Key elements of the development and evaluation process, adapted from [7].

Table 2

Phases, Methods and Proposed projects and activities.

PhaseMethodsProjects/activities
Development (Operationalization)
  • Identifying the layered domains or strata

  • Identifying the mechanisms and evidence base

  • Undertaking collaborative design

  • Defining the intervention and program theory

  • Modelling the process and outcomes

  • Building realist causal theory [17]

  • Building realist program theory [18]

  • Designing initiatives for vulnerable families [3]

  • Designing HHAN integrated care initiative [2]

  • Systematic literature reviews

  • Meta-narrative and realist synthesis reviews.

  • Building the detailed HHAN Logic Model.

Feasibility/piloting (Contextualisation)
  • Define historical and current context

  • Define instrumentation and testing procedures

  • Assess acceptance by people, practitioners and the system

  • Determine parameter estimates

  • Delphi study of HHAN context

  • Define the HHAN intervention indicator KPI data set

  • Define and test HHAN patient reported measures

  • Data-linkage studies including GIS and Epidemiology studies

  • Base-line qualitative and mixed method studies of each HHAN intervention component.

Evaluation
  • Program theory evaluation

  • Intervention theory evaluation

  • Effectiveness evaluation

  • Understanding the change process

  • Assessing cost-effectiveness

  • Realist qualitative and mixed-method HHAN studies, including:

    • Partner-level studies

    • Place-based studies (including practitioners and consumer studies)

  • Quantitative modelling studies of:

    • Patient reported measures

    • Data-linkage studies.

  • Consideration of control designs for clinical component.

Implementation (Theory Testing and Refining)
  • Dissemination and scale-up

  • Longitudinal realist/action evaluation

  • System modelling

  • Surveillance and monitoring

  • Long-term follow-up

  • Longitudinal HHAN intervention evaluation. Including monitoring of KPIs, system modelling, and ongoing qualitative interviews

  • Longitudinal mixed method study, including HHAN PDSA cycles and monitoring of HHAN PRMs.

DOI: https://doi.org/10.5334/ijic.4217 | Journal eISSN: 1568-4156
Language: English
Submitted on: Sep 2, 2018
|
Accepted on: May 21, 2019
|
Published on: Jul 26, 2019
Published by: Ubiquity Press
In partnership with: Paradigm Publishing Services
Publication frequency: 4 issues per year

© 2019 John G. Eastwood, Susan Woolfenden, Erin Miller, Miranda Shaw, Pankaj Garg, Hueiming Liu, Denise E. De Souza, Roelof G.A. Ettema, published by Ubiquity Press
This work is licensed under the Creative Commons Attribution 4.0 License.