Supporting the Evaluation of Patient Engagement in Ontario Health System Organizations

Background: Patient engagement has become a common practice across health system organizations, yet there is limited evidence of these organizations conducting thorough evaluations of their engagement efforts. Evaluation is crucial to identify successful engagement practices, understand how methods can be adapted to different settings, explore patient partner experiences, and overcome barriers to engagement; therefore, this study examined the challenges faced by health system organizations in evaluating their patient engagement initiatives and the factors that support effective evaluation practices.

 

Approach: A qualitative description methodology was employed. 32 semi-structured interviews were completed with individuals who were involved in designing, organizing, and implementing the evaluation of patient engagement initiatives across Ontario.

 

Participants worked in a variety of roles, including patient engagement specialists, patient partners in leadership positions (e.g., committee chairs or co-chairs), health system managers, and quality improvement specialists within diverse health system organizations, such as Ontario Health Teams (OHTs), hospital networks, community health organizations, and government organizations.

 

Participants were recruited via email, and purposive sampling ensured maximum variation in participants. Interviews lasted 30–60 minutes and explored topics such as approaches to and methods for evaluating patient engagement initiatives, challenges associated with evaluation, supports required to foster evaluation, and organizational culture related to continuous learning and improvement. Interviews were recorded, transcribed verbatim, inductively coded and thematically analyzed.

 

Results: Several key themes were identified, including how organizations approach patient engagement evaluation work, significant challenges that organizations encountered when carrying out their evaluation work and key supports which helped to foster their evaluation efforts. The first theme was whether and how organizations are carrying out evaluation work. Several organizations indicated that they did not complete patient engagement evaluation work, and few organizations had formal evaluation plans, often relying on informal methods like casual check-ins for feedback. A second theme was challenges in evaluating patient engagement, including hesitancy, limited patient partner capacity, and resource constraints like time, funding, and personnel. A third theme was the need for organizational supports to foster evaluation efforts.

 

Participants highlighted the value of evaluation training, staff with evaluation expertise, and senior leadership support. Lastly, organizational culture was identified as a significant influence on evaluation. Participants highlighted the importance of a culture that values openness to change, continuous improvement, and collaboration with quality improvement staff to enable more effective evaluation efforts within health system organizations.

 

Implications: Health system organizations face significant challenges in evaluating patient engagement, with few having robust evaluation frameworks. Addressing these gaps requires training, dedicated resources, leadership support, and formalization of evaluation methods. Strengthening evaluation capacity is crucial for fostering meaningful patient partnerships and advancing continuous learning and improvement in health system organizations.