Using Patient-Reported Outcomes to Assess Healthcare Quality: Toward Better Measurement of Patient-Centered Care in Cardiovascular Disease

PSYCHOMETRIC PROPERTY

DESCRIPTION

EXAMPLES

Validity

When the designated measure clearly quantifies the concept it seeks to represent. There are different types of validity, including: content validity (the PRO measures all relevant concepts that are important to patients); face validity (the questions make sense and are capturing what is intended); criterion validity (the measure is associated with other measures, external to the questionnaire, that quantify the same concept); and predictive validity (the measure is associated with subsequent clinical events as would be expected by clinical logic).

Example of content validity: Qualitative interviews with patients reveal that the concepts captured by the PRO align with the most common symptoms and impacts of the disease from patients’ perspectives. Example of criterion validity: When assessing the physical limitations of a patient through a PRO, the score of the PRO appropriately matches with the limitations found within a 6-minute walk test. Example of predictive validity: When assessing the symptoms or physical limitations of a patient through a PRO, the score of the PRO appropriately matches logical subsequent clinical events.

Reliability

Test-retest reliability demonstrates very comparable results in the PRO in patients whose condition has not changed. This is usually described by comparing mean scores using paired t-tests or intra-class correlation coefficients. Internal reliability or consistency describes the similarity of different items in measuring a conceptual domain, as quantified by Chronbach’s alpha.

When a PRO has good test-retest reliability, the measure’s scores are very similar when serially administered to patients who are stable between administrations (ie, the status/severity of their disease does not change).

Responsiveness/sensitivity

The opposite of reliability, a responsive PRO measure is one that changes directionally and proportionately to the magnitude of clinical change that a patient experiences.

A PRO for coronary artery disease should change substantially when a patient with Class III angina is rendered asymptomatic by coronary revascularization.

Interpretability

The ability to translate either cross-sectional or longitudinal scores on an instrument into a clinically interpretable construct that is meaningful to patients and providers.

PROs are built on a scale that often ranges between 0 and 100 but is not intuitively familiar to patients, providers, or regulators. Defining the clinical importance of changes (or scores) is among the most difficult and important features of a PRO so that the impact of novel therapies or the quality of a patient’s health status can be appreciated. For example, after an extensive series of studies examining the mean changes in KCCQ scores for patients (or their providers) who felt a small, moderate, or large clinical change had occurred, changes of 5, 10, and 20 points are considered to be small but clinically important, moderate to large, and large to very large clinical changes, respectively. As an example of the clinical interpretability of cross-sectional PRO assessments, SAQ Angina Frequency scores of 0-30, 31-60, 61-99, and 100 reflect daily, weekly, monthly, and no angina, respectively.37

Actionability

When information from the designated measure is useful in informing how to modify treatment.

While largely a consequence of the interpretability of a PRO, a key advantage of disease-specific PROs is that by directly measuring the severity of symptoms or limitations from a particular disease, there are often treatments than can be offered (or care escalated) for either severe symptoms/limitations or acute worsening. For example, worsening symptoms in heart failure might trigger better blood pressure control or more diuresis, whereas worsening angina might support more aggressive antianginal therapy or coronary revascularization.

Feasibility

To be practical to use in clinical care, it is essential that the PROs be as brief as possible while still maintaining the requisite psychometric properties outlined above.

Example(s): Efforts to reduce PROs—such as the SF-36 to 12 items, the SAQ from 19 to 7 items, or the KCCQ from 23 to 12 items—are intended to capture as much psychometrically sound information as possible in as few items as are necessary.

CARDIOVASCULAR DISEASE

EXAMPLES OF PATIENT-REPORTED OUTCOME

Atrial fibrillation

Atrial Fibrillation Effect on Quality-of-life (AFEQT), Atrial Fibrillation Quality of Life (AF-QoL)21, 22, 23

Coronary artery disease

Seattle Angina Questionnaire (SAQ)15, 16

Heart failure, cardiomyopathy

Kansas City Cardiomyopathy Questionnaire (KCCQ),16, 17 Minnesota Living with Heart Failure Questionnaire (MLWHFQ)^25,26

Peripheral arterial disease

Peripheral Artery Questionnaire (PAQ), Vascular Quality of Life Questionnaire (VASCUQOL)27, 28, 29

Stroke

Stroke Impact Scale,31 Stroke Specific Quality of Life Scale30, 31, 32