Table 1
Sequentially ordered key factors scored during the patient encounters
|
1 Polydipsia |
2 Duration |
|
3 How much drinking |
4 Urinary frequency |
|
5 Nocturia |
6 Urine volume |
|
7 UTI symptoms |
8 Weight changes |
|
9 Polyphagia |
10 Blurred vision |
|
11 Paresthesias |
12 Sexual dysfunction |
|
13 Past medical history |
14 CAGE (alcohol use screening) |
|
15 Alcohol use |
16 Dyspnoea on exertion |
|
17 Short of breath |
18 Family history of DM |
|
19 Family history of CAD |
20 Previous diagnosis of DM |
CAD coronary artery disease; CAGE cutback, angry, guilty, eye-opener; DM diabetes mellitus; UTI urinary tract infection
Fig. 1
Demonstrates a visual representation of the method to compute a coherence score. First, a checklist sequence is generated from a student/patient encounter, and a clinician educator (physician) independently generates a global rating score for the overall student performance. All of the sequences for a given global rating (2.5 in this example) are put into an average ordering matrix, and the probabilities of any key factor occurring after another are calculated. Finally, a coherence score for each individual checklist sequence associated with a specific global rating is calculated. In this example, the coherence score of (4, 3, 1, 5) is 0.3125 for a global rating of 2.5
Fig. 2
Compact graphical representation of the average-ordering matrices for each of the ratings from 0 to 5. Each cell (i, j) in matrices is the probability that key factor j is asked later than key factor i in a consult that received that rating. Darker cells indicate a higher probability of i preceding j. The 20 key factors are ordered in the matrices in the most logical order for the encounter (see Table 1), hence the right upper triangle of the matrices are expected to be darker in the matrices with higher ratings. The matrix for grade 0.5 is missing since no encounter received this rating
Table 2
Pearson’s correlation coefficients and linear regression analysis between global rating scores and coherence scores for each of the average-ordering matrices. Matrix ‘0’ stands for the average ordering matrix computed for students with global rating 0
|
Matrix |
Correlation |
Intercept |
Slope |
P intercept |
P slope |
R2 |
|---|---|---|---|---|---|---|
|
0 |
0.132 |
1.64 |
0.069 |
< 0.001 |
0.112 |
0.017 |
|
1 |
0.115 |
1.46 |
0.030 |
< 0.001 |
0.169 |
0.013 |
|
1.5 |
0.002 |
2.01 |
0.001 |
< 0.001 |
0.983 |
< 0.001 |
|
2 |
0.243 |
1.79 |
0.064 |
< 0.001 |
0.003 |
0.058 |
|
2.5 |
0.399 |
1.75 |
0.116 |
< 0.001 |
< 0.001 |
0.159 |
|
3 |
0.478 |
1.95 |
0.183 |
< 0.001 |
< 0.001 |
0.228 |
|
3.5 |
0.501 |
1.79 |
0.201 |
< 0.001 |
< 0.001 |
0.250 |
|
4 |
0.618 |
1.88 |
0.262 |
< 0.001 |
< 0.001 |
0.382 |
|
4.5 |
0.490 |
2.47 |
0.299 |
< 0.001 |
< 0.001 |
0.240 |
|
5 |
0.519 |
2.24 |
0.308 |
< 0.001 |
< 0.001 |
0.270 |
Fig. 3
Represents a distribution of the coherence scores within each global rating score (consult grade) as a function of average ordering matrix score (matrix grade). Within a panel, a distribution of boxplots is produced for the coherence scores for that average ordering matrix per global rating score
Table 3
Pearson’s correlation coefficients analysis between overall checklist scores and coherence scores for each of the average-ordering matrices. Matrix ‘0’ stands for the average ordering matrix computed for students with global rating 0
|
Matrix |
Correlation |
|---|---|
|
0 |
0.304 |
|
1 |
0.293 |
|
1.5 |
−0.081 |
|
2 |
0.135 |
|
2.5 |
0.313 |
|
3 |
0.484 |
|
3.5 |
0.398 |
|
4 |
0.520 |
|
4.5 |
0.489 |
|
5 |
0.488 |