Table 1
Domains Measured on Original ART and New Assessment Tool.
| ASSESSMENT OF REASONING TOOL | NEW ASSESSMENT TOOL |
|---|---|
| Hypothesis-directed data collection | Not assessed (measured elsewhere on OSCE) |
| Problem representation | Problem representation |
| Prioritized differential diagnosis | Prioritized differential diagnosis |
| Direct evaluation/treatment towards high priority diagnoses | Initial management plan (as expected for graduating medical student) |
| Metacognition | Not assessed |
[i] OSCE: Observed Structured Clinical Examination.
Table 2
Mean score for each of the overall performance groups on each of the OSCE tasks.
| MEAN TASK SCORE FOR EACH PERFORMANCE GROUP | ||||||
|---|---|---|---|---|---|---|
| HISTORY TAKING | PHYSICAL EXAMINATION | CLINICAL REASONING | COMMUNICATION | DIFFICULT CONVERSATION | ||
| Examinee Group Based on Overall Performance in EOA OSCE | Pass (n = 106) | 89.6% | 83.2% | 86.2% | 94.1% | 87.9% |
| Borderline (n = 15) | 85.3% | 79.1% | 83.6% | 88.6% | 72.3% | |
| Fail (n = 15) | 83.2% | 78.1% | 68.6% | 86.3% | 65.0% | |
| ALL Examinees (n = 136) | 88.40% | 82.16% | 83.98% | 92.65% | 83.66% | |
| Eta2 effect size | 0.227 | 0.101 | 0.191 | 0.239 | 0.562 | |
[i] Scores are highlighted for instances when a group differed from other groups on a specific task (ANOVA [F(2,133)]; p < .05). Mean scores for History Taking and Physical Examination were calculated as the percentage of skills completed based on a validated checklist for each case. Mean scores for Communication and Difficult Conversations were calculated as a percentage of total available points based on a rating scale that was designed and validated for each case. Abbreviations: ART – Assessment of Reasoning Tool, EOA OSCE – End of Applications Objective Structured Clinical Encounter, ANOVA – analysis of variance.
Table 3
Mean sub-score for each of the overall performance groups on the 3 clinical reasoning components.
| MEAN SUB-SCORE FOR EACH PERFORMANCE GROUP | ||||
|---|---|---|---|---|
| DIFFERENTIAL DIAGNOSIS | PROBLEM REPRESENTATION | NEXT STEPS | ||
| Examinee Group Based on Overall Performance on the Clinical Reasoning Task | Pass (n = 107) | 92.6% | 87.9% | 87.5% |
| Borderline (n = 22) | 79.5% | 43.8% | 78.4% | |
| Fail (n = 7) | 75.0% | 12.5% | 76.8% | |
| ALL (n = 136) | 89.61% | 76.84% | 85.48% | |
| Eta2 effect size | 0.257 | 0.641 | 0.097 | |
[i] Scores are highlighted for instances when a group differed from other groups on a specific component (ANOVA [F(2,133)]; p < .05). Abbreviations: ART – Assessment of Reasoning Tool, EOA OSCE – End of Applications Objective Structured Clinical Encounter, ANOVA – analysis of variance.