Table 1
Feedback quality rating scale
|
Score |
Description |
Example |
|---|---|---|
|
1 |
Judgmental Non-specific praise Appearance only |
‘Very well done.’ ‘Smartly dressed.’ ‘Good.’ |
|
2 |
Description of performance OR Suggestion for improvement |
‘Very nervous.’ ‘Estimate very low.’ ‘Could slow down slightly on technique.’ |
|
3 |
Description of performance AND Suggestion for improvement |
‘Make sure you have short fingernails, skirt a little on the short side. Good technique for hand washing.’ ‘Use soap! Answer the question. Very nervous. Nice manner despite borderline result.’ |
|
4 |
Objective appraisal of performance |
‘Poor performance with no demonstration to patient and no attempt to correct poor patient technique nor ensure that the patient can read the meter reading.’ ‘Slightly chaotic, disorganized sequence of examination. However covered most areas and those that were covered were done well. Good rapport with patient.’ |
|
5 |
Objective appraisal of performance AND Suggestion for improvement |
‘Done in less than 1.25 min. Lots of white. Did notions but not thoroughly. Suggestion go slower and be more thorough.’ ‘Full marks. One of very few to observe the patient. Telling me what he’s doing but not findings (as many others). Very efficient, could be a bit gentler e.g. turning arms over. Told patient to breathe consistently—very good!’ |
Table 2
Quantities of comments provided in formative and summative assessments, 2012 and 2013
|
Exam style |
Date |
Number and percentage of examiner-student interactions where a comment was made (%) |
Mean number of words |
Standard deviation |
Total number and percentage of examiner-student interactions where a comment was made (%) |
Total mean number of words |
|---|---|---|---|---|---|---|
|
Paper- based |
Jan 2012 |
558 (68) |
19 |
14 |
1960 (41) |
16 |
|
May 2013 |
1402 (30) |
13 |
10 | |||
|
iPad-based |
Jan 2012 |
498 (66) |
15 |
11 |
1842 (51) |
13 |
|
May 2013 |
1344 (45) |
12 |
8 |
Table 3
Quantity of feedback received related to global score
|
Total number of students receiving Global Score |
Written (Jan and May 2012) |
iPad (Jan and May 2013) | |
|---|---|---|---|
|
5 ‘Excellent’ |
Number and percentage of students receiving feedback |
111 (n = 299) 37 % |
148 (n = 270) 55 % |
|
Mean number of words of feedback |
12 |
9 | |
|
4 ‘Highly satisfactory’ |
Number and percentage of students receiving feedback |
212 (n = 622) 34 % |
235 (n = 539) 44 % |
|
Mean number of words of feedback |
14 |
13 | |
|
3 ‘Satisfactory’ |
Number and percentage of students receiving feedback |
296 (n = 736) 40 % |
317 (n = 724) 44 % |
|
Mean number of words of feedback |
17 |
13 | |
|
2 ‘Borderline’ |
Number and percentage of students receiving feedback |
113 (n = 219) 52 % |
179 (n = 242) 74 % |
|
Mean number of words of feedback |
18 |
15 | |
|
1 ‘Unsatisfactory’ |
Number and percentage of students receiving feedback |
74 (n = 84) 88 % |
58 (n = 67) 87 % |
|
Mean number of words of feedback |
18 |
19 | |
Table 4
Mean and standard deviation of the iPad versus written OSCE comments, when rated by the Feedback Quality Scale
|
Comment Feedback Quality Score | ||||||
|---|---|---|---|---|---|---|
|
Jan 2012 (written) |
Jan 2013 (iPad) |
May 2012 (written) |
May 2013 (iPad) |
Overall Written |
Overall iPad | |
|
Mean |
2.36 |
2.56 |
2.02 |
2.32 |
2.18 |
2.41 |
|
SD |
0.95 |
1.02 |
0.56 |
0.81 |
0.78 |
0.89 |
|
N |
381 |
330 |
425 |
607 |
806 |
937 |