Fig. 1
Variance in difficulty coefficients (Panel A) and discrimination coefficients (Panel B) across repeat use of 32 MCQ items across 22 small student clerkship cohorts (n = 10–15 students). Error bars represent range
Fig. 2
Variability in difficulty and discrimination indices for a single MCQ item (Item 6) graphed across cohort
Table 1
Frequency of items being categorized in each of Ebel and Frisbie’s [13] categories, displayed for each item
|
Discrimination categorization | |||||
|---|---|---|---|---|---|
|
Item |
Poor |
Low |
Acceptable |
Good |
Excellent |
|
1 |
12 |
0 |
2 |
2 |
6 |
|
2 |
19 |
1 |
0 |
0 |
2 |
|
3 |
20 |
1 |
0 |
1 |
0 |
|
4 |
12 |
4 |
3 |
2 |
1 |
|
5 |
14 |
0 |
0 |
5 |
3 |
|
6 |
10 |
1 |
4 |
2 |
5 |
|
7 |
11 |
2 |
1 |
2 |
6 |
|
8 |
14 |
2 |
2 |
2 |
2 |
|
9 |
12 |
1 |
3 |
3 |
3 |
|
10 |
16 |
3 |
1 |
1 |
1 |
|
11 |
14 |
1 |
3 |
1 |
3 |
|
12 |
7 |
1 |
4 |
2 |
8 |
|
13 |
5 |
4 |
2 |
5 |
6 |
|
14 |
12 |
2 |
2 |
1 |
5 |
|
15 |
8 |
4 |
2 |
1 |
7 |
|
16 |
19 |
0 |
1 |
0 |
2 |
|
17 |
18 |
1 |
1 |
2 |
0 |
|
18 |
15 |
0 |
2 |
0 |
5 |
|
19 |
13 |
3 |
0 |
0 |
6 |
|
20 |
11 |
3 |
1 |
1 |
6 |
|
21 |
19 |
0 |
0 |
1 |
2 |
|
22 |
5 |
1 |
3 |
3 |
10 |
|
23 |
16 |
0 |
1 |
2 |
3 |
|
24 |
19 |
0 |
1 |
1 |
1 |
|
25 |
20 |
0 |
1 |
1 |
0 |
|
26 |
9 |
2 |
0 |
5 |
6 |
|
27 |
12 |
1 |
1 |
1 |
7 |
|
28 |
8 |
3 |
5 |
2 |
4 |
|
29 |
9 |
6 |
2 |
2 |
3 |
|
30 |
9 |
3 |
1 |
1 |
8 |
|
31 |
9 |
1 |
2 |
2 |
8 |
|
32 |
13 |
2 |
2 |
1 |
4 |
Table 2
Frequency of items either removed or kept for the total exam score, presented by item, and for each quality monitoring guidelinea
|
Item |
Number of times item is removed from the total score applying Guideline 1 |
Number of times item is removed from the total score applying Guideline 2 |
Number of times item is removed from the total score applying Guideline 3 |
|---|---|---|---|
|
1 |
0 |
7 |
18 |
|
2 |
0 |
18 |
21 |
|
3 |
0 |
17 |
22 |
|
4 |
9 |
1 |
1 |
|
5 |
0 |
11 |
21 |
|
6 |
17 |
3 |
2 |
|
7 |
0 |
1 |
14 |
|
8 |
0 |
12 |
22 |
|
9 |
0 |
10 |
21 |
|
10 |
0 |
14 |
22 |
|
11 |
0 |
3 |
11 |
|
12 |
2 |
0 |
7 |
|
13 |
0 |
2 |
10 |
|
14 |
0 |
6 |
20 |
|
15 |
0 |
1 |
11 |
|
16 |
0 |
15 |
21 |
|
17 |
0 |
15 |
22 |
|
18 |
0 |
14 |
22 |
|
19 |
1 |
6 |
18 |
|
20 |
2 |
1 |
6 |
|
21 |
0 |
15 |
22 |
|
22 |
2 |
1 |
6 |
|
23 |
0 |
14 |
22 |
|
24 |
0 |
17 |
21 |
|
25 |
0 |
15 |
22 |
|
26 |
3 |
1 |
3 |
|
27 |
0 |
8 |
19 |
|
28 |
1 |
1 |
4 |
|
29 |
0 |
4 |
20 |
|
30 |
6 |
1 |
2 |
|
31 |
0 |
1 |
11 |
|
32 |
0 |
12 |
21 |
aGuideline 1: an item should be excluded from the examination if the item difficulty was ± two standard deviations from the average difficulty of the examination. Guideline 2: an item should be excluded if it fell ± two standard deviations from the passing score. Guideline 3: an item should be excluded if difficulty was less than 0.2 or more than 0.8 [15].