Table 1
Mean test results (% correct–incorrect) per training year and number of failures per training year
|
Training year |
N |
Min |
Max |
Mean |
SD |
Fail |
|---|---|---|---|---|---|---|
|
1 |
49 |
15 |
52 |
32 |
8.5 |
0 |
|
2 |
46 |
23 |
60 |
41 |
8.1 |
1 |
|
3 |
41 |
29 |
66 |
45 |
9.5 |
0 |
|
4 |
51 |
−14 |
63 |
46 |
12 |
1 |
|
5 |
24 |
27 |
59 |
45 |
7.4 |
1 |
|
6 |
51 |
28 |
67 |
47 |
8.7 |
0 |
Table 2
Mean number of correct, incorrect and question mark answers per training year
|
Training year |
Correct mean number |
Incorrect mean number |
Question mark mean number |
|---|---|---|---|
|
1 |
64 |
46 |
41a |
|
2 |
78 |
45 |
28 |
|
3 |
83 |
42 |
25 |
|
4 |
85 |
45 |
20 |
|
5 |
85 |
45 |
21 |
|
6 |
89 |
50 |
12 |
aThe questionnaire consisted of 151 questions in 2010
Table 3
Results of trainees’ and educational supervisors’ questionnaire on acceptability and educational impact
|
Supervisor: mean (n, SD) |
Trainees: mean (n, SD) |
Significance Mann–Whitney test (p) |
Training year |
Significance One-way Anova + post hoc test | ||||||
|---|---|---|---|---|---|---|---|---|---|---|
|
1 |
2 |
3 |
4 |
5 |
6 | |||||
|
General acceptance | ||||||||||
|
The progress test should continue to be obligatory |
4.5 (38, 0.9) |
3.4 (205, 1.2) |
0.000 |
3.8 |
3.5 |
3.3 |
3.0 |
3.2 |
3.2 |
0.118 |
|
I believe that our society deserves an adequate exam policy during medical specialist training |
4.2 (38, 0.9) |
3.7 (206, 1.0) |
0.000 |
3.6 |
3.7 |
4.0 |
3.5 |
3.8 |
3.5 |
0.393 |
|
The pass–fail standard is: 1. to low, 2. fair, 3. to high |
1.8 (32, 0.4) |
2.1 (158, 0.4) |
0.000 |
2.1 |
2.2 |
2.1 |
2.2 |
2.0 |
1.9 |
0.088 |
|
This kind of testing within adult education is belittling |
1.7 (38, 0.8) |
2.4 (236, 1.1) |
0.000 |
1.8 |
2.4 |
2.5 |
2.6 |
2.3 |
2.6 |
0.004 (6 + 4-rest) |
|
Educational impact | ||||||||||
|
The progress test is a major assessment of my development/the development of the trainee |
3.6 (38, 0.7) |
2.7 (214, 1.0) |
0.000 |
3.1 |
2.9 |
2.5 |
2.6 |
2.3 |
2.4 |
0.001 (1–2) |
|
The progress test could be helpful in compiling my learning course/the learning course of the trainee together with my supervisor |
4.0 (38, 0.9) |
3.0 (236, 1.1) |
0.000 |
3.6 |
3.2 |
2.7 |
2.8 |
2.8 |
3.0 |
0.01 (1–2) |
|
The results of the progress test should be discussed during the in-training assessment with the educational supervisor |
4.7 (38, 0.5) |
3.6 (234, 0.9) |
0.000 |
3.8 |
3.6 |
3.4 |
3.6 |
3.5 |
3.4 |
0.519 |
|
The progress test helps me to get a good impression of the functional knowledge of the trainee |
3.5 (38, 0.9) |
3.2 (206, 1.0) |
n.s. |
3.8 |
3.5 |
2.9 |
3.0 |
2.9 |
2.9 |
0.000 (1–2) |
|
The results of the progress test increase my self-confidence |
2.7 (228, 1.0) |
n.a. |
2.6 |
2.7 |
2.8 |
2.6 |
2.8 |
2.8 |
0.864 | |
|
I use the progress test to compile my learning course together with my educational supervisor |
3.4 (38, 1.1) |
1.9 (235, 0.9) |
0.000 |
2.5 |
1.9 |
1.7 |
2.0 |
1.6 |
1.8 |
0.000 (1 + 4-rest) |
|
The progress test is a predominant part of my in-training assessment |
2.5 (38, 0.9) |
1.8 (205, 0.8) |
0.000 |
2.2 |
1.7 |
1.7 |
2.0 |
1.6 |
1.5 |
0.001 (1 + 4-rest) |
|
The results on the different test domains influence my study activities |
3.0 (205, 1.1) |
n.s. |
3.7 |
3.1 |
2.7 |
3.0 |
2.7 |
2.6 |
0.000 (1 + 4-rest) | |
|
The results of the progress test influence my training programme |
3.0 (38, 1.0) |
n.s. | ||||||||
|
I/trainees need this kind of stimulus in order to study |
4.2 (38, 0.8) |
2.7 (235, 1.3) |
0.000 |
2.7 |
3.0 |
2.5 |
2.8 |
2.8 |
2.7 |
0.496 |
|
The test results should have more serious consequences for the training progression of a trainee |
2.9 (38, 1.1) |
2.0 (206, 1.0) |
0.000 |
1.9 |
1.9 |
1.9 |
2.0 |
1.8 |
2.2 |
0.719 |
|
Supervisor: mean (n, SD) |
Trainees: mean (n, SD) |
Training year |
Significance One-way Anova + post hoc test | ||||||
|---|---|---|---|---|---|---|---|---|---|
|
1 |
2 |
3 |
4 |
5 |
6 | ||||
|
Acceptability of test content | |||||||||
|
The progress test properly reflects the whole domain of O&G |
– |
3.4 (234, 1.0) |
3.6 |
3.3 |
3.2 |
3.2 |
3.4 |
3.4 |
0.563 |
|
The distribution of the questions over the sub-domains of O&G reflects daily practice |
– |
3.5 (204, 1.0) |
3.3 |
3.2 |
3.6 |
3.4 |
3.2 |
3.2 |
0.580 |
|
The progress test is a good instrument to test knowledge |
– |
2.9 (205, 1.1) |
3.5 |
3.2 |
2.7 |
2.8 |
2.8 |
2.6 |
0.000 (1–2) |
|
The progress test tests the knowledge that is needed to be able to work as a gynaecologist |
– |
2.4 (236, 1.0) |
3.0 |
2.5 |
2.2 |
2.3 |
2.3 |
2.3 |
0.002 (1–2) |
n.s. not significant
Table 4
Results open question: how to improve postgraduate progress testing—trainees
|
Test content |
|
Less factual/percentage questions (46) |
|
Different question format (23) |
|
More daily practice orientated questions (20) |
|
Test questions should be of irrefutable content (13) |
|
Free internet access (‘open book’) (12) |
|
Clear description of study material 10) |
|
Educational impact |
|
Add literature references (8) |
|
Release answers (5) |
|
Unsatisfactory test scores should have consequences (4) |
Table 5
Results of open question: how to improve test graduate progress testing—educational supervisors
|
Test content |
|
Add open questions (3) |
|
Yearly evaluation progress test by trainees (2) |
|
Free internet access (‘open book’)(2) |
|
Add questions that assess profound understanding (1) |
|
Clear description of study material (1) |
|
Test questions should be relevant in daily clinical practice (1) |
|
Improve the array of questions: from simple to complex (1) |
|
Educational impact |
|
Test results should have consequences (4) |
|
Organize test result evaluation sessions with educational supervisors (3) |
|
Test should be obligatory to consultant obstetricians and gynaecologists as a part of re-certification (2) |