Table 1
Minimal requirements of the Dutch Association of Paediatrics for the creation of a CCC
|
The program director must coordinate the formation and the evaluation process of a CCC |
|
The CCC must meet at least twice a year |
|
The judgment of the group members about the residents must be delivered on paper before the actual meeting to ensure its objectivity |
|
The residents must provide relevant information about their progress before the meeting |
|
It is optional for a program director to be present at the meeting. If they choose not to join the meeting, they must receive a written report of the meeting |
|
It is optional to use input from people other than the members of the meeting |
|
Residents’ progress of clinical competence must be judged using CanMEDs competencies, EPAs, exposure to clinical presentations and non-clinical duties |
Table 2
Main observations
|
Design principles |
Cycle #1 |
Modification |
Cycle #2 |
|---|---|---|---|
|
1. Multiple assessment data and multiple perspectives |
Residents delivered multiple assessment data |
No modification needed |
Residents delivered multiple assessment data |
|
2. Shared mental model |
Discussions about the way of assessing residents |
GL gave the CCC members an additional explanation of the EPA levels and CANMED levels |
Still some discussions, but less than in first meeting |
|
3. Interaction during the meeting |
Time pressure |
Instead of twice a year 24 residents, the frequency changed to 4 times a year 12 residents |
No time pressure |
|
Group leader (GL) structured the meeting |
No modification needed |
GL structured the meeting | |
|
No equal participation of CCC members |
We advised the GL to actively invite silent members to speak |
GL actively invited silent members to speak up; more equal participation of the members | |
|
Extra |
– |
– |
No feedback loop |
Table 3
Main results from the interviews
|
Design principles |
Cycle #1 |
Modification |
Cycle #2 |
Quotes |
|---|---|---|---|---|
|
1. Multiple assessment data and multiple perspectives |
Not all members discussed resident performance with their colleagues before the meeting |
The group leader asked the CCC members to consult colleagues before the meeting |
More members consulted their colleagues and said they formed a broader picture about resident performance by doing so |
M6: ‘I did not consult my colleagues before the meeting, but I am going to schedule time before the next meeting to do so, because I think it is valuable’ |
|
2. Shared mental model |
Members learned from each other about different approaches of assessing from colleagues |
No modification needed |
Members learned from each other about different approaches of assessing from colleagues |
M6: ‘All supervision levels must be the same at every ward. So that we assess residents in the same way’ |
|
3. Interaction during the meeting |
Safe atmosphere |
No modification needed |
Safe atmosphere |
M3: ‘There was a really good atmosphere, I had the feeling I could say everything’ |
|
Some members were hesitant to give a negative opinion |
The CCC discussed this hesitation and discussed the need to give negative opinions when necessary |
The members were more comfortable in giving a negative opinion when necessary |
M2: ‘I cannot do that, can I? To somebody who is such a nice person, to give, to (…) I have the feeling I cannot do that!’ [give a negative opinion] | |
|
Extra |
Broad and rich picture about the performance of residents |
No modification needed |
Broad and rich picture about the performance of residents |
M1: ‘I have a broader image of the resident, especially because I now know his extracurricular activities. We did not know that before the introduction of the CCC’ |
|
They were concerned about the extent to which private matters of the resident should be discussed |
It was decided to ask the permission of residents before the meeting to discuss private matters |
Private matters were only discussed when residents gave their permission |
M4: ‘This level of feedback and assessment cannot be reached with feedback on paper. Now we were able to ask questions about somebody’s opinion and discuss this’ |
Table 4
Main results from the questionnaires
|
Design principles |
Cycle #1 |
Modification |
Cycle #2 |
Quotes |
|---|---|---|---|---|
|
1. Multiple assessment data and multiple perspectives |
All residents were able to deliver multiple assessment data prior to the CCC |
No modification needed |
All residents were able to deliver multiple assessment data prior to the CCC |
Q14: ‘Collecting multiple data points and reviewing those again is a good thing. It made me realise again what my points for improvement are’ |
|
Extra |
Residents felt that their performance was seriously discussed in the CCC |
None |
Residents felt that their performance was seriously discussed in the CCC |
Q8: ‘It creates a broad picture of you as a person, as a doctor, about your strengths and weaknesses and not just a picture from one rotation or from one supervisor’ |
|
All residents thought that there was too little time between the meeting, the feedback they got back and preparation for the next meeting |
Feedback was delivered as soon as possible after the second meeting |
They were satisfied with the early delivery of feedback and felt they had more time to work on the feedback before the next meeting |
Q6: ‘The second CCC was too soon after the feedback from the first CCC. Therefore, the feedback from the second meeting was still the same’ | |
|
Some residents were not satisfied with the content of the feedback they received after the meeting |
None (because this was outside the scope of our study) |
Some residents were not satisfied with the content of the feedback they received after the meeting |
Q11: ‘Feedback should be founded on concrete examples of behaviour, not on vague remarks like: ‘I had the feeling that […]’ Then it is just one person’s opinion | |
|
Q3: ‘The feedback was exactly the same as during the latest rotation. I did not feel like the group added something to the opinion of my most recent supervisor’ |