Fig. 1
Sample MSF report (translated from Dutch)
Table 1
MSF items, categorized in CanMEDS roles, and information sources
|
Item |
Role |
Information source | ||
|---|---|---|---|---|
|
I know this doctor well enough to evaluate his/her work |
Physicians |
Allied health | ||
|
Please give one or more recommendations so this doctor can function (even) better |
Physicians |
Allied health |
Patients | |
|
Do you know this doctor from the clinic, the ward or both? |
Physicians |
Allied health |
Patients | |
|
How long is the period you have been able to observe the resident? |
Physicians |
Allied health | ||
|
How many times did you see this doctor? |
Patients | |||
|
I would entrust this colleague with conducting a bad-news conversation |
Physicians | |||
|
I would entrust this colleague with conducting a peer consultation |
Physicians | |||
|
This doctor diagnoses patient problems effectively |
Expert |
Physicians | ||
|
This doctor deals with patient problems independently |
Expert |
Physicians | ||
|
This doctor weighs cost-benefit considerations about diagnostics and treatment |
Expert |
Physicians | ||
|
This doctor communicates adequately with patients and their family |
Communicator |
Physicians |
Allied health | |
|
This doctor communicates adequately with colleagues |
Communicator |
Physicians |
Allied health | |
|
This doctor is open to verbal and non-verbal reactions and emotions |
Communicator |
Physicians |
Allied health | |
|
This doctor presents a patient case accurately and briefly during a meeting |
Communicator |
Physicians | ||
|
This doctor gives clear and complete oral commands |
Communicator |
Allied health | ||
|
This doctor gives clear and complete written commands |
Communicator |
Allied health | ||
|
This doctor is clear and complete in written report |
Communicator |
Physicians |
Allied health | |
|
This doctor showed understanding for me |
Communicator |
Patients | ||
|
This doctor showed empathy with me |
Communicator |
Patients | ||
|
This doctor took me seriously |
Communicator |
Patients | ||
|
This doctor listened carefully to me |
Communicator |
Patients | ||
|
This doctor asked understandable questions |
Communicator |
Patients | ||
|
This doctor gave clear explanations |
Communicator |
Patients | ||
|
This doctor gave clear information about investigations and treatment |
Communicator |
Patients | ||
|
This doctor verified whether I understood everything right |
Communicator |
Patients | ||
|
This doctor hands over patient care effectively and carefully |
Collaborator |
Physicians |
Allied health | |
|
This doctor consults other caregivers in a timely manner |
Collaborator |
Physicians |
Allied health | |
|
This doctor appreciates and respects the knowledge and experience of others |
Collaborator |
Physicians |
Allied health | |
|
This doctor adheres to agreements and to an agreed policy |
Collaborator |
Physicians |
Allied health | |
|
This doctor takes responsibility for his/her own actions |
Collaborator |
Physicians |
Allied health | |
|
This doctor has found an adequate balance of patient care and personal development |
Manager |
Physicians | ||
|
This doctor coordinates the care of patients effectively |
Manager |
Physicians |
Allied health | |
|
This doctor is reachable and accessible |
Manager |
Physicians |
Allied health | |
|
This doctor organizes time effectively with the correct priorities |
Manager |
Physicians |
Allied health | |
|
This doctor timely passes on agreements and/or changes in plans |
Manager |
Allied health | ||
|
This doctor is willing and able to teach or train others |
Scholar |
Physicians |
Allied health | |
|
This doctor acts in an evidence-based manner if possible |
Scholar |
Physicians |
Allied health | |
|
This doctor provides feedback to others correctly |
Scholar |
Physicians |
Allied health | |
|
This doctor puts the interests of patients first during daily work |
Professional |
Physicians |
Allied health | |
|
This doctor takes patients privacy into account during physical examination |
Professional |
Physicians |
Allied health | |
|
This doctor knows the condition of the patient and the patient record information |
Professional |
Physicians |
Allied health | |
|
This doctor knows his/her own limitations and acts within them |
Professional |
Physicians |
Allied health | |
|
This doctor accepts feedback from others |
Professional |
Physicians |
Allied health | |
|
This doctor acts according to codes of ethical conduct |
Professional |
Physicians |
Allied health | |
|
I would recommend this doctor to my family and friends |
Professional |
Physicians | ||
Table 2
Survey item scores, categorised in levels of Kirkpatrick
|
Question (‘my/the resident’s’ reflect the two versions of the questionnaire) |
Residents |
Programme directors |
|---|---|---|
|
(N = 108) |
(N = 22) | |
|
Mean (SD) |
Mean (SD) | |
|
Kirkpatrick level 1 – REACTION | ||
|
The multisourcefeedback.nl tool is user friendly |
3.9 (0.9) |
4.1 (0.8) |
|
The final report is comprehensible and useful |
3.7 (1.0) |
4.0 (0.7) |
|
The questions are generally applicable in my/the resident’s situation |
3.4 (0.9) |
3.7 (0.6) |
|
Time between MSF procedure and progress interview (at least a month) is (1 = too short; 5 = too long) |
3.0 (0.6) |
3.1 (0.6) |
|
The number of respondents to approach (6 medical colleagues, 6 allied health colleagues, 10 patients) was doable |
3.0 (1.3) |
3.7 (1.0) |
|
It is good that the report is anonymous |
3.9 (1.1) |
3.9 (1.3) |
|
The MSF reports yield an adequate impression of myself/of the resident |
3.5 (0.9) |
4.1 (0.5) |
|
The MSF reports have added value compared with other sources of feedback |
3.5 (1.0) |
4.3 (0.5) |
|
A progress interview using MSF leads to more concrete learning goals than when without MSF |
3.0 (1.1) |
3.7 (0.8) |
|
Kirkpatrick level 2 – LEARNING | ||
|
My/the resident’s medical knowledge improved due to MSF |
2.4 (0.9) |
2.7 (0.9) |
|
The latest MSF report yielded adequate points for my personal development |
3.5 (1.0) |
– |
|
The latest MSF report taught me something about myself I did not know before |
2.6 (1.0) |
– |
|
Working with colleagues improved due to the latest MSF |
3.1 (0.9) |
3.4 (0.8) |
|
Kirkpatrick level 3 – BEHAVIOUR | ||
|
I think my/the resident’s performance improved due to the latest MSF |
2.8 (1.0) |
3.4 (0.8) |
|
I think my/the resident’s professional attitude improved due to the latest MSF |
3.2 (1.0) |
3.7 (0.8) |
|
The relationship with patients improved due to the latest MSF |
2.8 (0.9) |
3.2 (0.9) |
|
Did latest the MSF procedure make you/the residents adapt your/their behaviour? Yes |
17 % |
30 % |
|
Slightly |
37 % |
55 % |
|
No |
46 % |
15 % |
|
Kirkpatrick level 4 – IMPACT | ||
|
My/the resident’s medical colleagues benefited from positive changes in my/his or her performance due to the latest MSF |
3.1 (0.9) |
3.3 (1.0) |
|
My/the resident’s allied health colleagues benefited from positive changes in my/his or her performance due to the latest MSF |
3.2 (0.9) |
3.4 (1.0) |
|
Patients benefited from positive changes in my/his or her performance due to the latest MSF |
3.0 (1.0) |
3.4 (1.0) |
|
The clinical care improved due to the latest MSF |
3.0 (1.1) |
3.2 (0.9) |