Guidelines: The do’s, don’ts and don’t knows of direct observation of clinical skills in medical education
References
- Frank JR Snell LS Cate OT Competency-based medical education: theory to practice Med Teach 2010 32 638 645 10.3109/0142159X.2010.501190
- Frank JR Mungroo R Ahmad Y Wang M De Rossi S Horsley T Toward a definition of competency-based education in medicine: a systematic review of published definitions Med Teach 2010 32 631 637 10.3109/0142159X.2010.500898
- Iobst WF Sherbino J Cate OT Competency-based medical education in postgraduate medical education Med Teach 2010 32 651 656 10.3109/0142159X.2010.500709
- Holmboe ES Sherbino J Long DM Swing SR Frank JR The role of assessment in competency-based medical education Med Teach 2010 32 676 682 10.3109/0142159X.2010.500704
- Carraccio C Wolfsthal SD Englander R Ferentz K Martin C Shifting paradigms: from Flexner to competencies Acad Med 2002 77 361 367 10.1097/00001888-200205000-00003
- Liaison Committee of Medical Education Functions and structure of a medical school 2017
- Accreditation Council for Graduate Medical Education Common program requirements 2017
- Royal College of Physicians. 2017. https://www.rcplondon.ac.uk/. Accessed 7 May 2017.
- Holmboe ES Hawkins RE Huot SJ Effect of training in direct observation of medical residents’ clinical competence: a randomized trial Ann Intern Med 2004 140 874 881 10.7326/0003-4819-140-11-200406010-00008
- Holmboe ES Realizing the promise of competency-based medical education Acad Med 2015 90 411 412 10.1097/ACM.0000000000000515
- LaDonna KA Hatala R Lingard L Voyer S Watling C Staging a performance: learners’ perceptions about direct observation during residency Med Educ 2017 51 498 510 10.1111/medu.13232
- McGaghie WC Varieties of integrative scholarship: why rules of evidence, criteria, and standards matter Acad Med 2015 90 294 302 10.1097/ACM.0000000000000585
- Lau AMS ‘Formative good, summative bad?’—a review of the dichotomy in assessment literature J Furth High Educ 2016 40 509 525 10.1080/0309877X.2014.984600
- Lefroy J Watling C Teunissen PW Brand P Guidelines: the do’s, don’ts and don’t knows of feedback for clinical education Perspect Med Educ 2015 4 284 99
- Miller GE The assessment of clinical skills/competence/performance Acad Med 1990 65 S63 S67 10.1097/00001888-199009000-00045
- van der Vleuten CP Schuwirth LW Scheele F Driessen EW Hodges B Assessment of professional competence: building blocks for theory development Best Pract Res Clin Obstet Gynaecol 2010 24 703 719 10.1016/j.bpobgyn.2010.04.001
- Eva KW Bordage G Campbell C Toward a program of assessment for health professionals: from training into practice Adv Health Sci Educ Theory Pract 2016 2 897 913 10.1007/s10459-015-9653-6
- Teunissen PW Scheele F Scherpbier AJJA How residents learn: qualitative evidence for the pivotal role of clinical activities Med Educ 2007 41 763 770 10.1111/j.1365-2923.2007.02778.x
- Teunissen PW Boor K Scherpbier AJJA Attending doctors’ perspectives on how residents learn Med Educ 2007 41 1050 1058 10.1111/j.1365-2923.2007.02858.x
- Watling C Driessen K van der Vleuten CP Lingard L Learning from clinical work: the roles of learning cues and credibility judgements Med Educ 2012 46 192 200 10.1111/j.1365-2923.2011.04126.x
- Kneebone R Nestel D Wetzel C The human face of simulation: patient-focused simulation training Acad Med 2006 81 919 924 10.1097/01.ACM.0000238323.73623.c2
- Lane C Rollnick S The use of simulated patients and role-play in communication skills training: a review of the literature to August 2005 Patient Educ Couns 2007 67 13 20 10.1016/j.pec.2007.02.011
- Bokken L Rethans JJ van Heurn L Duvivier R Scherpbier A van der Vleuten C Students’ views on the use of real patients and simulated patients in undergraduate medical education Acad Med 2009 84 958 963 10.1097/ACM.0b013e3181a814a3
- Norcini JJ Blank LL Arnold GK Kimball HR The mini-CEX (clinical evaluation exercise); a preliminary investigation Ann Intern Med 1995 123 796 799 10.7326/0003-4819-123-10-199511150-00008
- Bates J Konkin J Suddards C Dobson S Pratt D Student perceptions of assessment and feedback in longitudinal integrated clerkships Med Educ 2013 47 362 374 10.1111/medu.12087
- Harrison CJ Könings KD Dannefer EF Schuwirth LW Wass V van der Vleuten CP Factors influencing students’ receptivity to formative feedback emerging from different assessment cultures Perspect Med Educ 2016 5 276 284 10.1007/s40037-016-0297-x
- Paradis E Sutkin G Beyond a good story: from Hawthorne effect to reactivity in health professions education research Med Educ 2017 51 31 39 10.1111/medu.13122
- Pintrich PR Conley AM Kempler TM Current issues in achievement goal theory research Int J Educ Res 2003 39 319 337 10.1016/j.ijer.2004.06.002
- Mutabdzic D Mylopoulos M Murnaghan ML Coaching surgeons: is culture limiting our ability to improve? Ann Surg 2015 262 213 216 10.1097/SLA.0000000000001247
- Kusurkar RA Croiset G Ten Cate TJ Twelve tips to stimulate intrinsic motivation in students through autonomy-supportive classroom teaching derived from self-determination theory Med Teach 2011 33 978 982 10.3109/0142159X.2011.599896
- Voyer S Cuncic C Butler DL MacNeil K Watling C Hatala R Investigating conditions for meaningful feedback in the context of an evidence-based feedback programme Med Educ 2016 50 943 954 10.1111/medu.13067
- Sandars J Cleary TJ Self-regulation theory: applications to medical education: AMEE Guide No. 58 Med Teach 2011 33 875 886 10.3109/0142159X.2011.595434
- Ali JM Getting lost in translation? Workplace based assessments in surgical training Surgeon 2013 11 286 289 10.1016/j.surge.2013.03.001
- Butler DL Winne PH Feedback and self-regulated learning: a theoretical synthesis Rev Educ Res 1995 65 245 281 10.3102/00346543065003245
- Sagasser MH Kramer AW van der Vleuten CP How do postgraduate GP trainees regulate their learning and what helps and hinders them? A qualitative study BMC Med Educ 2012 12 67 10.1186/1472-6920-12-67
-
Pulito
AR
Donnelly
MB
Plymale
M
Mentzer
RM
Jr. What do faculty observe of medical students’ clinical performance? Teach Learn Med 2006 18 99 104 10.1207/s15328015tlm1802_2 - Hasnain M Connell KJ Downing SM Olthoff A Yudkowsky R Toward meaningful evaluation of clinical competence: the role of direct observation in clerkship ratings Acad Med 2004 79 S21 S24 10.1097/00001888-200410001-00007
- St-Onge C Chamberland M Levesque A Varpio L The role of the assessor: exploring the clinical supervisor’s skill set Clin Teach 2014 11 209 213 10.1111/tct.12126
- Watling CJ Kenyon CF Zibrowski EM Rules of engagement: residents’ perceptions of the in-training evaluation process Acad Med 2008 83 S97 S100 10.1097/ACM.0b013e318183e78c
- Kennedy TJ Regehr G Baker GR Lingard LA ‘It’s a cultural expectation…’ The pressure on medical trainees to work independently in clinical practice Med Educ 2009 43 645 653 10.1111/j.1365-2923.2009.03382.x
- Kennedy TJ Regehr G Baker GR Lingard L Preserving professional credibility: grounded theory study of medical trainees’ requests for clinical support BMJ 2009 338 b128 10.1136/bmj.b128
- Watling C LaDonna KA Lingard L Voyer S Hatala R ‘Sometimes the work just needs to be done’: socio-cultural influences on direct observation in medical training Med Educ 2016 50 1054 1064 10.1111/medu.13062
- Madan R Conn D Dubo E Voore P Wiesenfeld L The enablers and barriers to the use of direct observation of trainee clinical skills by supervising faculty in a psychiatry residency program Can J Psychiatry 2012 57 269 272 10.1177/070674371205700411
- Goldszmidt M Aziz N Lingard L Taking a detour: positive and negative effects of supervisors’ interruptions during admission case review discussions Acad Med 2012 87 1382 1388 10.1097/ACM.0b013e3182675b08
- Govaerts MJ Schuwirth LW van der Vleuten CP Muijtjens AM Workplace-based assessment: effects of rater expertise Adv Health Sci Educ Theory Pract 2011 16 151 165 10.1007/s10459-010-9250-7
- Govaerts MJB Van de Wiel MW Schuwirth LW Van der Vleuten CP Muijtjens AM Workplace-based assessment: raters’ performance theories and constructs Adv Health Sci Educ Theory Pract 2013 18 375 396 10.1007/s10459-012-9376-x
- Gingerich A Regehr G Eva KW Rater-based assessments as social judgments: rethinking the etiology of rater errors Acad Med 2011 86 S1 S7 10.1097/ACM.0b013e31822a6cf8
- Wood TJ Exploring the role of first impressions in rater-based assessments Adv Health Sci Educ Theory Pract 2014 19 409 427 10.1007/s10459-013-9453-9
- Yeates P O’Neill P Mann K Eva KW ‘You’re certainly relatively competent’: assessor bias due to recent experiences Med Educ 2013 47 910 922 10.1111/medu.12254
- Cohen SN Farrant PB Taibjee SM Assessing the assessments: U.K. dermatology trainees’ views of the workplace assessment tools Br J Dermatol 2009 161 34 39 10.1111/j.1365-2133.2009.09097.x
- Shute VJ Focus on formative feedback Rev Educ Res 2008 78 153 189 10.3102/0034654307313795
- Kogan JR Conforti LN Iobst WF Holmboe ES Reconceptualizing variable rater assessments as both an educational and clinical care problem Acad Med 2014 89 721 727 10.1097/ACM.0000000000000221
- Ten Cate O Hart D Ankel F Entrustment decision making in clinical training Acad Med 2016 91 191 198 10.1097/ACM.0000000000001044
- Chen HC Fogh S Kobashi B Teherani A ten Cate O O’Sullivan P An interview study of how clinical teachers develop skills to attend to different level learners Med Teach 2016 36 578 584 10.3109/0142159X.2015.1073238
- Alves de Lima A Henquin R Thierer HR A qualitative study of the impact on learning of the mini clinical evaluation exercise in postgraduate training Med Teach 2005 27 46 52 10.1080/01421590400013529
- Weston PS Smith CA The use of mini-CEX in UK foundation training six years following its introduction: lessons still to be learned and the benefit of formal teaching regarding its utility Med Teach 2014 36 155 163 10.3109/0142159X.2013.836267
- Malhotra S Hatala R Courneya CA Internal medicine residents’ perceptions of the mini-clinical evaluation exercise Med Teach 2008 30 414 419 10.1080/01421590801946962
- Hamburger EK Cuzzi S Coddington DA Observation of resident clinical skills: outcomes of a program of direct observation in the continuity clinic setting Acad Pediatr 2011 11 394 402 10.1016/j.acap.2011.02.008
- Bindal T Wall D Goodyear HM Trainee doctors’ views on workplace-based assessments: are they just a tick box exercise? Med Teach 2011 33 919 927 10.3109/0142159X.2011.558140
- Bok HG Teunissen PW Favier RP Programmatic assessment of competency-based workplace learning: when theory meets practice BMC Med Educ 2013 13 123 10.1186/1472-6920-13-123
- Holmboe ES Williams F Yepes M Huot S Feedback and the mini-cex J Gen Intern Med 2004 19 558 561 10.1111/j.1525-1497.2004.30134.x
- Guadagnoli M Morin MP Dubrowski A The application of the challenge point framework in medical education Med Educ 2012 46 447 453 10.1111/j.1365-2923.2011.04210.x
- Bjork RA Metcalfe J Shimamura A Memory and metamemory considerations in the training of human beings Metacognition: knowing about knowing 1994 Cambridge MIT Press 185 205
- Telio S Regehr G Ajjawi R Feedback and the educational alliance: examining credibility judgements and their consequences Med Educ 2016 50 933 942 10.1111/medu.13063
- Al-Kadri HM Al-Kadi MT van der Vleuten CP Workplace-based assessment and students’ approaches to learning: a qualitative inquiry Med Teach 2013 35 S31 S38 10.3109/0142159X.2013.765547
- Kogan JR Holmboe ES Hauer KE Tools for direct observation and assessment of clinical skills of medical trainees: a systematic review JAMA 2009 302 1316 1326 10.1001/jama.2009.1365
- Hanson JL Rosenberg AA Lane JL Narrative descriptions should replace grades and numerical ratings for clinical performance in medical education in the United States Front Psychol 2013 4 688 10.3389/fpsyg.2013.00668
-
William
D
Lester
FK
Jr. Keeping learning on track: classroom assessment and the regulation of learning Second handbook of mathematics teaching and learning 2007 Greenwich Information Age Publishing 1053 1098 - Butler R Task-involving and ego-involving properties of evaluation: effects of different feedback conditions on motivational perceptions, interest, and performance J Educ Psych 1987 79 474 482 10.1037/0022-0663.79.4.474
- McColskey W Leary MR Differential effects of norm-referenced and self-referenced feedback on performance expectancies, attribution, and motivation Contemp Educ Psychol 1985 10 275 284 10.1016/0361-476X(85)90024-4
- Daelmans HE Mak-van der Vossen MC Croiset G Kusurkar RA What difficulties do faculty members face when conducting workplace-based assessments in undergraduate clerkships? Int J Med Educ 2016 7 19 24 10.5116/ijme.5689.3c7f
- Rogers HD Carline JD Paauw DS Examination room presentations in general internal medicine clinic: patients’ and students’ perceptions Acad Med 2003 78 945 949 10.1097/00001888-200309000-00023
- Schultz KW Kirby J Delva D Medical students’ and residents’ preferred site characteristics and preceptor behaviours for learning in the ambulatory setting: a cross-sectional survey Bmc Med Educ 2004 4 12 10.1186/1472-6920-4-12
- Kernan WN Lee MY Stone SL Freudigman KA O’Connor PG Effective teaching for preceptors of ambulatory care: a survey of medical students Am J Med 2000 108 6 499 502 10.1016/S0002-9343(00)00301-6
- Lehmann LS Brancati FL Chen MC Roter D Dobs AS The effect of bedside case presentations on patients’ perceptions of their medical care N Engl J Med 1997 336 1150 1155 10.1056/NEJM199704173361606
- Tavares W Eva KW Exploring the impact of mental workload on rater-based assessments Adv Health Sci Educ Theory Pract 2013 18 291 303 10.1007/s10459-012-9370-3
- Tavares W Ginsburg S Eva KW Selecting and simplifying: rater performance and behavior when considering multiple competencies Teach Learn Med 2016 28 41 51 10.1080/10401334.2015.1107489
- Kogan JR Hess BJ Conforti LN Holmboe ES What drives faculty ratings of residents’ clinical skills? The impact of faculty’s own clinical skills Acad Med 2010 85 S25 S28 10.1097/ACM.0b013e3181ed1aa3
- Byrne A Tweed N Halligan C A pilot study of the mental workload of objective structured clinical examination examiners Med Educ 2014 48 262 267 10.1111/medu.12387
- Bok HG Jaarsma DA Spruijt A Van Beukelen P Van der Vleuten CP Teunissen PW Feedback-giving behavior in performance evaluations during clinical clerkships Med Teach 2016 38 88 95 10.3109/0142159X.2015.1017448
- Wenrich MD Jackson MB Ajam KS Wolfhagen IH Ramsey PG Scherpbier AJ Teachers as learners: the effect of bedside teaching on the clinical skills of clinician-teachers Acad Med 2011 86 846 852 10.1097/ACM.0b013e31821db1bc
- Kogan JR Conforti L Bernabeo E Iobst W Holmboe ES Opening the black box of clinical skills assessment via observation: a conceptual model Med Educ 2011 45 1048 1060 10.1111/j.1365-2923.2011.04025.x
- Berendonk C Stalmeijer RE Schuwirth LW Expertise in performance assessment: assessors’ perspectives Adv Health Sci Educ Theory Pract 2013 18 559 571 10.1007/s10459-012-9392-x
- Leape LL Fromson JA Problem doctors: is there a system-level solution? Ann Intern Med 2006 144 107 115 10.7326/0003-4819-144-2-200601170-00008
- Asch DA Nicholson S Srinivas SK Herrin J Epstein AJ How do you deliver a good obstetrician? Outcome-based evaluation of medical education Acad Med 2014 89 24 26 10.1097/ACM.0000000000000067
- Asch DA Nicholson S Srinivas S Herrin J Epstein AJ Evaluating obstetrical residency programs using patient outcomes JAMA 2009 302 1277 1283 10.1001/jama.2009.1356
- Epstein AJ Srinivas SK Nicholson S Herrin J Asch DA Association between physicians’ experience after training and maternal obstetrical outcomes: cohort study BMJ 2013 346 f1596 10.1136/bmj.f1596
- Sirovich BE Lipner RS Johnston M Holmboe ES The association between residency training and internists’ ability to practice conservatively JAMA Intern Med 2014 174 1640 1648 10.1001/jamainternmed.2014.3337
- Bansal N Simmons KD Epstein AJ Morris JB Kelz RR Using patient outcomes to evaluate general surgery residency program performance JAMA Surg 2016 151 111 119 10.1001/jamasurg.2015.3637
- Chen C Petterson S Phillips R Bazemore A Mullan F Spending patterns in region of residency training and subsequent expenditures for care provided by practicing physicians for Medicare beneficiaries JAMA 2014 312 2385 2393 10.1001/jama.2014.15973
- Kogan JR Conforti LN Bernabeo E Iobst W Holmboe E How faculty members experience workplace-based assessment rater training: a qualitative study Med Educ 2015 49 692 708 10.1111/medu.12733
- Pelgrim EA Kramer AW Mokkink HG van den Elsen L Grol RP van der Vleuten CP In-training assessment using direct observation of single-patient encounters: a literature review Adv Health Sci Educ Theory Pract 2011 16 131 142 10.1007/s10459-010-9235-6
- Hauer KE Holmboe ES Kogan JR Twelve tips for implementing tools for direct observation of medical trainees’ clinical skills during patient encounters Med Teach 2011 33 27 33 10.3109/0142159X.2010.507710
-
Artino
AR
Jr La Rochelle JS Dezee KJ Gehlbach H Developing questionnaires for educational research: AMEE Guide No. 87 Med Teach 2014 36 463 474 10.3109/0142159X.2014.889814 - Donato AA Pangaro L Smith C Evaluation of a novel assessment form for observing medical residents: a randomised, controlled trial Med Educ 2008 42 1234 1242 10.1111/j.1365-2923.2008.03230.x
- Crossley J Johnson G Booth J Wade W Good questions, good answers: construct alignment improves the performance of workplace-based assessment scales Med Educ 2011 45 560 569 10.1111/j.1365-2923.2010.03913.x
- Ilgen JS Ma IW Hatala R Cook DA A systematic review of validity evidence for checklists versus global rating scales in simulation-based assessment Med Educ 2015 49 161 173 10.1111/medu.12621
- Yeates P O’Neill P Mann K Eva K Seeing the same thing differently: mechanisms that contribute to assessor differences in directly-observed performance assessments Adv Health Sci Educ Theory Pract 2013 18 325 341 10.1007/s10459-012-9372-1
- Gingerich A Ramlo SE van der Vleuten CP Eva KW Regehr G Inter-rater variability as mutual disagreement: identifying raters’ divergent points of view Adv Health Sci Educ Theory Pract 2016
- Govaerts MJ van der Vleuten CP Schuwirth LW Muijtjens AM Broadening perspectives on clinical performance assessment: rethinking the nature of in-training assessment Adv Health Sci Educ Theory Pract 2007 12 239 260 10.1007/s10459-006-9043-1
- Roch SG Woehr DJ Mishra V Kieszczynska U Rater training revisited: an updated meta-analytic review of frame-of-reference training J Occup Organ Psychol 2012 85 370 395 10.1111/j.2044-8325.2011.02045.x
- Woehr DJ Rater training for performance appraisal: a quantitative review J Occup Organ Psychol 1994 67 189 205 10.1111/j.2044-8325.1994.tb00562.x
- Cook DA Dupras DM Beckman TJ Thomas KG Pankratz VS Effect of rater training on reliability and accuracy of mini-CEX scores: a randomized, controlled trial J Gen Intern Med 2009 24 74 79 10.1007/s11606-008-0842-3
- Yelon SL Ford JK Anderson WA Twelve tips for increasing transfer of training from faculty development programs Med Teach 2014 36 945 950 10.3109/0142159X.2014.929098
- Steinert Y Mann K Centeno A A systematic review of faculty development initiatives designed to improve teaching effectiveness in medical education: BEME Guide No. 8 Med Teach 2006 28 497 526 10.1080/01421590600902976
- Steinert Y Mann K Anderson B A systematic review of faculty development initiatives designed to enhance teaching effectiveness: a 10 year update: BEME Guide No. 40 Med Teach 2016 38 769 786 10.1080/0142159X.2016.1181851
- Kogan JR Conforti LN Bernabeo E Durning SJ Hauer KE Holmboe ES Faculty staff perceptions of feedback to residents after direct observation of clinical skills Med Educ 2012 46 201 215 10.1111/j.1365-2923.2011.04137.x
- Pangaro L ten Cate O Frameworks for learner assessment in medicine: AMEE Guide No. 78 Med Teach 2013 35 e1197 e1210 10.3109/0142159X.2013.788789
- Ten Cate O Entrustability of professional activities and competency-based training Med Educ 2005 39 1176 1177 10.1111/j.1365-2929.2005.02341.x
- Heeneman S Oudkerk Pool A Schuwirth LW van der Vleuten CP Driessen EW The impact of programmatic assessment on student learning—the theory versus practice Med Educ 2015 49 487 498 10.1111/medu.12645
- Berkhout JJ Helmich E Teunissen PW van den Berg JW van der Vleuten CP Jaarsma AD Exploring the factors influencing clinical students’ self-regulated learning Med Educ 2015 49 589 600 10.1111/medu.12671
- Dweck CS Motivational processes affecting learning Am Psychol 1986 41 1040 1048 10.1037/0003-066X.41.10.1040
- Wong BM Holmboe ES Transforming the academic faculty perspective in graduate medical education to better align educational and clinical outcomes Acad Med 2016 91 473 479 10.1097/ACM.0000000000001035
- Wong RY Chen L Dhadwal G Twelve tips for teaching in a provincially distributed education program Med Teach 2012 34 116 122 10.3109/0142159X.2011.588731
- Pitts S Borus J Goncalves A Gooding H Direct versus remote clinical observation: assessing learners’ milestones while addressing adolescent patients’ needs J Grad Med Educ 2015 7 253 255 10.4300/JGME-D-14-00484.1
- Evans SJ Effective direct student observation strategies in neurology Med Educ 2010 44 500 501 10.1111/j.1365-2923.2010.03642.x
- Block L Habicht R Wu AW In the wake of the 2003 and 2011 duty hours regulations, how do internal medicine interns spend their time? J Gen Intern Med 2013 28 1042 1047 10.1007/s11606-013-2376-6
- Fletcher KE Visotcky AM Slagle JM Tarima S Weinger MB Schapira MM The composition of intern work while on call J Gen Intern Med 2012 27 1432 1437 10.1007/s11606-012-2120-7
- Bernabeo EC Holtman MC Ginsburg S Rosenbaum JR Holmboe ES Lost in transition: the experience and impact of frequent changes in the inpatient learning environment Acad Med 2011 86 591 598 10.1097/ACM.0b013e318212c2c9
- Mazotti L O’Brien B Tong L Hauer KE Perceptions of evaluation in longitudinal versus traditional clerkships Med Educ 2011 45 464 470 10.1111/j.1365-2923.2010.03904.x
- Warm EJ Schauer DP Diers T The ambulatory long-block: an accreditation council for graduate medical education (ACGME) educational innovations project (EIP) J Gen Intern Med 2008 23 921 926 10.1007/s11606-008-0588-y
- Francis MD Wieland ML Drake S Clinic design and continuity in internal medicine resident clinics: findings of the educational innovations project ambulatory collaborative J Grad Med Educ 2015 7 36 41 10.4300/JGME-D-14-00358.1
- Gordon MJ Cutting the Gordian knot: a two-part approach to the evaluation and professional development of residents Acad Med 1997 72 876 880
- Ten Cate O Chen HC Hoff RG Peters H Bok H van der Schaaf M Curriculum development for the workplace using Entrustable Professional Activities (EPAs): AMEE Guide No. 99 Med Teach 2015 37 983 1002 10.3109/0142159X.2015.1060308
- Touchie C ten Cate O The promise, perils, problems and progress of competency-based medical education Med Educ 2016 50 93 100 10.1111/medu.12839
- Pryor J Crossouard B A socio-cultural theorisation of formative assessment Oxford Rev Educ 2008 34 1 20 10.1080/03054980701476386
- Looney A Cumming J van Der Kleij F Harris K Reconceptualising the role of teachers as assessors: teacher assessment identity Assess Educ Princ Policy Pract 2017
- McCambridge J Witton J Elbourne DR Systematic review of the Hawthorne effect: new concepts are needed to study research participation effects J Clin Epidemiol 2014 67 267 277 10.1016/j.jclinepi.2013.08.015
- Norcini JJ Blank LL Duffy FD Fortna GS The mini-CEX: a method for assessing clinical skills Ann Intern Med 2003 138 476 481 10.7326/0003-4819-138-6-200303180-00012
- Clauser BE Margolis MJ Swanson DB Holmboe E Hawkins R Issues of validity and reliability for assessments in medical education Practical guide to the evaluation of clinical competence 2008 Philadelphia Elsevier Mosby
- Cronbach LJ Meehl PE Construct validity in psychological tests Psychol Bull 1955 52 281 302 10.1037/h0040957
- Streiner DL Norman GR Health measurement scales: a practical guide to their development and use 2008 Oxford Oxford University Press 10.1093/acprof:oso/9780199231881.001.0001
- van der Vleuten CP Schuwirth LW Assessing professional competence: from methods to programmes Med Educ 2005 39 309 317 10.1111/j.1365-2929.2005.02094.x
- Schuwirth LW van der Vleuten CP Programmatic assessment: from assessment of learning to assessment for learning Med Teach 2011 33 478 485 10.3109/0142159X.2011.565828
- Schuwirth LW van der Vleuten CP The use of clinical simulations in assessment Med Educ 2003 37 Suppl 1 65 71 10.1046/j.1365-2923.37.s1.8.x
- Kane MT Validating the interpretations and uses of test scores J Educ Meas 2013 50 1 73 10.1111/jedm.12000
- Cook DA Brydges R Ginsburg S Hatala R A contemporary approach to validity arguments: a practical guide to Kane’s framework Med Educ 2015 49 560 575 10.1111/medu.12678
- Chan T Sherbino J McMAP Collaborators The mcmaster modular assessment program (mcMAP) Acad Med 2015 90 900 905 10.1097/ACM.0000000000000707
- Laine C Davidoff F Lewis CE Nelson E Kessler RC Delbanco TL Important elements of outpatient care: a comparison of patients’ and physicians’ opinions Ann Intern Med 2016 125 640 645 10.7326/0003-4819-125-8-199610150-00003
- van der Vleuten CP Schuwirth LW Driessen E A model for programmatic assessment fit for purpose Med Teach 2012 34 205 214 10.3109/0142159X.2012.652239
-
Artino
AR
Jr Dong T DeZee KJ Achievement goal structures and self-regulated learning: relationships and changes in medical school Acad Med 2012 87 1375 1381 10.1097/ACM.0b013e3182676b55 - Eva KW Regehr G Effective feedback for maintenance of competence: from data delivery to trusting dialogues CMAJ 2013 185 263 264 10.1503/cmaj.121772
- Watling C Driessen E van der Vleuten CP Vanstone M Lingard L Understanding responses to feedback: the potential and limitations of regulatory focus theory Med Educ 2012 46 593 603 10.1111/j.1365-2923.2012.04209.x
- Sargeant J Bruce D Campbell CM Practicing physicians’ needs for assessment and feedback as part of professional development J Contin Educ Health Prof 2013 33 Suppl 1 S54 S62 10.1002/chp.21202
- Holmboe ES The journey to competency-based medical education-implementing milestones Marshall J Med 2017
- Sargeant J Lockyer J Mann K Facilitated reflective performance feedback: developing an evidence-and theory-based model that builds relationship, explores reactions and content, and coaches for performance change (R2C2) Acad Med 2015 90 1698 1706 10.1097/ACM.0000000000000809
- Andolsek K Padmore J Hauer KE Holmboe E Clinical competency committees: a guidebook for programs. Accreditation council for graduate medical education 2017
- Hauer KE Cate OT Boscardin CK Ensuring resident competence: a narrative review of the literature on group decision making to inform the work of clinical competency committees J Grad Med Educ 2016 8 156 164 10.4300/JGME-D-15-00144.1
- Hauer KE Chesluk B Iobst W Reviewing residents’ competence: a qualitative study of the role of clinical competency committees in performance assessment Acad Med 2015 90 1084 1092 10.1097/ACM.0000000000000736
- Misch DA Evaluating physicians’ professionalism and humanism: the case for humanism ‘connoisseurs’ Acad Med 2002 77 489 495 10.1097/00001888-200206000-00004
- Reckman J Gofton W Dudek N Gofton T Hamstra SJ Entrustability scales: outlining their usefulness for competency based clinical assessment Acad Med 2016 91 186 190 10.1097/ACM.0000000000001045
- Weller JM Misur M Nicolson S Can I leave the theatre? A key to more reliable workplace-based assessment Br J Anaesth 2014 112 1083 1091 10.1093/bja/aeu052
- Regehr G Ginsburg S Herold J Hatala R Eva K Oulanova O Using ‘standardized narratives’ to explore new ways to represent faculty opinions of resident performance Acad Med 2012 87 419 427 10.1097/ACM.0b013e31824858a9
- DaRosa DA Zwischenberger JB Meyerson SL A theory-based model for teaching and assessing residents in the operating room J Surg Educ 2013 70 24 30 10.1016/j.jsurg.2012.07.007
- Teman NR Gauger PG Mullan PB Tarpley JL Minter RM Entrustment of general surgery residents in the operating room: factors contributing to provision of resident autonomy J Am Coll Surg 2014 219 778 787 10.1016/j.jamcollsurg.2014.04.019
- Sanhu G Teman NR Minter RM Training autonomous surgeons: more time or faculty development Ann Surg 2015 261 843 845 10.1097/SLA.0000000000001058
- George BC Teitelbaum EN DaRosa DA Duration of faculty training needed to ensure reliable OR performance ratings J Surg Educ 2013 70 703 708 10.1016/j.jsurg.2013.06.015
- George BC Teitelbaum EN Meyerson SL Reliability, validity and feasibility of the Zwisch scale for the assessment of intraoperative performance J Surg Educ 2014 71 e90 e96 10.1016/j.jsurg.2014.06.018
- Ferenchick GS Solomon D Foreback J Mobile technology for the facilitation of direct observation and assessment of student performance Teach Learn Med 2013 25 292 299 10.1080/10401334.2013.827972
- Torre DM Simpson DE Elnicki DM Sebastian JL Holmboe ES Feasibility, reliability and user satisfaction with a PDA-based mini-CEX to evaluate the clinical skills of third-year medical students Teach Learn Med 2007 19 271 277 10.1080/10401330701366622
- Coulby C Hennessey S Davies N Fuller R The use of mobile technology for work-based assessment: the student experience Br J Educ Technol 2011 42 251 265 10.1111/j.1467-8535.2009.01022.x
- Page CP Reid A Coe CL Learnings from the pilot implementation of mobile medical milestones application J Grad Med Educ 2016 8 569 575 10.4300/JGME-D-15-00550.1
- Lockyer JM Holmboe ES Durning S Hawkins RE Multisource feedback Practical guide to the evaluation of clinical competence 2017 2 Philadelphia Elsevier 204 214
- Warm EJ Schauer D Revis B Boex JR Multisource feedback in the ambulatory setting J Grad Med Educ 2010 2 269 277 10.4300/JGME-D-09-00102.1
- Lockyer J Armson H Chesluk B Feedback data sources that inform physician self-assessment Med Teach 2011 33 e113 e120 10.3109/0142159X.2011.542519
- Sargeant J Reflection upon multisource feedback as ‘assessment for learning.’ Perspect Med Educ 2015 4 55 56 10.1007/s40037-015-0175-y
- Concato J Feinstein AR Asking patients what they like: overlooked attributes of patient satisfaction with primary care Am J Med 1997 102 399 406 10.1016/S0002-9343(97)00092-2
- Ericsson KA Acquisition and maintenance of medical expertise: a perspective from the expert-performance approach with deliberate practice Acad Med 2015 90 1471 1486 10.1097/ACM.0000000000000939
- Levinson W Lesser CS Epstein RM Developing physician communication skills for patient-centered care Health Aff 2010 29 1310 1318 10.1377/hlthaff.2009.0450
- Braddock CH Edwards KA Hasenberg NM Laidley TL Levinson W Informed decision making in outpatient practice: time to get back to basics JAMA 1999 282 2312 2320
- Mattar SG Alseidi AA Jones DB General surgery residency inadequately prepares trainees for fellowship: results of a survey of fellowship program directors Ann Surg 2013 258 440 449 10.1097/SLA.0b013e3182a191ca
- Crosson FJ Leu J Roemer BM Ross MN Gaps in residency training should be addressed to better prepare doctors for a twenty-first-century delivery system Health Aff 2011 30 2142 2148 10.1377/hlthaff.2011.0184
DOI: https://doi.org/10.1007/S40037-017-0376-7 | Journal eISSN: 2212-277X
Language: English
Page range: 286 - 305
Published on: Sep 27, 2017
Published by: Bohn Stafleu van Loghum
In partnership with: Paradigm Publishing Services
Publication frequency: 1 issue per year
© 2017 Jennifer R. Kogan, Rose Hatala, Karen E. Hauer, Eric Holmboe, published by Bohn Stafleu van Loghum
This work is licensed under the Creative Commons Attribution 4.0 License.