References
- Halman S Dudek N Wood T Direct Observation of Clinical Skills Feedback Scale: Development and Validity Evidence Teach Learn Med 2016 28 385 394
- Miller A, Archer J. Impact of workplace based assessment on doctors’ education and performance: a systematic review. BMJ. 2010;341:c5064. https://doi.org/10.1136/bmj.c5064.
- Schuwirth LW Van der Vleuten CP Programmatic assessment: From assessment of learning to assessment for learning Med Teach 2011 33 478 485
- Al Ansari A Ali SK Donnon T The construct and criterion validity of the mini-CEX: a meta-analysis of the published research Acad Med 2013 88 413 420
- Leep Hunderfund AN Rubin DI Laughlin RS Validity and feasibility of the EMG direct observation tool (EMG-DOT) Neurology 2016 86 1627 1634
- Olupeliyawa AM O’Sullivan AJ Hughes C Balasooriya CD The Teamwork Mini-Clinical Evaluation Exercise (T-MEX): a workplace-based assessment focusing on collaborative competencies in health care Acad Med 2014 89 359 365
- Watanabe Y Bilgic E Lebedeva E A systematic review of performance assessment tools for laparoscopic cholecystectomy Surg Endosc 2016 30 832 844
- Watson MJ Wong DM Kluger R Psychometric evaluation of a direct observation of procedural skills assessment tool for ultrasound-guided regional anaesthesia Anaesthesia 2014 69 604 612
- Feraco AM Starmer AJ Sectish TC Spector ND West DC Landrigan CP Reliability of Verbal Handoff Assessment and Handoff Quality Before and After Implementation of a Resident Handoff Bundle Acad Pediatr 2016 16 524 531
- Norcini JJ Blank LL Arnold GK Kimball HR The mini-CEX (clinical evaluation exercise): a preliminary investigation Ann Intern Med 1995 123 795 799
- Young JQ Rasul R O’Sullivan PS Evidence for the Validity of the Psychopharmacotherapy-Structured Clinical Observation Tool: Results of a Factor and Time Series Analysis Acad Psychiatry 2018 42 759 764
- Cheung WJ Patey AM Frank JR Mackay M Boet S Barriers and Enablers to Direct Observation of Trainees’ Clinical Performance: A Qualitative Study Using the Theoretical Domains Framework Acad Med 2019 94 101 114
- Massie J Ali JM Workplace-based assessment: a review of user perceptions and strategies to address the identified shortcomings Adv Health Sci Educ Theory Pract 2016 21 455 473
- Cendan JC, Castiglioni A, Johnson TR, et al. Quantitative and Qualitative Analysis of the Impact of Adoption of a Mobile Application for the Assessment of Professionalism in Medical Trainees. Acad Med. 2017;92(11S Association of American Medical Colleges Learn Serve Lead: Proceedings of the 56th Annual Research in Medical Education Sessions):S33–s42.
- Hicks PJ Margolis MJ Carraccio CL A novel workplace-based assessment for competency-based decisions and learner feedback Med Teach 2018 40 1143 1150
-
Cooney
CM
Redett
RJ
3rd Dorafshar AH Zarrabi B Lifchez SD Integrating the NAS Milestones and handheld technology to improve residency training and assessment J Surg Educ 2014 71 39 42 - Page CP Reid A Coe CL Learnings From the Pilot Implementation of Mobile Medical Milestones Application J Grad Med Educ 2016 8 569 575
- Bohnen JD George BC Williams RG The Feasibility of Real-Time Intraoperative Performance Assessment With SIMPL (System for Improving and Measuring Procedural Learning): Early Experience From a Multi-institutional Trial J Surg Educ 2016 73 e118 e30
- Eaton M Scully R Schuller M Value and Barriers to Use of the SIMPL Tool for Resident Feedback J Surg Educ 2019 76 620 627
- Fitzpatrick R Paterson NR Watterson J Seabrook C Roberts M Development and implementation of a mobile version of the O-SCORE assessment tool and case log for competency-based assessment in urology residency training: An initial assessment of utilization and acceptance among residents and faculty Can Urol Assoc J 2019 13 45 50
- Torre DM Simpson DE Elnicki DM Sebastian JL Holmboe ES Feasibility, reliability and user satisfaction with a PDA-based mini-CEX to evaluate the clinical skills of third-year medical students Teach Learn Med 2007 19 271 277
- Malone K Supri S A critical time for medical education: the perils of competence-based reform of the curriculum Adv Health Sci Educ Theory Pract 2012 17 241 246
- ten Cate O Scheele F Competency-based postgraduate training: can we bridge the gap between theory and clinical practice? Acad Med 2007 82 542 547
- Caverzagie KJ Cooney TG Hemmer PA Berkowitz L The development of entrustable professional activities for internal medicine residency training: a report from the Education Redesign Committee of the Alliance for Academic Internal Medicine Acad Med 2015 90 479 484
- Schultz K Griffiths J Lacasse M The Application of Entrustable Professional Activities to Inform Competency Decisions in a Family Medicine Residency Program Acad Med 2015 90 888 897
- Young JQ Hasser C Hung EK Developing End-of-Training Entrustable Professional Activities for Psychiatry: Results and Methodological Lessons Acad Med 2018 93 1048 1054
- Saliken D Dudek N Wood TJ MacEwan M Gofton WT Comparison of the Ottawa Surgical Competency Operating Room Evaluation (O-SCORE) to a Single-Item Performance Score Teach Learn Med 2019 31 146 153
- Warm EJ Held JD Hellmann M Entrusting Observable Practice Activities and Milestones Over the 36 Months of an Internal Medicine Residency Acad Med 2016 91 1398 1405
- Young JQ, McClure M. Fast, Easy, and Good: Assessing Entrustable Professional Activities in Psychiatry Residents with a Mobile App. Acad Med. 2020. https://doi.org/10.1097/ACM.0000000000003390.
- Gunderson K Sullivan S Warner-Hillard C Examining the Impact of Using the SIMPL Application on Feedback in Surgical Education J Surg Educ 2018 75 e246 e54
- Fryer JP Teitelbaum EN George BC Effect of Ongoing Assessment of Resident Operative Autonomy on the Operating Room Environment J Surg Educ 2018 75 333 343
- George BC Teitelbaum EN Meyerson SL Reliability, validity, and feasibility of the Zwisch scale for the assessment of intraoperative performance J Surg Educ 2014 71 e90 6
- Denison A Bate E Thompson J Tablet versus paper marking in assessment: feedback matters Perspect Med Educ 2016 5 108 113
- Kozin ED Bohnen JD George BC Novel Mobile App Allows for Fast and Validated Intraoperative Assessment of Otolaryngology Resid Oto Open 2017 1 2473974 x16685705
- Kaban LB Cappetta A George BC Lahey ET Bohnen JD Troulis MJ Evaluation of Oral and Maxillofacial Surgery Residents’ Operative Skills: Feasibility and Engagement Study Using SIMPL Software for a Mobile Phone J Oral Maxillofac Surg 2017 75 2041 2047
- Kobraei EM Bohnen JD George BC Uniting Evidence-Based Evaluation with the ACGME Plastic Surgery Milestones: A Simple and Reliable Assessment of Resident Operative Performance Plast Reconstr Surg 2016 138:349e 57e
- Kirk MA Kelley C Yankey N Birken SA Abadie B Damschroder L A systematic review of the use of the Consolidated Framework for Implementation Research Implement Sci 2016 11 72
- O’Sullivan PS Yuan P Satre DD Wamsley M Satterfield J A Sequential Implementation Model for Workforce Development: A Case Study of Medical Residency Training for Substance Use Concerns Teach Learn Med 2018 30 84 94
- Price DW Wagner DP Krane NK What are the implications of implementation science for medical education? Med Educ Online 2015 20 27003
- Damschroder LJ Aron DC Keith RE Kirsh SR Alexander JA Lowery JC Fostering implementation of health services research findings into practice: a consolidated framework for advancing implementation science Implement Sci 2009 4 50
- Young JQ Irby DM Kusz M O’Sullivan PS Performance Assessment of Pharmacotherapy: Results from a Content Validity Survey of the Psychopharmacotherapy-Structured Clinical Observation (P-SCO) Tool Acad Psychiatry 2018 42 765 772
- Young JQ Lieu S O’Sullivan P Tong L Development and initial testing of a structured clinical observation tool to assess pharmacotherapy competence Acad Psychiatry 2011 35 27 34
- Young JQ Sugarman R Holmboe E O’Sullivan PS Advancing Our Understanding of Narrative Comments Generated by Direct Observation Tools: Lessons From the Psychopharmacotherapy-Structured Clinical Observation J Grad Med Educ 2019 11 570 579
- Hsieh HF Shannon SE Three approaches to qualitative content analysis Qual Health Res 2005 15 1277 1288
- Zhang Y Wildemuth BM Qualitative analysis of content In: Wildemuth Bm Ed Appl Soc Res Methods To Quest Inf Libr Sci Westport Ct: Libr Unlimited; 2009 p 308 319
- Varpio L Ajjawi R Monrouxe LV O’Brien BC Rees CE Shedding the cobra effect: problematising thematic emergence, triangulation, saturation and member checking Med Educ 2017 51 40 50
- Pereira EA Dean BJ British surgeons’ experiences of a mandatory online workplace based assessment portfolio resurveyed three years on J Surg Educ 2013 70 59 67
- Li SA Sherbino J Chan TM McMaster Modular Assessment Program (McMAP) Through the Years: Residents’ Experience With an Evolving Feedback Culture Over a 3-year Period Aem Educ Train 2017 1 5 14
- Govaerts M Workplace-Based Assessment and Assessment for Learning: Threats to Validity J Grad Med Educ 2015 7 265 267
- Ten Cate O Hart D Ankel F Entrustment Decision Making in Clinical Training Acad Med 2016 91 191 198
- Cook DA Dupras DM Beckman TJ Thomas KG Pankratz VS Effect of rater training on reliability and accuracy of mini-CEX scores: a randomized, controlled trial J Gen Intern Med 2009 24 74 79
- Kogan JR Hatala R Hauer KE Holmboe E Guidelines: the do’s, don’ts and don’t knows of direct observation of clinical skills in medical education Perspect Med Educ 2017 6 286 305
- Watling CJ, Kenyon CF, Zibrowski EM, et al. Rules of engagement: residents’ perceptions of the in-training evaluation process. Acad Med. 2008;83(10 Suppl):S97–100.
- Holmboe ES Work-based Assessment and Co-production in Postgraduate Medical Training GMS J Med Educ 2017 34 Doc58
- Nicol DJ Macfarlane-Dick D Formative assessment and self-regulated learning: a model and seven principles of good feedback practice Stud High Educ 2006 31 199 218
- Watling C LaDonna KA Lingard L Voyer S Hatala R “Sometimes the work just needs to be done”: socio-cultural influences on direct observation in medical training Med Educ 2016 50 1054 1064
- Harrison CJ Konings KD Molyneux A Schuwirth LW Wass V van der Vleuten CP Web-based feedback after summative assessment: how do students engage? Med Educ 2013 47 734 744
- Driessen EW van Tartwijk J Govaerts M Teunissen P van der Vleuten CP The use of programmatic assessment in the clinical workplace: a Maastricht case report Med Teach 2012 34 226 231