Approaches to the Evaluation of Competency-Based Health Professions Education Implementation from Around the Globe
By: Andrew K. Hall, Dorothy Andriole, Aleda M. H. Chen, Robert Englander, David Rojas, Michael S. Ryan, Christy K. Boscardin, Holly Caretta-Weyer, Timothy R. Dalseg, Ian W. Incoll, Roberta I. Ladenheim, Leila Niemi-Murola, Anna Oswald, Severin Pinilla, David A. Turner, Mary Ellen J. Goldhamer and As members of the International Competency-Based Health Professions Educators Collaborative
References
- Frye AW, Hemmer PA. Program evaluation models and related theories: Amee guide no. 67. Med Teach. 2012;34(5):e288–e299. DOI: 10.3109/0142159X.2012.668637
- Englander R, Goldhamer ME, Chen AMH, Oswald A, Atkinson A, Caretta-Weyer H, Chan TM, Chou FC, Dalseg TR, Kini RG, Ladenheim RI, Niemi-Murola L, Turner DA, Snell L, Hall AK, As members of the International Competency-Based Health Professions Educators Collaborative. Competency-Based Health Professions Education in Evolution: Challenges and Adaptations After Implementation. Perspectives on Medical Education. 2026;15(1):930–945. DOI: 10.5334/pme.2580
- Chen AMH, Rhoney DH, Goldhamer MEJ, Ekpenyong A, Humphrey-Murto S, Caretta Weyer H, Snell L, Damodaran A, Englander R, Hall AK, Huwendiek S, Molgaard L, Oswald A, Schumacher DJ, Chou FC, Tanaka P, Frank JR, As members of the International Competency-based Education in the Health Professions Collaborators. The CBHPE Outcomes Theory of Change: An International Consensus Statement on the Mechanisms and Outcomes of Competency Based Education in the Health Professions. Perspectives on Medical Education. 2026;15(1):732–747. DOI: 10.5334/pme.2553
- Patton MQ. Evaluation science. American Journal of Evaluation. 2018;39(2):183–200. DOI: 10.1177/1098214018763121
- Van Melle E, Frank JR, Holmboe ES, et al. A core components framework for evaluating implementation of competency-based medical education programs. Acad Med. 2019;94(7):1002–1009. DOI: 10.1097/ACM.0000000000002743
- Weiner BJ, Amick H, Lee SY. Conceptualization and measurement of organizational readiness for change: A review of the literature in health services research and other fields. Med Care Res Rev. 2008;65(4):379–436. DOI: 10.1177/1077558708317802
- Kotter JP. Leading change. Boston, MA, USA: Harvard Business School Press; 1996.
- Scaccia JP, Cook BS, Lamont A, et al. A practical implementation science heuristic for organizational readiness: R = mc(2). J Community Psychol. 2015;43(4):484–501. DOI: 10.1002/jcop.21698
- Cheung WJ, Hall AK, Skutovich A, et al. Ready, set, go! Evaluating readiness to implement competency-based medical education. Med Teach. 2022;44(8):886–892. DOI: 10.1080/0142159X.2022.2041585
- Van Melle E, Hall AK, Schumacher DJ, et al. Capturing outcomes of competency-based medical education: The call and the challenge. Med Teach. 2021;43(7):794–800. DOI: 10.1080/0142159X.2021.1925640
- Mowbray CT, Holter MC, Teague GB, Bybee D. Fidelity criteria: Development, measurement, and validation. American journal of evaluation. 2003;24(3):315–340. DOI: 10.1177/109821400302400303
- Hall AK, Rich J, Dagnone JD, et al. It’s a marathon, not a sprint: Rapid evaluation of competency-based medical education program implementation. Acad Med. 2020;95(5):786–793. DOI: 10.1097/ACM.0000000000003040
- Patton MQ. Essentials of utilization-focused evaluation. Thousand Oaks, CA: Sage Publications; 2011.
- Hall G, Hord S. Implementing change: Patterns, principles, and potholes. 4th ed. Upper Saddle River, NJ: Pearson; 2015.
- Van Melle E. Using a logic model to assist in the planning, implementation, and evaluation of educational programs. Acad Med. 2016;91(10):1464. DOI: 10.1097/ACM.0000000000001282
- Pawson R, Greenhalgh T, Harvey G, Walshe K. Realist review—a new method of systematic review designed for complex policy interventions. J Health Serv Res Policy. 2005;10 Suppl 1(1_suppl):21–34. DOI: 10.1258/1355819054308530
- Hall AK, Oswald A, Frank JR, et al. Evaluating competence by design as a large system change initiative: Readiness, fidelity, and outcomes. Perspect Med Educ. 2024;13(1):95–107. DOI: 10.5334/pme.962
- Van Melle E,
editor Competency-based medical education: How do we know it works? (plenary) . World Summit on CBME; 2018; Basel, Switzerland. - Competence by design adaptations plan 2025. Available from:
https://news.royalcollege.ca/content/dam/document/newsroom/CBD%20Adaptations%20Plan.pdf . - Braund H, Dagnone JD, Hall AK, et al. Competency based medical education implementation at the institutional level: A cross-discipline comparative program evaluation. Med Teach. 2025;47(4):705–712. DOI: 10.1080/0142159X.2024.2362909
- Acker A, Leifso K, Crawford L, et al. Lessons learned and new strategies for success: Evaluating the implementation of competency-based medical education in queen’s pediatrics. Paediatr Child Health. 2023;28(8):463–467. DOI: 10.1093/pch/pxad021
- Chung AD, Kwan BYM, Wagner N, et al. An adaptation-focused evaluation of canada’s first competency-based medical education implementation in radiology. Eur J Radiol. 2022;147:110109. DOI: 10.1016/j.ejrad.2021.110109
- Szulewski A, Braund H, Dagnone DJ, et al. The assessment burden in competency-based medical education: How programs are adapting. Acad Med. 2023;98(11):1261–1267. DOI: 10.1097/ACM.0000000000005305
- Rachul C, Collins B, Chan MK, Srinivasan G, Hamilton J. Rivalries for attention: Insights from a realist evaluation of a postgraduate competency-based medical education implementation in canada. BMC Med Educ. 2022;22(1):583. DOI: 10.1186/s12909-022-03661-8
- Amiel J, Ryan MS, Andriole DA, Whelan AJ. Core entrustable professional activities for entering residency: Summary of the 10-school pilot, 2014–2021. Association of American Medical Colleges; 2022.
- Brown DR, Warren JB, Hyderi A, et al. Finding a path to entrustment in undergraduate medical education: A progress report from the AAMC core entrustable professional activities for entering residency entrustment concept group. Acad Med. 2017;92(6):774–779. DOI: 10.1097/ACM.0000000000001544
- Lomis K, Amiel JM, Ryan MS, et al. Implementing an entrustable professional activities framework in undergraduate medical education: Early lessons from the AAMC core entrustable professional activities for entering residency pilot. Acad Med. 2017;92(6):765–770. DOI: 10.1097/ACM.0000000000001543
- Amiel JM, Andriole DA, Biskobing DM, et al. Revisiting the core entrustable professional activities for entering residency. Acad Med. 2021;96(7S):S14–S21. DOI: 10.1097/ACM.0000000000004088
- Englander R, Flynn T, Call S, et al. Toward defining the foundation of the md degree: Core entrustable professional activities for entering residency. Acad Med. 2016;91(10):1352–1358. DOI: 10.1097/ACM.0000000000001204
- Hamza DM, Ross S, Oandasan I. Process and outcome evaluation of a cbme intervention guided by program theory. J Eval Clin Pract. 2020. DOI: 10.1111/jep.13344
- Riaz Q, Khan MR, Ahmed M. Competency-based postgraduate medical education as an institutional initiative in a low- middle-income country: Program directors’ perceptions of early implementation. BMC Med Educ. 2025;25(1):912. DOI: 10.1186/s12909-025-07065-2
- Meriranta N, Kaartinen S, Antila A, et al. The quality of finland’s postgraduate medical education: A national perspective from programme directors. BMC Med Educ. 2026;26(1). DOI: 10.1186/s12909-026-09062-5
- Thoma B, Hall AK, Clark K, et al. Evaluation of a national competency-based assessment system in emergency medicine: A candream study. Journal of Graduate Medical Education. 2020;12(4):425–434. DOI: 10.4300/jgme-d-19-00803.1
- Lockyer J, Carraccio C, Chan MK, et al. Core principles of assessment in competency-based medical education. Med Teach. 2017;39(6):609–616. DOI: 10.1080/0142159X.2017.1315082
- Touchie C, Kinnear B, Schumacher D, et al. On the validity of summative entrustment decisions. Med Teach. 2021;43(7):780–787. DOI: 10.1080/0142159X.2021.1925642
- Messick S. Validity of psychological assessment: Validation of inferences from persons’ responses and performances as scientific inquiry into score meaning. American Psychologist. 1995;50(9):741–749. DOI: 10.1037/0003-066x.50.9.741
- Cook DA, Brydges R, Ginsburg S, Hatala R. A contemporary approach to validity arguments: A practical guide to kane’s framework. Med Educ. 2015;49(6):560–575. DOI: 10.1111/medu.12678
- Touchie C, ten Cate O, Park YS, Kinnear B, Taylor DR.
Validity theory applied to entrustment as an approach to assessment . In: ten Cate O, Burch VC, Chen HC, Chou FC, Hennus MP, editors. Entrustable professional activities and entrustment decision-making in health professions education. 1st ed. London: Ubiquity Press; 2024. pp. 51–63. DOI: 10.5334/bdc.e - Baynouna AlKetbi L, Nagelkerke N, AlZarouni AA, et al. Assessing the impact of adopting a competency-based medical education framework and acgme-i accreditation on educational outcomes in a family medicine residency program in abu dhabi emirate, united arab emirates. Front Med (Lausanne). 2023;10:1257213. DOI: 10.3389/fmed.2023.1257213
- Ryan MS, Khan AR, Park YS, et al. Workplace-based entrustment scales for the core epas: A multisite comparison of validity evidence for two proposed instruments using structured vignettes and trained raters. Acad Med. 2022;97(4):544–551. DOI: 10.1097/ACM.0000000000004222
- Cutrer WB, Russell RG, Davidson M, Lomis KD. Assessing medical student performance of entrustable professional activities: A mixed methods comparison of co-activity and supervisory scales. Med Teach. 2020;42(3):325–332. DOI: 10.1080/0142159X.2019.1686135
- Ryan MS, Gielissen KA, Shin D, et al. How well do workplace-based assessments support summative entrustment decisions? A multi-institutional generalisability study. Med Educ. 2024;58(7):825–837. DOI: 10.1111/medu.15291
- Brown DR, Moeller JJ, Grbic D, et al. Comparing entrustment decision-making outcomes of the core entrustable professional activities pilot, 2019–2020. JAMA Netw Open. 2022;5(9):
e2233342 . DOI: 10.1001/jamanetworkopen.2022.33342 - Swing SR. The acgme outcome project: Retrospective and prospective. Med Teach. 2007;29(7):648–654. DOI: 10.1080/01421590701392903
- Nasca TJ, Philibert I, Brigham T, Flynn TC. The next gme accreditation system—rationale and benefits. N Engl J Med. 2012;366(11):1051–1956. DOI: 10.1056/NEJMsr1200117
- Edgar L, Hatlak K, Haynes I, et al. The milestones guidebook: Competency-based medical education and milestones development. Chicago, IL, USA: Accreditation Council for Graduate Medical Education (ACGME). 2025.
- Hamstra SJ, Yamazaki K. A validity framework for effective analysis and interpretation of milestones data. J Grad Med Educ. 2021;13(2 Suppl):75–80. DOI: 10.4300/JGME-D-20-01039.1
- Yamazaki K, Shammet R, Lekdee A, et al.
Ppv report 2024 . Chicago, IL, USA: Accreditation Council for Graduate Medical Education (ACGME). 2024. - Hamstra SJ, Yamazaki K, Barton MA, et al. A national study of longitudinal consistency in acgme milestone ratings by clinical competency committees: Exploring an aspect of validity in the assessment of residents’ competence. Acad Med. 2019;94(10):1522–1531. DOI: 10.1097/ACM.0000000000002820
- Beeson MS, Holmboe ES, Korte RC, et al. Initial validity analysis of the emergency medicine milestones. Acad Emerg Med. 2015;22(7):838–844. DOI: 10.1111/acem.12697
- Murray KE, Lane JL, Carraccio C, et al. Crossing the gap: Using competency-based assessment to determine whether learners are ready for the undergraduate-to-graduate transition. Acad Med. 2019;94(3):338–345. DOI: 10.1097/ACM.0000000000002535
- Violato C, Cullen MJ, Englander R, et al. Validity evidence for assessing entrustable professional activities during undergraduate medical education. Acad Med. 2021;96(7S):S70–S75. DOI: 10.1097/ACM.0000000000004090
- Hobday PM, Borman-Shoap E, Cullen MJ, Englander R, Murray KE. The minnesota method: A learner-driven, entrustable professional activity-based comprehensive program of assessment for medical students. Acad Med. 2021;96(7S):S50–S55. DOI: 10.1097/ACM.0000000000004101
- Schumacher DJ, Michelson C, Winn A, et al. Graduating residents’ readiness for unsupervised practice. Pediatrics. 2025;155(5). DOI: 10.1542/peds.2024-070307
- Mink RB, Schwartz A, Mahan JD, et al. Level of supervision for the entrustable professional activities common to general pediatrics and the subspecialties decreases from residency to fellowship. Acad Pediatr. 2024;24(7):1025–1030. DOI: 10.1016/j.acap.2024.04.004
- Czaja AS, Schwartz A, James SH, et al. Validity evidence for level of supervision scales with pediatric subspecialty-specific entrustable professional activities. Acad Pediatr. 2025;25(8):103116. DOI: 10.1016/j.acap.2025.103116
- Hall AK, Schumacher DJ, Thoma B, et al. Outcomes of competency-based medical education: A taxonomy for shared language. Med Teach. 2021;43(7):788–793. DOI: 10.1080/0142159X.2021.1925643
- Nousiainen MT, Mironova P, Hynes M, et al. Eight-year outcomes of a competency-based residency training program in orthopedic surgery. Med Teach. 2018:1–13. DOI: 10.1080/0142159X.2017.1421751
- de Graaf J, Bolk M, Dijkstra A, et al. The implementation of entrustable professional activities in postgraduate medical education in the netherlands: Rationale, process, and current status. Acad Med. 2021;96(7S):S29–S35. DOI: 10.1097/ACM.0000000000004110
- Dalseg TR, Hall AK, Williams K, et al.
Resident physician pulse check; report from the 2021 collaborative study . Ottawa, ON: Royal College of Physicians and Surgeons of Canada & Resident Doctors of Canada; 2022. - Fédération des médecins résidents du Québec.
Profile of competence by design – year 4 . Fédération des médecins résidents du Québec; 2022. - Ott MC, Pack R, Cristancho S, et al. “The most crushing thing”: Understanding resident assessment burden in a competency-based curriculum. J Grad Med Educ. 2022;14(5):583–592. DOI: 10.4300/JGME-D-22-00050.1
- Day LB, Colbourne T, Ng A, et al. A qualitative study of canadian resident experiences with competency-based medical education. Can Med Educ J. 2023;14(2):40–50. DOI: 10.36834/cmej.72765
- Balmer DF, Rama JA, Simpson D. Implementing the evaluation plan and analysis: Who, what, when, and how. J Grad Med Educ. 2021;13(1):129–130. DOI: 10.4300/JGME-D-20-01523.1
- Simpson D, Riddle JM, Hamel DL
Jr. , Balmer DF. Blueprinting program evaluation evidence through the lens of key stakeholders. J Grad Med Educ. 2020;12(5):629–630. DOI: 10.4300/JGME-D-20-01015.1 - Goldhamer MEJ, Pusic MV, Nadel ES, Co JPT, Weinstein DF. Promotion in place: A model for competency-based, time-variable graduate medical education. Acad Med. 2024;99(5):518–523. DOI: 10.1097/ACM.0000000000005652
- Balmer DF, Pusic MV, Weinstein DF, Co JPT, Goldhamer MEJ. In the eye of the beholder: A stakeholder analysis of the value of the “promotion in place” competency-based time-variable graduate medical education pilot. Acad Med. 2025;100(3):331–339. DOI: 10.1097/ACM.0000000000005928
- Bok HG, Teunissen PW, Favier RP, et al. Programmatic assessment of competency-based workplace learning: When theory meets practice. BMC Med Educ. 2013;13(1):123. DOI: 10.1186/1472-6920-13-123
- Kinnear B, Santen SA, Kelleher M, et al. How does timeless training impact resident motivation for learning, assessment, and feedback? Evaluating a competency-based time-variable training pilot. Acad Med. 2023;98(7):828–835. DOI: 10.1097/ACM.0000000000005147
- Lee SY, Shin JS, Lee SH. How to execute context, input, process, and product evaluation model in medical health education. J Educ Eval Health Prof. 2019;16:40. DOI: 10.3352/jeehp.2019.16.40
- İlhan E. Evaluation of competency based medical education curriculum. International Journal of Progressive Education. 2021;17(3):153–168. DOI: 10.29329/ijpe.2021.346.10
- Andrews JS, Bale JF
Jr. , Soep JB, et al. Education in pediatrics across the continuum (epac): First steps toward realizing the dream of competency-based education. Acad Med. 2018;93(3):414–420. DOI: 10.1097/ACM.0000000000002020 - O’Connor MM, Hobday PM, Long M, et al. Competency-based advancement in medical education: Outcomes of the education in pediatrics across the continuum (epac) project. Academic Medicine; 2025. DOI: 10.1093/acamed/wvaf066
- Caro Monroig AM, Chen HC, Carraccio C, et al. Medical students’ perspectives on entrustment decision making in an entrustable professional activity assessment framework: A secondary data analysis. Acad Med. 2021;96(8):1175–1181. DOI: 10.1097/ACM.0000000000003858
- Balmer DF, Gottlieb-Smith RJ, Hobday PM, et al. Pediatric career choice: Insights from a novel, medical school pathway program. Acad Pediatr. 2020;20(1):97–103. DOI: 10.1016/j.acap.2019.07.013
- Ross S, Binczyk NM, Hamza DM, et al. Association of a competency-based assessment system with identification of and support for medical residents in difficulty. JAMA Netw Open. 2018;1(7):
e184581 . DOI: 10.1001/jamanetworkopen.2018.4581 - Martinez-Lage M, Goldhamer MEJ, Pusic MV, et al. Implementing the promotion in place model of competency-based time-variable graduate medical education in pathology. J Grad Med Educ. 2025;17(2 Suppl):64–71. DOI: 10.4300/JGME-D-24-00664.1
- Chan TM, Paterson QS, Hall AK, et al. Outcomes in the age of competency-based medical education: Recommendations for emergency medicine training in canada from the 2019 symposium of academic emergency physicians. Cjem. 2020;22(2):204–214. DOI: 10.1017/cem.2019.491
- Ashman B, Cosenza A, Frank J, et al. Competency-based training for orthopedic surgery: An evaluation of the australian orthopedic association training program 2017–2021. J Surg Educ. 2025;82(8):103550. DOI: 10.1016/j.jsurg.2025.103550
- Frank JR, Snell LS, Cate OT, et al. Competency-based medical education: Theory to practice. Med Teach. 2010;32(8):638–645. DOI: 10.3109/0142159X.2010.501190
- Asch DA, Nicholson S, Srinivas S, Herrin J, Epstein AJ. Evaluating obstetrical residency programs using patient outcomes. JAMA. 2009;302(12):1277–1283. DOI: 10.1001/jama.2009.1356
- Tamblyn R. Outcomes in medical education: What is the standard and outcome of care delivered by our graduates? Adv Health Sci Educ Theory Pract. 1999;4(1):9–25. DOI: 10.1023/A:1009893715930
- Castellanos-Ortega A, Broch Porcar MJ, Palacios-Castaneda D, et al. Effect of a competence based medical education program on training quality in intensive care medicine. Cobalidation trial. Med Intensiva (Engl Ed). 2025;49(6):502126. DOI: 10.1016/j.medine.2024.502126
- Han M, Hamstra SJ, Hogan SO, et al. Trainee physician milestone ratings and patient complaints in early posttraining practice. JAMA Netw Open. 2023;6(4):
e237588 . DOI: 10.1001/jamanetworkopen.2023.7588 - Smith BK, Yamazaki K, Tekian A, et al. Accreditation council for graduate medical education milestone training ratings and surgeons’ early outcomes. JAMA Surg. 2024;159(5):546–552. DOI: 10.1001/jamasurg.2024.0040
- Michie S, van Stralen MM, West R. The behaviour change wheel: A new method for characterising and designing behaviour change interventions. Implement Sci. 2011;6(1):42. DOI: 10.1186/1748-5908-6-42
- Young M, St-Onge C, Xiao J, Vachon Lachiver E, Torabi N. Characterizing the literature on validity and assessment in medical education: A bibliometric study. Perspect Med Educ. 2018;7(3):182–191. DOI: 10.1007/s40037-018-0433-x
- Holmboe ES. The transformational path ahead: Competency-based medical education in family medicine. Fam Med. 2021;53(7):583–589. DOI: 10.22454/FamMed.2021.296914
- Schumacher DJ, Gielissen K, Kinnear B. Competency-based medical education: Connecting training outcomes to patient care. Curr Probl Pediatr Adolesc Health Care. 2024;54(10):101675. DOI: 10.1016/j.cppeds.2024.101675
- Haji F, Morin MP, Parker K. Rethinking programme evaluation in health professions education: Beyond ‘did it work?’. Med Educ. 2013;47(4):342–351. DOI: 10.1111/medu.12091
- Chen AMH, Rhoney DH, Humphrey-Murto S, Goldhamer MEJ, Snell L, Caretta-Weyer H, Schumacher DJ, Huwendiek S, Damodaran A, Hall AK, Englander R, Molgaard L, Oswald A, Tanaka P, Chou FC, Frank JR, as members of the International Competencybased Education in the Health Professions Collaborators. CORE-HPE Reporting Guidelines: Standardizing Reporting for Competency-Based Health Professions Education Outcomes Research. Perspectives on Medical Education. 2026;15(1): 894–915. DOI: 10.5334/pme.2554
- Dados N, Connell R. The global south. Contexts. 2012;11(1):12–13. DOI: 10.1177/1536504212436479
- Hamza DM, Regehr G. Eco-normalization: Evaluating the longevity of an innovation in context. Acad Med. 2021;96(11S):S48–S53. DOI: 10.1097/ACM.0000000000004318
- Hobday PM, Borman-Shoap E, Cullen M, et al. Competency-based advancement in medical education: Outcomes of the education in pediatrics across the continuum (epac) project. Academic Pediatrics. 2025;25(1). DOI: 10.1016/j.acap.2024.102656
DOI: https://doi.org/10.5334/pme.2628 | Journal eISSN: 2212-277X
Language: English
Page range: 1001 - 1016
Submitted on: Mar 29, 2026
Accepted on: Jul 7, 2026
Published on: Sep 29, 2026
Published by: Ubiquity Press
In partnership with: Paradigm Publishing Services
© 2026 Andrew K. Hall, Dorothy Andriole, Aleda M. H. Chen, Robert Englander, David Rojas, Michael S. Ryan, Christy K. Boscardin, Holly Caretta-Weyer, Timothy R. Dalseg, Ian W. Incoll, Roberta I. Ladenheim, Leila Niemi-Murola, Anna Oswald, Severin Pinilla, David A. Turner, Mary Ellen J. Goldhamer, As members of the International Competency-Based Health Professions Educators Collaborative, published by Ubiquity Press
This work is licensed under the Creative Commons Attribution 4.0 License.