Have a personal or library account? Click to login
Competence By Design: a transformational national model of time-variable competency-based postgraduate medical education Cover

Competence By Design: a transformational national model of time-variable competency-based postgraduate medical education

Perspectives on Medical Education
Volume 13 (2024): Issue 1
By: Jason R. Frank,  Jolanta Karpinski,  Jonathan Sherbino,  Linda S. Snell,  Adelle Atkinson,  Anna Oswald,  Andrew K. Hall,  Lara Cooke,  Susan Dojeiji,  Denyse Richardson,  Warren J. Cheung,  Rodrigo B. Cavalcanti,  Timothy R. Dalseg,  Brent Thoma,  Leslie Flynn,  Wade Gofton,  Nancy Dudek,  Farhan Bhanji,  Brian M.-F. Wong,  Saleem Razack,  Robert Anderson,  Daniel Dubois,  Andrée Boucher,  Marcio M. Gomes,  Sarah Taber,  Lisa J. Gorman,  Jane Fulford,  Viren Naik,  Kenneth A. Harris,  Rhonda St. Croix and  Elaine van Melle  
Open Access
|Mar 2024

References

  1. 1Snell L, Frank JR, Pihlak R, Sa J. Postgraduate medical education: a ‘pipeline’ to competence. In: Dent JA, Harden RM, Hunt D, (eds.), A Practical Guide for Medical Teachers. 5th ed. New York: Elsevier; 2017.
    Back to article
  2. 2Cooke M, Irby DM, Sullivan W, Ludmerer K. American medical education 100 years after the Flexner report. NEJM. 2006; 355: 1339–44. DOI: 10.1056/NEJMra055445
    Back to article
  3. 3Ludmerer KM. Time to Heal: American Medical Education from the Turn of the Century to the Era of Managed Care. New York: Oxford University Press; 1999.
    Back to article
  4. 4Neufeld VR, Maudsley RF, Pickering RJ, Walters BC, Turnbull JM, Spasoff RA, et al. Demand-side medical education: educating future physicians for Ontario. CMAJ. 1993; 148: 1471–7.
    Back to article
  5. 5Ludmerer KM, Johns MME. Reforming graduate medical education. JAMA. 2005; 294: 1083–7. DOI: 10.1001/jama.294.9.1083
    Back to article
  6. 6Ludmerer KM. Let Me Heal: The Opportunity to Preserve Excellence in American Medicine. New York: Oxford University Press; 2015.
    Back to article
  7. 7Sales CS, Schlaff AL. Reforming medical education: a review and synthesis of five critiques of medical practice. Soc Sci Med. 2010; 70: 1665–8. DOI: 10.1016/j.socscimed.2010.02.018
    Back to article
  8. 8Iobst WF, Sherbino J, Ten Cate O, Richardson DL, Dath D, Swing SR, et al. Competency-based medical education in postgraduate medical education. Med Teach. 2010; 32: 651–6. DOI: 10.3109/0142159X.2010.500709
    Back to article
  9. 9Irby D. Educating physicians for the future: Carnegie’s calls for reform. Med Teach. 2011; 33: 547–50. DOI: 10.3109/0142159X.2011.578173
    Back to article
  10. 10Frenk J, Chen L, Bhutta ZA, Cohen J, Crisp N, Evans T, et al. Health professionals for a new century: transforming education to strengthen health systems in an interdependent world. Lancet. 2010; 376: 1923–58. DOI: 10.1016/S0140-6736(10)61854-5
    Back to article
  11. 11Boelen C. Adapting health care institutions and medical schools to societies’ needs. Acad Med. 1999; 74(8 Suppl.): S11–S20. DOI: 10.1097/00001888-199908000-00024
    Back to article
  12. 12Barber C, van der Vleuten C, Leppink J, Chahine S. Social accountability frameworks and their implications for medical education and program evaluation: a narrative review. Acad Med. 2020; 95: 1945–54. DOI: 10.1097/ACM.0000000000003731
    Back to article
  13. 13McGaghie WC, Sajid W, Miller GE, Telder TV, Lipson L, et al. Competency-based Curriculum Development in Medical Education. Geneva: World Health Organization; 1978.
    Back to article
  14. 14Harden RM, Crosby JR, Davis MH, Friedman M. AMEE Guide No. 14: Outcome-based education. Part 1. Med Teach. 1999; 21: 546–52. DOI: 10.1080/01421599978951
    Back to article
  15. 15Carraccio C, Wolfsthal SD, Englander R, Ferentz K, Martin C. Shifting paradigms: from Flexner to competencies. Acad Med. 2002; 77: 361–7. DOI: 10.1097/00001888-200205000-00003
    Back to article
  16. 16ten Cate O, Scheele F. Competency-based postgraduate training: Can we bridge the gap between theory and clinical practice? Acad Med. 2007; 82: 542–7. DOI: 10.1097/ACM.0b013e31805559c7
    Back to article
  17. 17Swing SR. The ACGME outcome project: retrospective and prospective. Med Teach. 2007; 29: 648–54. DOI: 10.1080/01421590701392903
    Back to article
  18. 18Frank JR, Snell LS, Ten Cate O, Holmboe ES, Carraccio C, Swing SR, et al. Competency-based medical education: theory to practice. Med Teach. 2010; 32: 638–45. DOI: 10.3109/0142159X.2010.501190
    Back to article
  19. 19Vasquez JA, Marcotte K, Gruppen LD. The parallel evolution of competency-based education in medical and higher education. J Competency-based Educ. 2021; 6: e1234. DOI: 10.1002/cbe2.1234
    Back to article
  20. 20Accreditation Council for Graduate Medical Education (ACGME). Outcome Project. Chicago: ACGME. www.acgme.org/outcome.
    Back to article
  21. 21Powell DE, Carraccio C. Toward competency-based medical education. NEJM. 2018; 378: 3–5. DOI: 10.1056/NEJMp1712900
    Back to article
  22. 22Nasca TJ, Philibert I, Brigham T, Flynn TC. The next GME accreditation system-rationale and benefits. NEJM. 2012; 366: 1051–6. DOI: 10.1056/NEJMsr1200117
    Back to article
  23. 23Holmboe ES, Sherbino J, Englander R, Snell R, Frank JR, ICBME Collaborators. A call to action: the controversy of and rationale for competency-based medical education. Med Teach. 2017; 39: 574–81. DOI: 10.1080/0142159X.2017.1315067
    Back to article
  24. 24McGaghie, WC, Sajid W, Miller GE, Telder TV, Lipson L, et al. Competency-based Curriculum Development in Medical Education. Geneva: World Health Organization; 1978. DOI: 10.1097/ACM.0000000000002743
    Back to article
  25. 25Van Melle E, Frank JR, Holmboe ES, Dagnone D, Stockley D, Sherbino J, International Competency-based Medical Education Collaborators. A core components framework for evaluating implementation of competency-based medical education programs. Acad Med. 2019; 94: 1002–9. DOI: 10.1097/ACM.0000000000002743
    Back to article
  26. 26Chou FC, Hsiao CT, Yang CW, Frank JR. Glocalization in medical education: a framework underlying implementing CBME in a local context. J Formos Med Assoc. 2022; 121: 1523–31. DOI: 10.1016/j.jfma.2021.10.024
    Back to article
  27. 27Schultz K, Griffiths J. Implementing competency-based medical education in a postgraduate family medicine residency training program. Acad Med. 2016; 91: 685–9. DOI: 10.1097/ACM.0000000000001066
    Back to article
  28. 28Shrivasta SR, Shrivastava PS. How to successfully implement competency-based education in India. Educ Health Prof. 2018; 1: 61–63. DOI: 10.4103/EHP.EHP_20_18
    Back to article
  29. 29Yoon CH, Myung SJ, Park WB. Implementing CBME in internal medicine residency training program. J Korean Med Sci. 2019; 34: e201. DOI: 10.3346/jkms.2019.34.e201
    Back to article
  30. 30McKenzie-White J, Mubuuke AG, Westergaard S, Munabi IG, Bollinger RC, et al. Evaluation of a competency based medical curriculum in a Sub-Saharan African medical school. BMC Med Educ. 2022; 22: 724. DOI: 10.1186/s12909-022-03781-1
    Back to article
  31. 31Weller JM, Naik VN, San Diego RJ. Systematic review and narrative synthesis of CBME in anaesthesia. Br J Anaesth. 2020; 124: 748–60. DOI: 10.1016/j.bja.2019.10.025
    Back to article
  32. 32Greiner AC, Knebel E. eds. Health Professions Education: A Bridge to Quality. Washington: National Academies Press; 2003. DOI: 10.1111/j.1945-1474.2004.tb00473.x
    Back to article
  33. 33Shojania KG, Fletcher KE, Saint S. Graduate medical education and patient safety: a busy — and occasionally hazardous — intersection. Ann Intern Med. 2006; 145: 592–8. DOI: 10.7326/0003-4819-145-8-200610170-00008
    Back to article
  34. 34Crosson FJ, Leu J, Roemer BM, Ross MN. Gaps in residency training should be addressed to better prepare doctors for a twenty-first century delivery system. Health Aff (Millwood). 2011; 30: 2412–8. DOI: 10.1377/hlthaff.2011.0184
    Back to article
  35. 35Asch DA, Nicholson S, Srinivas SK, Herrin J, Epstein AJ. How do you deliver a good obstetrician? Outcome-based evaluation of medical education. Academic Medicine. 2014; 89: 24–6. DOI: 10.1097/ACM.0000000000000067
    Back to article
  36. 36Wennberg JE. Unwarranted variations in healthcare delivery: implications for academic medical centres. BMJ. 2002; 325: 961–4. DOI: 10.1136/bmj.325.7370.961
    Back to article
  37. 37Kennedy PJ, Leathley CM, Hughes CF. Clinical practice variation. Med J Aust. 2010; 193: S97–S99. DOI: 10.5694/j.1326-5377.2010.tb04021.x
    Back to article
  38. 38Tamblyn R, Abrahamowicz M, Dauphinee WD, Hanley JA, Norcini J, Girard N, et al. Association between licensure examination scores and practice in primary care. JAMA. 2002; 288: 3019–26. DOI: 10.1001/jama.288.23.3019
    Back to article
  39. 39Teunissen PW, Westerman M. Opportunity, or threat: the ambiguity of the consequences of transitions in medical education. Med Educ. 2011; 45: 51–9. DOI: 10.1111/j.1365-2923.2010.03755.x
    Back to article
  40. 40Colbert-Getz JM, Baumann S, Shaffer K, Lamb S, Lindsley JE, et al. What’s in a transition? An integrative perspective on transitions in medical education. Teach Learn Med. 2016; 28: 347–52. DOI: 10.1080/10401334.2016.1217226
    Back to article
  41. 41Matheson C, Matheson D. How well prepared medical students for their first year as doctors? Postgrad Med J. 2009; 85: 582–9. DOI: 10.1136/pgmj.2008.071639
    Back to article
  42. 42Kilminster S, Zukas M, Quinton N, Roberts T. Preparedness is not enough: understanding transitions as critically intensive learning periods. Med Educ. 2011; 45: 1006–15. DOI: 10.1111/j.1365-2923.2011.04048.x
    Back to article
  43. 43Goldacre MJ, Davidson JM, Lambert TW. The first house officer year: views of graduate and non-graduate entrants to medical school. Med Educ. 2008; 42: 286–93. DOI: 10.1111/j.1365-2923.2007.02992.x
    Back to article
  44. 44Holmboe ES. Faculty and the observation of trainees’ clinical skills: problems and opportunities. Acad Med. 2004; 79: 16–22. DOI: 10.1097/00001888-200401000-00006
    Back to article
  45. 45Holmboe ES. Realizing the promise of competency-based medical education. Acad Med. 2015; 90: 411–3. DOI: 10.1097/ACM.0000000000000515
    Back to article
  46. 46Halpern SD, Detsky AS. Graded autonomy in medical education – managing things that go bump in the night. NEJM. 2014; 370: 1086–9. DOI: 10.1056/NEJMp1315408
    Back to article
  47. 47Cottrell D, Kilminster S, Jolly B, Grant J. What is effective supervision and how does it happen? A critical incident study. Med Educ. 2002; 36: 1042–9. DOI: 10.1046/j.1365-2923.2002.01327.x
    Back to article
  48. 48Kilminster SM, Jolly BC. Effective supervision in clinical practice settings: a literature review. Med Educ. 2001; 34: 827–40. DOI: 10.1046/j.1365-2923.2000.00758.x
    Back to article
  49. 49Baldwin DC, Daugherty SR, Ryan PM. How residents view their clinical supervision: a reanalysis of classic national survey data. J Grad Med Educ. 2010; 2(1): 37–45. DOI: 10.4300/JGME-D-09-00081.1
    Back to article
  50. 50Rietmeijer CBT, Huisman D, Blankenstein AH, de Vries H, Scheele F, Kramer AWM, et al. Patterns of direct observation and their impact during residency: general practice supervisors’ views. Med Educ. 2018: 52: 981–91. DOI: 10.1111/medu.13631
    Back to article
  51. 51Kennedy TJT, Lingard L, Baker GR, Kitchen L, Regehr G. Clinical oversight: conceptualizing the relationship between supervision and safety. J Gen Intern Med. 2017; 22: 1080–5. DOI: 10.1007/s11606-007-0179-3
    Back to article
  52. 52Watling C, LaDonna KA, Lingard L, Voyer S, Hatala R. “Sometimes the work just needs to be done”: socio-cultural influences on direct observation in medical training. Med Educ. 2016; 50: 1054–64. DOI: 10.1111/medu.13062
    Back to article
  53. 53Massie J, Ali JM. Workplace-based assessment: a review of user perceptions and strategies to address the identified shortcomings. Adv Health Sci Educ. 2016; 21: 455–73. DOI: 10.1007/s10459-015-9614-0
    Back to article
  54. 54Holmboe ES, Sherbino J, Long DM, Swing SR, Frank JR. The role of assessment in competency-based medical education. Med Teach. 2010; 32: 646–82. DOI: 10.3109/0142159X.2010.500704
    Back to article
  55. 55Govaerts MJ, van der Vleuten CPM, Schuwirth LWT, Muijtjens AM. Broadening perspectives on clinical performance assessment: rethinking the nature of in-training assessment. Adv Health Sci Educ. 2007; 12: 239–60. DOI: 10.1007/s10459-006-9043-1
    Back to article
  56. 56Herbers JE, Noel GL, Cooper GS, Harvey J, Pangaro LN, Weaver MJ. How accurate are faculty evaluations of clinical competence? J Gen Intern Med. 1989; 4: 202–8. DOI: 10.1007/BF02599524
    Back to article
  57. 57Lockyer J, Carraccio C, Chan MK, Hart D, Smee S, Touchie C, et al. ICBME Collaborators. Core principles of assessment in competency-based medical education. Med Teach. 2017; 39: 609–16. DOI: 10.1080/0142159X.2017.1315082
    Back to article
  58. 58Gingerich A, Regehr G, Eva KW. Rater-based assessments as social judgments: rethinking the etiology of rater errors. Acad Med. 2011; 86: S1–S7. DOI: 10.1097/ACM.0b013e31822a6cf8
    Back to article
  59. 59Sebok-Syer SS, Klinger DA, Sherbino J, Chan TM. Mixed messages or miscommunication? Investigating the relationship between assessors’ workplace-based assessment scores and written comments. Acad Med. 2017; 92: 1774–9. DOI: 10.1097/ACM.0000000000001743
    Back to article
  60. 60Klein R, Julian KA, Snyder ED, Koch J, Ufere NN, Volerman A, et al. Gender bias in resident assessment in graduate medical education: review of the literature. J Gen Intern Med. 2019; 34: 712–9. DOI: 10.1007/s11606-019-04884-0
    Back to article
  61. 61Ross DA, Boatright D, Nunez-Smith M, Jordan A, Chekroud A, Moore EZ. Differences in words used to describe racial and gender groups in medical student performance evaluations. PLOS One. 2017; 12: e0181659. DOI: 10.1371/journal.pone.0181659
    Back to article
  62. 62Archer JC. State of the science in health professional education: effective feedback. Med Educ. 2010; 44: 101–8. DOI: 10.1111/j.1365-2923.2009.03546.x
    Back to article
  63. 63Jensen AR, Wright AS, Kim S, Horvath KD, Calhoun KE. Educational feedback in the operating room: a gap between resident and faculty perceptions. Am J Surg. 2012; 204: 248–55. DOI: 10.1016/j.amjsurg.2011.08.019
    Back to article
  64. 64Watling CJ, Kenyon CF, Zibrowski EM, Schulz V, Goldszmidt MA, Singh I, et al. Rules of engagement: residents’ perceptions of the in-training evaluation process. Acad Med. 2008; 83: S97–S100. DOI: 10.1097/ACM.0b013e318183e78c
    Back to article
  65. 65Watling C, Driessen E, van der Vleuten CPM, Lingard L. Learning culture and feedback: an international study of medical athletes and musicians. Med Educ. 2014; 48: 713–23. DOI: 10.1111/medu.12407
    Back to article
  66. 66Kogan JR, Conforti LN, Bernabeo EC, Durning SJ, Hauer KE, Holmboe ES, et al. Faculty staff perceptions of feedback to residents after direct observation of clinical skills. Med Educ. 2012; 46: 201–15. DOI: 10.1111/j.1365-2923.2011.04137.x
    Back to article
  67. 67Onuoha O, Heins SJ, Clapp JT, Muralidharan M, Baranov DY, Fleisher LE, et al. Improving formative feedback in the operating room setting: developing and implementing an initiative to improve feedback quality and culture. Acad Med. 2022; 97: 222–7. DOI: 10.1097/ACM.0000000000004229
    Back to article
  68. 68Schuwirth LWT, Van Der Vleuten CPM. Programmatic assessment: from assessment of learning to assessment for learning. Med Teach. 2011; 33: 478–85. DOI: 10.3109/0142159X.2011.565828
    Back to article
  69. 69Schuwirth LWT, Van Der Vleuten CPM. Current assessment in medical education: programmatic assessment. J Appl Test Technol. 2019; 20(S2): 2–10.
    Back to article
  70. 70Ten Cate O, Chen HC, Hoff RG, Peters H, Bok H, van der Schaaf M. Curriculum development for the workplace using entrustable professional activities (EPAs): AMEE Guide No. 99. Med Teach. 2015; 37: 983–1002. DOI: 10.3109/0142159X.2015.1060308
    Back to article
  71. 71Ten Cate O. Nuts and bolts of entrustable professional activities. J Grad Med Educ. 2013; 5(1): 157–8. DOI: 10.4300/JGME-D-12-00380.1
    Back to article
  72. 72Graddy R, Reynolds SS, Wright WM. Coaching residents in the ambulatory setting: faculty direct observation and resident reflection. J Grad Med Educ. 2018; 10: 449–54. DOI: 10.4300/JGME-17-00788.1
    Back to article
  73. 73Lockyer J, Armson H, Könings KD, Lee-Krueger RCW, des Ordons AM, Ramani S, et al. In-the-moment feedback and coaching: improving R2C2 for a new context. J Grad Med Educ. 2020; 12: 27–35. DOI: 10.4300/JGME-D-19-00508.1
    Back to article
  74. 74McGaghie WC. Mastery learning: it is time for medical education to join the 21 st century. Acad Med. 2015; 90: 1438–41. DOI: 10.1097/ACM.0000000000000911
    Back to article
  75. 75McGaghie WC, Barsuk JH, Wayne DB. Mastery learning with deliberate practice in medical education. Acad Med. 2015; 90: 1575. DOI: 10.1097/ACM.0000000000000876
    Back to article
  76. 76Colbert CY, Dannefer EF, French JC. Clinical competency committees and assessment: changing the conversation in graduate medical education. J Grad Med Educ. 2015; 7: 162–5. DOI: 10.4300/JGME-D-14-00448.1
    Back to article
  77. 77Hauer KE, Chesluk B, Iobst W, Holmboe E, Baron RD, Boscardin CK, et al. Reviewing residents’ competence: a qualitative study of the role of clinical competency committees in performance assessment. Acad Med. 2015; 90: 1084–92. DOI: 10.1097/ACM.0000000000000736
    Back to article
  78. 78Marty AP, Linsenmeyer M, George B, Young JQ, Breckwoldt J, Ten Cate O. Mobile technologies to support workplace-based assessment for entrustment decisions: guidelines for programs and educators: AMEE Guide No. 154. Med Teach. 2023. Online ahead of print. DOI: 10.1080/0142159X.2023.2168527
    Back to article
  79. 79Van der Schaaf M, Donkers J, Slof B, Moonon-van Loon J, van Tartwijk J, Driessen E, et al. Improving workplace-based assessment and feedback by an eportfolio enhanced with learning analytics. Educ Technol Res Dev. 2017; 65: 359–80. DOI: 10.1007/s11423-016-9496-8
    Back to article
  80. 80Thoma B, Bandi V, Carey R, Mondal D, Woods R, Martin L, Chan T. Developing a dashboard to meet Competence Committee needs: a design-based research project. Canadian Medical Education Journal. 2020; 11(1): e16. DOI: 10.36834/cmej.68903
    Back to article
  81. 81Thoma B, Ellaway RH, Chan TM. From Utopia through Dystopia: charting a course for learning analytics in competency-based medical education. Academic Medicine. 2021; 96(7S): S89–S95. DOI: 10.1097/ACM.0000000000004092
    Back to article
  82. 82Taber S, Akdemir N, Gorman L, van Zanten M, Frank JR. A fit for purpose framework for medical education accreditation system design. BMC Med Educ. 2020; 20(Suppl. 1): 306. DOI: 10.1186/s12909-020-02122-4
    Back to article
  83. 83Edgar L, Jones MD Jr, Harsy B, Passiment M, Hauer KE. Better decision-making: shared mental models and the clinical competency committee. J Grad Med Educ. 2021; 13(2S): 51–8. DOI: 10.4300/JGME-D-20-00850.1
    Back to article
  84. 84Boyd VA, Whitehead CR, Thille P, Ginsburg S, Brydges R, Kuper A. Competency-based medical education: the discourse of infallibility. Med Educ. 2017; 52: 45–57. DOI: 10.1111/medu.13467
    Back to article
  85. 85Brydges R, Boyd VA, Tavares W, Ginsburg S, Kuper A, Anderson M, et al. Assumptions about competency-based medical education and the state of the underlying evidence: a critical narrative review. Acad Med. 2021; 96: 296–306. DOI: 10.1097/ACM.0000000000003781
    Back to article
  86. 86Royal College of Physicians and Surgeons of Canada. Competence by Design. Retrieved 5 February 2023. https://www.royalcollege.ca/rcsite/cbd/competence-by-design-cbd-e.
    Back to article
  87. 87Tugwell P. Postgraduate training in Canada. Postgrad Med J. 1986; 63: 707–9. DOI: 10.1136/pgmj.63.742.707
    Back to article
  88. 88Frank JR, Snell L, Sherbino J, editors. CanMEDS 2015 Physician Competency Framework. Ottawa: Royal College of Physicians and Surgeons of Canada; 2015.
    Back to article
  89. 89Frank JR, Danoff D. The CanMEDS initiative: implementing an outcomes-based framework of physician competencies. Med. Teach. 2007; 29: 642–647. DOI: 10.1080/01421590701746983
    Back to article
  90. 90McGaghie WC, Sajid W, Miller GE, Telder TV, Lipson L, et al. Competency-based Curriculum Development in Medical Education. Geneva: World Health Organization; 1978.
    Back to article
  91. 91Swing S. The ACGME Outcomes Project: retrospective and prospective. Med Teach. 2007; 29: 648–54. DOI: 10.1080/01421590701392903
    Back to article
  92. 92Frenk J, Chen L, Bhutta ZA, Cohen J, Crisp N, Evans T, et al. Health professionals for a new century. Lancet commission. Lancet. 2010; 376: 1923–58. DOI: 10.1016/S0140-6736(10)61854-5
    Back to article
  93. 93Nousiainen MT, Mironova P, Hynes M, Glover Takahashi S, Reznick R, Kraemer W, et al. Eight-year outcomes of a competency-based residency training program in orthopedic surgery. Med Teach. 2018; 40: 1042–54. DOI: 10.1080/0142159X.2017.1421751
    Back to article
  94. 94Ellaway RH, Mackay MP, Lee S, Hofmeister M, Malin G, Archibald D, et al. The impact of a national competency-based medical education initiative in family medicine. Acad Med. 2018; 93: 1850–7. DOI: 10.1097/ACM.0000000000002387
    Back to article
  95. 95Acai A, Li SA, Sherbino J, Chan TM. Attending emergency physicians’ perceptions of a programmatic workplace-based assessment system the McMaster Modular Assessment program (McMAP). Teach Learn Med. 2019; 31: 434–44. DOI: 10.1080/10401334.2019.1574581
    Back to article
  96. 96Li SA, Sherbino J, Chan TM. McMaster Modular Assessment Program (McMAP) through the years: residents’ experience with an evolving feedback culture over a 3-year period. AEM Educ Train. 2017; 1: 5–14. DOI: 10.1002/aet2.10009
    Back to article
  97. 97Bindal T, Wall D, Goodyear HM. Trainee doctors’ views on workplace-based assessments: Are they just a tick box exercise? Med Teach. 2011; 33: 919–27. DOI: 10.3109/0142159X.2011.558140
    Back to article
  98. 98Ott MC, Pack R, Cristancho S, Chin M, Van Koughnett JA, Ott M. “The most crushing thing”: understanding resident assessment burden in a competency-based curriculum. J Grad Med Ed. 2022; 14(5): 583–592. DOI: 10.4300/JGME-D-22-00050.1
    Back to article
  99. 99Yeager DS, Hanselman P, Walton GM, Murray JS, Crosnoe R, Muller C, et al. A national experiment reveals where a growth mindset improves achievement. Nature. 2019; 573: 364–9. DOI: 10.1038/s41586-019-1466-y
    Back to article
  100. 100Ramani S, Könings KD, Ginsburg S, van der Vleuten CPM. Twelve tips to promote a feedback culture with a growth mind-set. Med Teach. 2019; 41: 625–31.
    Back to article
  101. 101Richardson D, Kinnear B, Hauer KE, Turner TL, Warm EJ, Hall AK, et al. Growth mindset in competency-based medical education. Med Teach. 2021; 43: 751–7. DOI: 10.1080/0142159X.2021.1928036
    Back to article
  102. 102Teunissen PW, Westerman M. Opportunity or threat: the ambiguity of the consequences of transitions in medical education. Med Educ. 2011; 45: 51–9. DOI: 10.1111/j.1365-2923.2010.03755.x
    Back to article
  103. 103Holmboe ES, Ginsburg S, Bernabeo E. The rotational approach to medical education: time to confront our assumptions? Med Educ. 2011; 45: 69–80. DOI: 10.1111/j.1365-2923.2010.03847.x
    Back to article
  104. 104Prentice S, Benson J, Kirkpatrick E, Schuwirth L. Workplace-based assessment in postgraduate medical education: a hermeneutic review. Med Educ. 2020; 54: 981–92. DOI: 10.1111/medu.14221
    Back to article
  105. 105Young JQ, Sugarman R, Schwartz J, O’Sullivan PS. Faculty and resident engagement with a workplace-based assessment tool. Acad Med. 2020; 95: 1937–44. DOI: 10.1097/ACM.0000000000003543
    Back to article
  106. 106Frank JR, Taber S, van Zanten M, Scheele F, Blouin D, International Health Professions Accreditation Outcomes Consortium. The role of accreditation in 21 st century health professions education: report of an International Consensus Group. BMC Med Educ. 2020; 20(Suppl. 1): 305. DOI: 10.1186/s12909-020-02121-5
    Back to article
  107. 107Fishbain D, Danon YL, Nissanholz-Gannot R. Accreditation systems for postgraduate medical education: a comparison of five countries. Adv Health Sci Educ. 2019; 24: 503–24. DOI: 10.1007/s10459-019-09880-x
    Back to article
  108. 108Yilmaz Y, Carey R, Chan TM, Bandi V, Wang S, Woods RA, et al. Developing a dashboard for program evaluation in competency-based training programs: A design-based research project. Can Med Educ J. 2022; 13: 14–27. DOI: 10.36834/cmej.73554
    Back to article
  109. 109Frank JR, Jabbour M, Tugwell P; The Societal Needs Working Group. Skills for the new millennium: Report of the Societal Needs Working Group. CanMEDS 2000 Project. Ann R Coll Phys Surg Can. 1996; 29: 206–16.
    Back to article
  110. 110Frank JR, ed. The CanMEDS 2005 Physician Competency Framework. Ottawa: Royal College of Physicians and Surgeons of Canada.
    Back to article
  111. 111Frank JR. The CanMEDS project. In: Dinsdale HB, Hurteau G, (eds.), The Evolution of Specialty Medicine. Ottawa: Royal College of Physicians and Surgeons of Canada; 2004.
    Back to article
  112. 112Maudsley RF, et al. Report of the Task Force to Review Fundamental Issues in Specialty Education. Ottawa: Royal College of Physicians and Surgeons of Canada; 1996.
    Back to article
  113. 113Hornstein HA. The integration of project management and organizational change management is now a necessity. Int J Proj Manage. 2015; 33: 291–8. DOI: 10.1016/j.ijproman.2014.08.005
    Back to article
  114. 114Holmboe ES. Work-based assessment and co-production in postgraduate medical training. GMS J Med Educ. 2017; 34: 58.
    Back to article
  115. 115Buttemer S, Hall J, Berger L, Weersink K, Dagnone JD. Ten ways to get a grip on resident co-production within medical education change. Can Med Educ J. 2020; 11: e124–e129.
    Back to article
  116. 116Karpinski J, Stewart J, Oswald A, Dalseg TR, Atkinson A, Frank JR. Competency based medical education at scale: a road map for transforming national systems of postgraduate medical education. Perspect Med Educ. 2024; 13(1): 24–32. DOI: 10.5334/pme.957
    Back to article
  117. 117Royal College of Physicians and Surgeons of Canada. CBME Leads Directory. Retrieved 5 February 2023. https://www.royalcollege.ca/rcsite/cbd/implementation/getting-started-cbd-local-support-e/cbme-leads-e.
    Back to article
  118. 118Hall AK, Oswald A, Frank JR, Dalseg T, Cheung WJ, Cooke L, et al. Evaluating Competence by Design as a large system change initiative: readiness, fidelity, and outcomes. Perspect Med Educ. 2024; 13(1): 95–107. DOI: 10.5334/pme.962
    Back to article
  119. 119Royal College of Physicians and Surgeons of Canada. Milestones. Retrieved 5 February 2023. https://canmeds.royalcollege.ca/en/milestones.
    Back to article
  120. 120Karpinski J, Frank JR. The role of EPAs in creating a national system of time-variable competency-based medical education. Acad Med. 2021; 96(7S): S36–S41. DOI: 10.1097/ACM.0000000000004087
    Back to article
  121. 121Gofton WT, Dudek NL, Wood TJ, Balaa F, Hamstra SJ. The Ottawa Surgical Competency Operating Room Evaluation Score (O-Score). Acad Med. 2012; 87: 1401–7. DOI: 10.1097/ACM.0b013e3182677805
    Back to article
  122. 122Richardson D, Landreville JM, Trier J, Cheung WJ, Bhanji F, Hall AK, et al. Coaching in Competence by Design: coaching in the moment and coaching over time. Perspect Med Educ. 2024; 13(1): 33–43. DOI: 10.5334/pme.959
    Back to article
  123. 123Royal College of Physicians and Surgeons of Canada. Information by Discipline. Retrieved 5 February 2023. https://www.royalcollege.ca/rcsite/ibd-search-e.
    Back to article
  124. 124Thoma B, Hall AK, Clark, K, Meshkat N, Cheung WJ, Desaulniers P, et al. Evaluation of a national competency-based assessment system in emergency medicine: a CanDREAM study. J Grad Med Educ. 2020; 12: 425–34.
    Back to article
  125. 125Sender Liberman A, Liberman M, Steinert Y, McLeod P, Meterissian S. Surgery residents and attending surgeons have different perceptions of feedback. Med Teach. 2005; 5: 470–2. DOI: 10.1080/0142590500129183
    Back to article
  126. 126Holmboe ES. Direct observation. In: Holmboe ES, Durning SJ, Hawkins RS, (eds.), Practical Guide to the Evaluation of Clinical Competence. 2nd ed. Philadelphia: Mosby-Elsevier; 2017.
    Back to article
  127. 127Westberg J, Hilliard J. Collaborative Clinical Education. New York: Springer; 1993.
    Back to article
  128. 128ten Cate O, Snell L, Mann K, Vermunt J. Orienting teaching toward the learning process. Acad Med. 2004; 79: 219–28. DOI: 10.1097/00001888-200403000-00005
    Back to article
  129. 129Dweck CS. Mindset: The New Psychology of Success. New York: Penguin Random House; 2006.
    Back to article
  130. 130Wolcott MD, McLaughlin JE, Hanne A, Miklavec A, Dallaghan GLB, Rhoney DH, et al. A review to characterise and map the growth mindset theory in health professions education. Med Educ. 2021; 44: 430–40. DOI: 10.1111/medu.14381
    Back to article
  131. 131Goldhamer MEJ, Martinez-Lage M, Black-Schaffer WS, Huang JT, Co JPT, Weinstein DF, et al. Reimagining the clinical competency committee to enhance education and prepare for competency-based time-variable advancement. J Gen Intern Med. 2022; 37: 2280–90. DOI: 10.1007/s11606-022-07515-3
    Back to article
  132. 132Royal College of Physicians and Surgeons of Canada. Competence committees. Retrieved 5 February 2023. https://www.royalcollege.ca/rcsite/cbd/assessment/competence-committees-e.
    Back to article
  133. 133Oswald A, Dubois D, Snell L, Anderson R, Karpinski J, Hall AK, et al. Implementing competence committees on a national scale: design and lessons learned. Perspect Med Educ. 2024; 13(1): 56–67. DOI: 10.5334/pme.961
    Back to article
  134. 134Cheung WC, Bhanji F, Gofton W, Hall AK, Karpinski J, Richardson D, et al. Programmatic assessment in Canadian specialty residency education; implementation and lessons learned. Perspect Med Educ. 2024; 13(1): 44–55. DOI: 10.5334/pme.956
    Back to article
  135. 135Royal College of Physicians and Surgeons of Canada. 2016. CBD Working Group Communique: Assessment. p.4. https://www.royalcollege.ca/content/dam/documents/accreditation/competence-by-design/directory/cbd-policy-communique-assessment-e.pdf. Accessed March 10, 2023.
    Back to article
  136. 136Thoma B, Bandi V, Carey R, Mondal D, Woods R, Martin L, Chan T. Developing a dashboard to meet Competence Committee needs: a design-based research project. Canadian Medical Education Journal. 2020; 11(1): e16. DOI: 10.36834/cmej.68903
    Back to article
  137. 137Carey R, Wilson G, Bandi V, Mondal D, Martin LJ, Woods R, Thoma, B. Developing a dashboard to meet the needs of residents in a competency-based training program: a design-based research project. Canadian Medical Education Journal. 2020; 11(6): e31. DOI: 10.36834/cmej.69682
    Back to article
  138. 138Viberg O, Hatak M, Balter O, Mavroudi A. The current landscape of learning analytics in higher education. Comput Hum Behav. 2018; 89: 98–110. DOI: 10.1016/j.chb.2018.07.027
    Back to article
  139. 139Yilmaz Y, Carey R, Chan TM, Bandi V, Wang S, Woods RA, … Thoma, B. Developing a dashboard for faculty development in competency-based training programs: a design-based research project. Canadian Medical Education Journal. 2021; 12(4): 48–64. DOI: 10.36834/cmej.72067
    Back to article
  140. 140Royal College of Physicians and Surgeons of Canada. Exams and Credentialing. Retrieved 5 February 2023. https://www.royalcollege.ca/rcsite/cbd/assessment/cbd-exams-e.
    Back to article
  141. 141Bhanji F, Naik V, Skoll A, Pittini, Daniels VJ, Bacchus CM. Competence by Design: the role of high-stakes examinations in a competency based medical education system. Perspect Med Educ. 2024; 13(1): 68–74. DOI: 10.5334/pme.965
    Back to article
  142. 142CanRAC consortium. CanERA. Retrieved 5 February 2023. https://www.canera.ca/canrac/home-e.
    Back to article
  143. 143Dalseg TR, Thoma B, Wycliff-Jones K, Frank JR, Taber S. Enabling implementation of competency-based medical education through an outcomes-focused accreditation. Perspect Med Ed. 2024; 13(1): 75–84. DOI: 10.5334/pme.963
    Back to article
  144. 144Ten Cate O, Gruppen LD, Kogan JR, Lingard LA, Teunissen PW. Time-variable training in medicine: theoretical considerations. Acad Med. 2018; 93(3S): S6–S11. DOI: 10.1097/ACM.0000000000002065
    Back to article
  145. 145Tyack DB, Cuban L. Tinkering toward Utopia. Cambridge: Harvard University Press; 1995.
    Back to article
  146. 146Thoma B, Caretta-Weyer H, Schumacher DJ, Warm E, Hall AK, Hamstra SJ, ICBME Collaborators. Becoming a deliberately developmental organization: using competency based assessment data for organizational development. Medical Teacher. 2021; 43(7), 801–809. DOI: 10.1080/0142159X.2021.1925100
    Back to article
  147. 147Fulbrook P, Mooney S. Care bundles in critical care: a practical approach to evidence-based practice. Nurs Crit Care. 2003; 8: 249–55. DOI: 10.1111/j.1362-1017.2003.00039.x
    Back to article
  148. 148Wilson P, Kislov R. Implementation Science. Cambridge: Cambridge University Press. 2022. DOI: 10.1017/9781009237055
    Back to article
  149. 149Cheung WC, Wagner N, Frank JR, Oswald A, Van Melle E, Skutovich A, et al. Implementation of competence committees during the transition to CBME in Canada: national fidelity-focused evaluation. Med Teach. 2022; 44: 781–9. DOI: 10.1080/0142159X.2022.2041191
    Back to article
  150. 150Ahn E, Ladonna KA, Landreville JM, Mcheimech R, Cheung WJ. Only as strong as the weakest link: resident perspectives on entrustable professional activities and their impact on learning. J Grad Med Ed. 15(6): 676–684. DOI: 10.4300/JGME-D-23-00204.1
    Back to article
  151. 151Thoma B, Bandi V, Carey R, Mondal D, Woods R, Martin L, Chan, T. Developing a dashboard to meet Competence Committee needs: a design-based research project. Canadian Medical Education Journal. 2020; 11(1): e16. DOI: 10.36834/cmej.68903
    Back to article
  152. 152Carey R, Wilson G, Bandi V, Mondal D, Martin LJ, Woods R, Thoma, B. Developing a dashboard to meet the needs of residents in a competency-based training program: a design-based research project. Canadian Medical Education Journal. 2020; 11(6): e31. DOI: 10.36834/cmej.69682
    Back to article
  153. 153Yilmaz Y, Carey R, Chan TM, Bandi V, Wang S, Woods RA, Thoma, B. Developing a dashboard for faculty development in competency-based training programs: a design-based research project. Canadian Medical Education Journal. 2021; 12(4): 48–64. DOI: 10.36834/cmej.72067
    Back to article
  154. 154Yilmaz Y, Carey R, Chan TM, Bandi V, Wang S, Woods RA, Thoma, B. Developing a dashboard for program evaluation in competency-based training programs: a design-based research project. Canadian Medical Education Journal. 2022; 13(5): 14–27. DOI: 10.36834/cmej.73554
    Back to article
  155. 155Resident Doctors of Canada and the Royal College of Physicians and Surgeons of Canada. Competence by Design: Resident Physician Pulse Check Report from the 2021 Collaborative Study. September, 2022. Ottawa: Royal College.
    Back to article
  156. 156Hall AK, Nousiainen MT, Campisi P, Dagnone JD, Frank JR, Kroeker KI, et al. Training disrupted: practical tips for supporting competency-based medical education during the COVID-19 pandemic. Med Teach. 2020; 42: 756–61. DOI: 10.1080/0142159X.2020.1766669
    Back to article
  157. 157Prinsloo P, Slade S, Khalil M. The answer is (not only) technological: Considering student data privacy in learning analytics. Br J Educ Technol. 2022; 53: 876–93. DOI: 10.1111/bjet.13216
    Back to article
  158. 158The International CBME Collaborators. Implementation of CBME: A Typology of International Programs. 2021; Unpublished manuscript. Ottawa: ICBMEC.
    Back to article
  159. 159Oandasan I. Advancing Canada’s family medicine curriculum: Triple C. Can. Fam. Phys. 2011; 57(6): 739–740.
    Back to article
  160. 160The Triple-C Competency-based Curriculum. https://www.cfpc.ca/en/education-professional-development/educational-frameworks-and-reference-guides/triple-c-competency-based-curriculum. Accessed November 25, 2023.
    Back to article
  161. 161Incoll I, Atkin J, Owen J, Kean A, Khorshid O, Cosenza A, Frank JR. Australian orthopaedic surgery training: Australian Orthopaedic Association’s strategic education review. ANZ J Surg. 2020; 90(6): 997–1003. DOI: 10.1111/ans.15609
    Back to article
  162. 162de Graaf J, Bolk M, Dijkstra A, van der Horst M, Hoff RG, ten Cate O. The Implementation of Entrustable Professional Activities in Postgraduate Medical Education in the Netherlands: Rationale, Process, and Current Status. Acad. Med. 2021; 96(7S): S29–S35. DOI: 10.1097/ACM.0000000000004110
    Back to article
  163. 163Heard JK, Allen RM, Clardy J. Assessing the Needs of Residency Program Directors to Meet the ACGME General Competencies. Acad. Med. 2002; 77(7): 750. DOI: 10.1097/00001888-200207000-00040
    Back to article
  164. 164de Graaf et al. p.S32.
    Back to article
  165. 165Green ML, Holmboe E. Perspective: the ACGME Toolbox: half empty or half full? Acad. Med. 2010; 85(5): 787–790. DOI: 10.1097/ACM.0b013e3181d737a6
    Back to article
  166. 166Holmboe ES. Realizing the promise of Competency-based Medical Education. Acad. Med. 2015; 90(4): 411–413. DOI: 10.1097/ACM.0000000000000515
    Back to article
  167. 167Swing SR. The ACGME outcome project: retrospective and prospective. Med. Teach. 2007; 29: 648–654. DOI: 10.1080/01421590701392903
    Back to article
  168. 168Holmboe ES, Yamazaki K, Nasca TJ, Hamstra SJ. Using Longitudinal Milestones Data and Learning Analytics to Facilitate the Professional Development of Residents: Early Lessons From Three Specialties. Acad. Med. 2020; 95(1): 97–103. DOI: 10.1097/ACM.0000000000002899
    Back to article
  169. 169Australian Orthopedic Association. About AOA-21. https://aoa.org.au/orthopaedic-training/content-page/about-aoa-21. Accessed November 25, 2023.
    Back to article
DOI: https://doi.org/10.5334/pme.1096 | Journal eISSN: 2212-277X
Journal RSS Feed
Language: English
Submitted on: Jul 4, 2023
Accepted on: Feb 16, 2024
Published on: Mar 18, 2024
Published by: Ubiquity Press
In partnership with: Paradigm Publishing Services
Publication frequency: 1 issue per year

© 2024 Jason R. Frank, Jolanta Karpinski, Jonathan Sherbino, Linda S. Snell, Adelle Atkinson, Anna Oswald, Andrew K. Hall, Lara Cooke, Susan Dojeiji, Denyse Richardson, Warren J. Cheung, Rodrigo B. Cavalcanti, Timothy R. Dalseg, Brent Thoma, Leslie Flynn, Wade Gofton, Nancy Dudek, Farhan Bhanji, Brian M.-F. Wong, Saleem Razack, Robert Anderson, Daniel Dubois, Andrée Boucher, Marcio M. Gomes, Sarah Taber, Lisa J. Gorman, Jane Fulford, Viren Naik, Kenneth A. Harris, Rhonda St. Croix, Elaine van Melle, published by Ubiquity Press
This work is licensed under the Creative Commons Attribution 4.0 License.

Previous articleVolume 13 (2024): Issue 1Next article