Competence By Design: a transformational national model of time-variable competency-based postgraduate medical education

Jason R. Frank; Jolanta Karpinski; Jonathan Sherbino; Linda S. Snell; Adelle Atkinson; Anna Oswald; Andrew K. Hall; Lara Cooke; Susan Dojeiji; Denyse Richardson; Warren J. Cheung; Rodrigo B. Cavalcanti; Timothy R. Dalseg; Brent Thoma; Leslie Flynn; Wade Gofton; Nancy Dudek; Farhan Bhanji; Brian M.-F. Wong; Saleem Razack; Robert Anderson; Daniel Dubois; Andrée Boucher; Marcio M. Gomes; Sarah Taber; Lisa J. Gorman; Jane Fulford; Viren Naik; Kenneth A. Harris; Rhonda St. Croix; Elaine van Melle

doi:10.5334/pme.1096

Figures & Tables

Table 1

Drivers of the Competence by Design project.

ISSUES OF CONCERN IN PGME SYSTEM	OPPORTUNITIES FOR PGME SYSTEM ENHANCEMENT
Public expectations for greater social accountability of health professions and their education systems [11, 12] Calls for greater focus on outcomes of training [14, 15, 16, 17, 18] Patient safety concerns with care provided during and after postgraduate training [32, 33, 34] Evidence of unacceptable variability in the competence of medical graduates [35, 36, 37, 38]	Recommendations to shift to outcomes-oriented, competency-based systems have been made by major medical organizations (e.g., World Health Organization) [90, 91, 92] There are successful CBME implementations to build upon (e.g., Toronto Orthopedics [93], CFPC [94], ACGME [91])
Little direct observation of trainees at work Incidents of inadequate supervision of trainees [44, 45, 46, 47, 48, 49, 50, 51, 52]	Use of entrustable professional activities allow more faculty to provide better input on trainee progress [70, 71]
Failure to address identified weaknesses in trainees (“failure to fail”) Certification examination failures Promotion despite evidence of gaps or unreadiness for practice Concerns about promotion decision-making Concerns about inadequate workplace-based assessments Few supervisors involved in workplace assessment [53, 54, 55, 56, 57, 58, 59]	Use of programmatic assessment can enhance assessment decisions [68, 69, 95] Application of learning analytics to medical education allows for new insights into trainee progression toward competence [81, 96]
Reports of workplace assessments perceived as burdensome [53]	Development of electronic portfolio software allows digitization and documentation of assessments [78, 79, 80]
Reports of inadequate quality and frequency of the feedback given to trainees [62, 63, 64, 65, 66, 67]	New coaching models have been developed for medical education [72, 73] A CanMEDS framework update incorporates developmental milestones that can be used as scaffolding for supervisor feedback [88]
Reports of trainee anxiety with workplace assessment [97, 98]	Growth mindset may enhance learning [99, 100, 101]
Issues of transitions to postgraduate training, transition to senior trainee responsibilities, and transitions to practice [39, 40, 41, 42, 43, 103]	Mastery learning methods enhance learning [74, 75] Stages of training may allow for explicitly addressing transitions into and out of PGME [25, 103]
Reports of trainee disengagement [104]	Greater trainee engagement with training enhances learning outcomes [105]
Reports of assessment inequity [60, 61] Concerns that assessment of learning approaches overemphasize seeking trainees with problems instead of trainee development [77]	A developmental view of training allows for tailoring training and assessment to ensure every trainee progresses to competence (assessment for learning) [68]
Program reviews inordinately focused on process measures that may not enhance training [106]	New accreditation systems place greater emphasis on program outcomes and continuous improvement [106, 107] Application of learning analytics to medical education allows for new insights into program performance [108]

[i] ACGME Accreditation Council for Graduate Medical Education; CBME competency based medical education; CFPC College of Family Physicians of Canada; PGME postgraduate medical education.

Phases of Competence by Design development.

Table 2

Competence By Design Logic Model.

CORE COMPONENTS OF CBME*	ISSUES & OPPORTUNITIES	CBD DESIGN ELEMENTS	OUTPUTS	IMPACT
Outcomes as a Competency Framework for Graduates	PGME to ensure all graduates meet needed level of competence (focus on graduate outcomes for safe patient care) Program reviews focused on process, not outcomes	CanMEDS 2015 Framework New specialty-specific competencies New outcomes-oriented accreditation	Clear new competencies for every specialty New accreditation standards focused on outcomes of PGME	Competent graduates, ready for practice Enhanced training programs
Defined progression of training from novice to expert	Issues with transitions Time-based training produces variable graduates Patient safety concerns Incidents of inadequate supervision	Planned transitions 4 stages of PGME CanMEDS milestones	Better transitions to residency and practice Clear pathways to competence Better assessments for learning	Residents prepared for each stage of training Competent graduates, ready for practice Safer care
Tailored learning experiences	Generic training produces variable graduates Resident engagement with training enhances learning	Time variable training Flexible training requirements Promotions on achievements Individualized rotation plans Coaching over time	Residents with individualized pathways to certification	Residents prepared for each stage of training Competent graduates, ready for practice Greater resident satisfaction with training
Competency-based teaching	Little direct observation of trainees Inadequate feedback in workplace Growth mindset may enhance mastery of expertise EPAs provide opportunity for more faculty to provide better input Developmental view ensures no trainee left behind	Direct observation EPAs for workplace based assessment Coaching in the moment Developmental view of training Growth mindset	More direct observation More and better feedback Trainee portfolios provide rich picture of progress	Residents prepared for each stage of training Competent graduates, ready for practice Greater resident satisfaction with training
Programmatic assessment	Exam failures Promotions despite dyscompetence Few assessments Concerns about WBA Concerns about promotion decisions Opportunity to use learning analytics Opportunity to digitize assessment	Competence committee review of every trainee progress High number of EPA observations Learning analytics & eportfolios Developmental view of training Growth mindset Coaching over time New role for certification exam	Better promotion decisions Trainee portfolios provide rich picture of progress More faculty involved in WBA Clear pathways to competence Residents with individualized pathways to certification More and better feedback	Residents prepared for each stage of training Competent graduates, ready for practice Greater resident satisfaction with training Fewer appeals of assessments needed Same or higher exam pass rates

[i] *After Van Mell E, et al. International Competency-based Medical Education Collaborators. A core components framework for evaluating implementation of competency-based medical education programs. Acad Med. 2019; 94: 1002–9.

The Competence by Design Competence Continuum. Copyright 2012. The Royal College of Physicians and Surgeons of Canada. Reproduced with permission.

Table 3

Comparing CBD to Other CBME Implementations.

CORE COMPONENT	COMPETENCE BY DESIGN(ROYAL COLLEGE OF PHYSICIANS AND SURGEONS OF CANADA)	TRIPLE-C(COLLEGE OF FAMILY PHYSICIANS OF CANADA)	OUTCOMES PROJECT(ACCREDITATION COUNCIL FOR GRADUATE MEDICAL EDUCATION, USA)	AOA 21(AUSTRALIAN ORTHOPAEDIC ASSOCIATION)	INDIVIDUALIZING POSTGRADUATE MEDICAL TRAINING(DUTCH ASSOCIATION OF MEDICAL SPECIALISTS, NETHERLANDS)
Training outcomes organized as a competency framework for graduates	CanMEDS framework	CanMEDS-FM framework	ACGME 6 Competencies	AOA 21 Curriculum Framework	CanMEDS framework
Defined progression of training from novice to expert	Stages of training	Progression through training program	ACGME Milestones	Stages of training	Postgraduate years and EPAs
Tailored learning experiences to meet the needs of learners	Time-variable, flexible training	Tailoring within program	Tailoring within program	Time-variable, flexible training	Time-variable, flexible training
Teaching focused on competency achievement	EPA-driven, direct observation, and coaching in workplace. Growth mindset.	Teaching guided by Assessment Objectives for Certification in Family Medicine	Teaching guided by ACGME milestones	Teaching focused on stage-specific curriculum	Teaching focused on EPAs
Programmatic assessment	CBD program of assessment including Competence Committee review. Multiple eportfolios.	Triple-C program of assessment including Continuous Reflective Assessment for Training (CRAFT) reviewed by residency program committee. Multiple eportfolios.	Milestones-based program of assessment including Clinical Competency Committee review Multiple eportfolios.	AOA-21 program of assessment including Regional Training Committee review. National eportfolio.	EPA-based Program of assessment including Clinical Competency Committee review. Multiple eportfolios.

Competence By Design: a transformational national model of time-variable competency-based postgraduate medical education

Figures & Tables

Table 1

Figure 1

Table 2

Figure 2

Table 3

Paradigm

My account