Competency-Based Health Professions Education in Evolution: Challenges and Adaptations After Implementation

Challenge #1: Getting learners to the articulated outcomes

A nearly universal challenge has been the inability to have all learners demonstrate the desired level of competence across all the defined outcomes. An example of this is illustrated by the implementation of the Core EPAs for Entering Residency [17] as a component of teaching and assessing competence in undergraduate medical education (UME) at the University of Minnesota in the US. The goal was for at least 90% of students to achieve the “indirect supervision” level of entrustment by the end of their core clerkships. During the first 18 months of implementation, the goal was achieved for only 7 of the 13 defined EPAs. To address this gap, three adaptations were used: 1) altering the EPA description; 2) altering the curriculum and/or opportunities for assessment; and 3) elimination of the EPA as a requirement. An example of an EPA that required altering the description was “Identify system failures and contribute to a culture of safety and improvement.” By limiting the focus of teaching and assessing this EPA to specific functions – identifying system errors, activating the system through event reports, and participating in system-wide safety initiatives – gains were made towards the learner entrustment goal. An EPA that required adapting the curriculum and/or opportunities for assessment was “Give or receive a patient handover to transition care responsibility”. The assessment team recognized that most observations/assessments occurred in the required Intensive Care Unit (ICU) experience of the fourth year and subsequently designated this EPA as the major focus during that rotation. Finally, “Obtain informed consent” was the only EPA for which there was no alteration of the EPA itself or enhancement of the curriculum that allowed learners to reach the desired level of entrustment. Here, relevant procedures were identified as being under the purview of other professionals; thus, the EPA could not be considered a potential graduation requirement.

Challenge #2: Generic competency outcomes that require local adaptations

A second challenge has arisen when nationally or internationally developed competency frameworks do not fully align with local health systems, professional identities, or clinical practice. Programs may initially implement the framework as written, only to discover that some competencies do not “fit” their context, cannot be assessed meaningfully, or fail to capture what is valued in local practice. As a result, adaptation of the framework is needed to support effective and sustainable implementation. Before examining specific examples, it is critical to recognize a priori that the adaptation of competency frameworks from other countries or of a national framework within a country to a specialty ideally occurs at the macro-system level; that is, at the national or specialty level. At this level, the adaptations can still be studied across programs (e.g., undergraduate or graduate programs), allowing benchmarking and comparisons and ensuring all learners across programs at a similar level achieve the desired outcomes (i.e., “comparing apples to apples”). If the adaptations occur at a meso-system level (such as a school or consortium of graduate programs) or at the micro-system level (such as an individual program), these comparisons become impossible (i.e., “comparing apples to oranges”). A clear example of this second challenge comes from the South African Health Professions Council (HPCSA) national adaptation of the Canadian framework, CanMEDS [10], to “AfriMEDS” [18, 19]. Early implementation revealed that the central role of Medical Expert did not reflect the interprofessional, team-based, and community-oriented nature of South African healthcare. In response, the HPCSA replaced it with the broader role of Health Practitioner, explicitly anchoring the framework in local patterns of work and expectations of professional identity [18]. In anesthesiology specifically, a Delphi process identified two domains (Humaneness and Context Awareness) that clinicians considered essential but were missing from the imported framework [20]. Similar analyses in medicine and dentistry also led to reinterpretation to better align with South Africa’s health priorities and patient population [21, 22].

A parallel pattern emerged in Taiwan following the national adoption of the Accreditation Council for Graduate Medical Education (ACGME) Core Competencies. Early implementation showed that several competencies and milestones were difficult to operationalize or assess within the local training environment. In response, specialty committees engaged in a structured process of “glocalization,” revising and reframing competencies to reflect Taiwanese clinical realities [6]. Specific work in family medicine in Taiwan confirmed that some ACGME sub-competencies lacked contextual “fitness,” prompting modification [23].

In India, national mandates for postgraduate CBME articulated extensive competencies but provided limited guidance on how to operationalize them into assessable workplace activities [24]. In one discipline, faculty from multiple institutions across different regions collaboratively developed, refined, and validated a set of national discipline-specific EPAs [25]. This bottom-up adaptation addressed the absence of an actionable national-level assessment and curriculum framework during initial implementation.

Across these settings, the trajectory was similar: a global framework was adopted, early implementation revealed misalignment and challenges, and the framework itself was adapted to fit local needs. These cases highlight that even the most established competency frameworks cannot be assumed to transfer seamlessly across settings. Meaningful implementation often requires reshaping the framework such that it reflects local values, systems, and practices.

Challenge #1: Lack of progression markers

In the US in 1999, the ACGME and the American Board of Medical Specialties (ABMS) endorsed six core competency domains, each with delineated competencies within them, as the foundation for physician training and practice [11, 26, 27]. This “Outcome Project” mandated that all US ACGME-accredited programs use the core competencies as the framework for assessment, curriculum, and certification recommendations. Despite its intent, implementation proved challenging due to a lack of shared understanding by trainees, faculty, and program directors as to how the competencies applied across all specialties or what the expected trajectory of competency acquisition (sequenced progression) in GME training looked like. To address this challenge, the ACGME launched the Milestone Project [28], which created developmental narrative Milestones in each specialty for all of the competencies across the six core competency domains. For the first time, GME program directors had a road map for sequencing the developmental progression of competence in their specialty or sub-specialty. As they were implemented, however, evaluation revealed that because the milestones were specialty-specific, cross-specialty comparisons were difficult. The ACGME is now in its second iteration of the Milestones with standardization across four of the core competencies (Interpersonal and Communication Skills, Professionalism, Systems-based Practice, and Practice-based Learning and Improvement), reserving specialty variation for the Patient Care and Medical Knowledge competencies [29].

Challenge #2: Learning experiences and developmental outcome expectations are not aligned

Following implementation, many CBHPE programs have identified misalignment between their learning experiences and the intended developmental progression of competence. Several broad concepts underpin developmental progression of competence, including: a) sequencing should progress from care of common and simple problems to more complex ones; b) physician development progresses from dependence on external influences to a more intrinsically driven and regulated process; and c) the development of some competencies involves progressing from knowledge to competence (the ability to demonstrate a skill in context) to capability (the ability to adapt to new contexts) [30].

Tailored learning experiences that conflict with these principles of developmental progression can interfere with learning. An illustration occurred in implementing the competencies for Pediatrics in the US. At the time of implementation, programs were required to have both newborn care and neonatal intensive care (NICU) experiences within the residency program. In many programs, however, residents completed NICU experiences well before their newborn nursery experiences, violating the concept of simple-to-complex above. The resultant simple adaptation was the revision of rotation schedules to ensure newborn nursery care always preceded the NICU experience, thus tailoring the learning experiences to better support the sequenced progression of competence.

Challenge #1: Inability to provide/access tailored learning experiences

During CBHPE implementation, many programs have faced the challenge of either lacking adequate experiences to enable learners to attain the desired level of competence or having the experiences in place without learners being aware of them or able to access them. An example of both occurred during the implementation of the Education in Pediatrics Across the Continuum (EPAC) four-site pilot in the United States [31, 32]. EPAC was designed to test the feasibility of competency-based time-variable transitions from medical school to residency and from residency to practice or fellowship. For the former transition, students were required to achieve entrustment on the 13 Core EPAs for Entering Residency [17]. For the EPA, “Perform the general procedures of a physician”, students across the pilot schools frequently failed to attain the desired level of entrustment by the end of their core clerkships. Root cause analysis revealed that programs lacked a standardized curriculum for the procedures outlined and could not reliably ensure opportunities for performing them in the clinical setting. The primary adaptations were to expand the list of procedures included in the EPA and to ensure adequate learning and assessment opportunities in the required clerkship rotations.

Challenge #2: Providing tailored learning experiences for progressive autonomy

There is broad evidence in the medicine and nursing education literatures that some program graduates are unprepared for the transition to fully unsupervised practice, while others achieve competency early and would benefit from increased autonomy [33, 34, 35, 36, 37, 38]. Fixed-time programs and legal considerations, such as supervision-level requirements for safety and billing purposes, pose strong barriers to graduated autonomy over time. The primary adaptation for this challenge is the engineering of a structured transition that incorporates a period of learner-specific graduated autonomy, usually with an emphasis on preceptorship/mentoring, coaching, and reflection. These transition programs might be incorporated into the pre-graduation program or provided as an onboarding experience after the transition to unsupervised practice.

One example of a transition program within residency (prior to the transition to unsupervised practice) is “Promotion in Place” (PIP), a specific model of competency-based time-variable GME training designed to address this challenge by providing qualifying participants a unique period of “sheltered independence” as fully graduated, credentialed, and certifying Board-eligible physicians until what would have been their regularly scheduled graduation date [39, 40]. This model represents an adaptation to time-variable training, allowing practice in a “sheltered” environment with trusted mentors, an experience valued across participant groups [41]. The TIMELESS model from internal medicine at the University of Cincinnati takes a similar approach, allowing qualified residents to progress to distant oversight supervision while not making them the physician of record for billing and legal purposes. This solution addresses the ingrained bureaucracy of the US healthcare system, where multiple regulatory bodies have regulations that curtail what is possible [42]. Both PIP and TIMELESS provide tailored learning experiences with graded autonomy that allow residents the opportunity for time-variable, competency-based advancement [43].

The nursing literature, while not solely focused on competency-based education, is replete with examples of intentionally designed extended (6 month–1 year) programs as part of the on-boarding in a first unsupervised practice experience [44, 45]. This literature suggests that key ingredients to successful transition programs include trained preceptors in graduated autonomy, a formal skills curriculum (both clinical and professional), often including simulation, and guided reflection/mentorship. Outcomes include higher competency ratings across domains, increased confidence of the nurses, decreased stress, and improved retention.

Challenge #1: Faculty coaching or feedback not related to the competencies

Transitioning to a CBHPE framework often increases the need for specific competency-focused coaching and/or feedback. This may take time to evolve [46], and faculty can find the evolution to programmatic, competency-focused, workplace-based assessment systems challenging [47]. Sources of confusion for frontline faculty include the introduction of novel feedback forms and rating scales, and expectations for written comments [48] and newly designed, often online, systems [49]. Additional challenges arise from the multiple roles that individuals hold within the CBHPE system (e.g., teacher, assessor, and patient safety monitor). Tensions among these roles can complicate the provision of high-level competency-focused feedback to trainees.

Learner-oriented adaptations have focused on helping trainees actively seek targeted observation and feedback. One such adaptation at the University of Saskatchewan in Canada involved the creation of custom laminated cards that learners could attach to their lanyards during off-service rotations (e.g., rotations where they were not in their primary specialty) to ensure residents knew what EPAs were best taught and assessed in each of their rotations outside of their primary specialty [50], to ensure the residents would receive feedback and coaching designed specifically for the competencies assessed for that rotation. Faculty-oriented adaptations have included several interventions to strengthen coaching and feedback skills, a critical post-implementation need in many CBHPE programs [51, 52, 53, 54]. Other interventions include data systems that provide faculty with useful insights via dashboards [55] or scoring rubrics that assess the quality of faculty comments [56]. Some educators have created automated tools that score comments. These tools harness machine learning and natural language processing to train artificial intelligence (AI) to help faculty understand the gaps in their feedback or coaching language [57].

Challenge #2: Lack of focus on development: missing the assessment forest for the trees

Competency acquisition requires learners to view their progression in a developmental context over time. Programs implementing CBHPE may provide teaching and assessment that focus on the acquisition of competence within a typical course or clinical rotation but may find it more challenging to provide competency-focused instruction longitudinally across courses and clinical rotations. Without such integration, learners may receive fragmented feedback that is difficult to synthesize into actionable insights for progression. In response to this challenge, programs have used two primary adaptations: a) implementing a technological platform that organizes longitudinal data in accessible, interpretable formats to optimize learners’ insights and/or b) developing coaching programs that encourage reflection on longitudinal progress over time.

One multi-institutional adaptation that addresses the technological platform to allow longitudinal synthesis of assessment data comes from the Society for Improving Medical Professional Learning (SIMPL) [58, 59, 60]. SIMPL is used by over 400 training programs in 28 specialties across 6 countries and provides a technological platform with operating room-based and bedside-based modules for assessment and analytics. The platform facilitates assessments in EPAs and Observable Professional Activities (OPAs) [61], providing programs, faculty, learners, and certification boards with actionable longitudinal data.

At the program level, an example of the use of coaching and feedback to ensure longitudinal synthesis of assessment data evolved at the University of Minnesota Medical School during the implementation of an EPA framework for clinical instruction and assessment [62, 63]. To stimulate longitudinal reflection and coaching, each student was assigned to an Assessment and Coaching Expert (ACE). These were faculty focused on the EPA teaching and assessment program. Coaches and students together use longitudinal EPA assessment data to develop competency-focused instruction plans for future rotations, improving the students’ use of their data for growth.

Challenge #1: Perceived Burden of Assessment

Perhaps the most common challenge is the perceived burden of assessment. For example, through the implementation of Competence by Design (CBD) in Canadian specialist postgraduate medical training, faculty and program leaders reported a high burden of assessment [66, 67, 68, 69]. Program evaluation revealed several contributing factors [67], including: expectations of high volumes of assessment; heavy reliance on workplace based assessments (WBA) over other assessment strategies; misalignment between required contexts for assessments and availability of clinical opportunities in these contexts; challenging technology for assessments; variable institutional and leadership support; and an overarching lack of perceived value of the assessment process by supervisors and learners. Further, program and discipline-specific analyses highlighted that the burden of assessment was highly influenced by the pre-implementation assessment structure. Programs with prior regular daily assessment practices faced much less difficulty than those where assessments were previously infrequent. Key adaptations employed by local programs [70] and published in CBD 2.0 [71] included reframing EPA observation numbers as flexible targets instead of absolute requirements and explicitly positioning them as one piece of a larger program of assessment. Adaptations highlighted the shared responsibility between faculty and learners in the facilitation and timely completion of EPA observations. Individual programs identified local and specialty-specific innovations that would support the opportunities for feedback, coaching, and completion of WBAs. These adaptations were designed to reduce the assessment burden, enhance perceived value, and support sustainable CBHPE programs.

Challenge #2: Feasibility Challenges in Collecting and/or Using Assessment Data

Feasibility challenges in collecting and using assessment data provide a pressure point in the evolution of programmatic assessment. As programs increase the volume and diversity of workplace-based assessments and place greater emphasis on longitudinal synthesis, traditional processes prove inadequate. Digital systems intended to support CBHPE may be inadequate as they may negatively influence both the capture and organization of data and the evolution of assessment practices [72].

National experiences in Finland and Taiwan illustrate how different governance structures have generated similar adaptations in response to these feasibility pressures. Finland implemented programmatic assessment through a top-down mandate requiring all universities and specialties to adopt competency-based frameworks. In this case, a national electronic assessment system was created that could support structured WBA in a country where such practices had not previously been established [73]. Taiwan, however, followed a bottom-up pathway, beginning with small pilot programs and local innovations, ultimately resulting in the emergence of a prominent model [74] which was adopted by the accreditation body to guide the development of a digital system nationally [75].

Despite these adaptations, both contexts encountered a common tension once national systems were introduced. Platforms designed to address feasibility challenges inevitably introduce uniformity that may conflict with established specialty-specific assessment structures. In both Finland and Taiwan, specialties with longstanding educational models sometimes found national templates restrictive and developed blended approaches to maintain coherence with local practice while meeting national requirements. Interestingly, in this case, established assessment structures resulted in increased challenges to adoption. However, in the burden of assessment challenge outlined earlier, established programs of assessment often made the assessment burden easier to navigate.

Challenge #3: Data Inadequacy for Progression Decisions

A central premise of programmatic assessment is the use of aggregated data from several sources to provide a full picture of a learner’s competence and developmental trajectory [76]. These data are often designed for use by competency committees to render defensible decisions around learner readiness for progression. However, those who have implemented CBHPE often note challenges in obtaining the data needed to make progression decisions. Challenges range from poor quality or disconfirming data [77] to challenges with data accessibility, visualization, or synthesis [78, 79].

One such example can be seen in the Core EPAs for Entering Residency pilot in the US [80]. Four of the ten pilot schools convened a Trained Entrustment Group (TEG) to collate and review all available assessment data and render theoretical summative entrustment decisions for EPAs implemented at their school. They then evaluated the extent to which each school was able to compile sufficient data to make summative entrustment decisions for a given class, examined differences between EPAs on learners’ readiness for distant supervision (the desired level for readiness for residency), and looked at the volume of WBAs needed to make a summative entrustment decision [81]. Each TEG used a different combination and amount of data, including WBAs, end-of-clerkship assessments, objective structured clinical examinations (OSCEs), simulation, and narrative data. TEGs were much more likely to render a decision when WBA data were available. In contrast, the absence of WBA data far more frequently led to the inability to render a decision. Decision confidence was highest for EPAs assessed more frequently and longitudinally across the curriculum. Wide variability existed between and within pilot schools in the number of WBAs needed to make a decision for a given student [81]. These findings prompted adaptations to programmatic assessment. The first adaptation was an increase in the volume of assessment data. A second adaptation was to ensure a variety of data streams to supplement the WBAs, particularly in areas where the overall number of WBAs was lower or longitudinal data was not available [81]. Schools also expanded targeted faculty development to optimize narrative feedback within their programmatic assessment offerings, enhancing the quality of data reviewed by the TEGs in rendering their summative entrustment decisions [82].

Challenge #4: Performance-focused Assessment Systems

Operationalizing the dual-purpose of assessment for learning and assessment of learning in CBHPE has been challenging across many settings [65, 83]. CBHPE tends to focus on summative assessment rather than building formative assessment programs to drive coaching and learning. The unintended consequences of overly performance-focused assessment systems can be the degradation of feedback [84], assessment being viewed as ‘tick-boxing’ with maladaptive assessment-seeking behavior by trainees [85], and a loss of authenticity in assessment moments [86].

During the initial implementation of postgraduate medicine specialist training in CBD in Canada, evaluation findings suggested that, in several programs, performance-focused assessment was causing unintended consequences such as those above, as well as a negative impact on trainee wellness [66]. In CBD 2.0, several adaptations were put in place to shift the focus from performance- to growth-focused assessment, including: expanding programs of assessment to include an array of tools beyond workplace-based EPA assessments; prioritizing narrative assessments over numerical ratings; allowing space and opportunity for “ultra-low-stakes” [87] assessment with the purposes of feedback and coaching, and optimizing Competence Committee processes to encourage growth-focused coaching for all rather than binary performance-focused progression decisions alone [71].

Challenge #5: Lack of High-Utility Workplace-based Assessment Tools

WBA is a foundational component of programmatic assessment [88], yet many health professions education programs lack useful WBAs that collect meaningful data to inform robust decision-making. Local assessment tool development in isolation is resource-intensive and may not be driven by best practices in educational design. Further, this can lead to inconsistency across programs and variability in outcomes, constraining discipline-level program evaluation. As an example, in the initial CBHPE implementation, the US pediatric graduate medical education community committed to CBHPE for training and certification, but an authentic national competency-based program of assessment was not employed, resulting in variability in outcomes at the conclusion of training [35, 89, 90].

To address this issue, the American Board of Surgery began national implementation of a smartphone-based approach to EPA-based WBA in 2023. The surgery community has now accumulated over 500,000 standardized WBAs supporting formative learner development while also addressing curricular gaps locally and nationally, providing data towards national program evaluation, and ultimately improving assurance to the public [91]. Subsequently, the American Board of Pediatrics made an adaptation by partnering to develop a smartphone-based WBA tool based on an EPA framework [92]. This tool was designed to improve WBAs by indexing frontline formative WBAs on the outcomes of interest (the EPAs), creating a direct connection between the core outcomes and programmatic assessment. Large-scale pilot implementation is ongoing, with early unpublished data demonstrating feasibility, improved learner feedback, enhanced development of learning plans, and more meaningful competency committee discussions and decision-making. Similar adaptations are underway in several other US medical specialties to standardize and improve the approach to WBA as an aspect of programmatic assessment in their specialties [93, 94, 95, 96].