Table 1
Perspectives of the key elements that would comprise a faulty selection system as reported by TRIZ participants who acted as proxies on behalf of various interest-holder groups.
| KEY INTEREST-HOLDERS IN SELECTION PROCESSES | KEY ELEMENTS IN A WORST POSSIBLE SELECTION SYSTEM DERIVED FROM THE TRIZ EXERCISE | EXAMPLE CURRENTLY IN GME SYSTEMS REPRESENTED IN THE LITERATURE |
|---|---|---|
| Students/Applicants | Inequitable or biased selection process (e.g. systematic disadvantaging of a particular group). | Biases against international medical graduates and racial or gender bias in letters of recommendation [16, 17, 18, 19]. |
| Arbitrary selection process, subject to the whims of the selection program’s leadership who are fallible. | Selection perceived to be arbitrary due to poor data received from applicants and schools leaving program directors to rely on arbitrary scores and uneven group processes [20, 21]. | |
| Ability for a student to “game” a system via their assessments. | Students can ensure a higher probability of an “honors” grade by doing a clinical rotation at a specific program over another [22]. | |
| Using metrics where a portion of students cannot attain a good outcome (e.g., score curves and class ranks). | Norm referencing persists to stratify students for comparison in the GME selection process [2]. | |
| Use of artificial intelligence to create personal statements, answer essays, and flood the application with generalizable text data. | Students report using artificial intelligence to generate their personal statements and/or other written application content, while selection committees struggle to discern what is and is not AI-generated [23, 24]. | |
| Financial resources can provide unfair advantages (e.g., access to prep courses, ability to interview at more programs). | Students who identify as socioeconomically disadvantaged are less likely to be able to afford test preparation materials and score lower than their peers [25]. | |
| GME Training Program Directors | Selecting candidates based on their performance in a common curriculum or approach without accommodations or forgiveness/understanding of their educational progress or journey traveled. | Using score cut-offs to decrease the need to read all applications instead of holistic review for all applications [26]. |
| Recruiting and selecting students based on service needs within their training setting rather than with any foresight about job prospects within their specialty or community needs. | Selecting GME trainees to provide service but not ensuring they have a job upon graduation in their desired specialty or region. This can also manifest as junior doctors working in generalist specialties for extended periods while repeatedly applying to highly competitive specialties, such as ophthalmology [27, 28]. | |
| Resource-, time-, and bureaucracy-intensive process that does little to differentiate candidates (e.g., gathering much information with distracting information that does not predict clinical or other competence). | The current selection process in North America collects a large volume of data but does not meaningfully differentiate among students or reliably predict future success [4]. | |
| Misinformation from the training site about the academic achievements (or lack thereof) and/or progress of a student. | Data in the medical student performance evaluation (a formal document representing the entirety of medical school up to the time of residency application provided by the medical school in the United States) is heterogeneous and may obfuscate student performance [29]. | |
| Inadequate assessment of the true clinical and professional performance of students from their medical school performance evaluation. | Data in medical student performance evaluations are often incomplete, heterogeneous, and opaque, making it difficult for program directors to interpret them adequately [29, 30]. | |
| Create a “name-and-shame” culture by equating success in the ranking process with program and PD success. | Programs use post-interview communication as a pressure tactic to convince students to select them and determine where they might be on their rank list; however, this often backfires [31]. | |
| Foster a culture of produce-or-go-unselected within the applicant pool by persistently valuing numbers of publications or hours of participation/leadership. | Program directors and selection committees value the number of publications and extracurriculars over many other factors in selecting students to interview [7, 32, 33]. | |
| Selecting students who look, think, and act like them and who will ‘fit in’ with the existing group. | Many programs attempt to define and operationalize fit, but it often comes back to how a student “fits” with the current residents and/or faculty [34, 35]. | |
| Undergraduate Medical Education Leadership | Misrepresent performance of their students to “game” selection rates or pass through “problematic students.” | Many schools have been identified as indiscriminately inflating students’ achievements with adjectives such as “excellent” to help students be selected by programs [36]. |
| Poorly prepare students to be residents at all, distracting them with non-essential or misincentivized assessments. | The use of assessments for GME selection misincentivizes assessment, leading students to hide areas for growth rather than focus on preparedness for patient care [2]. | |
| Forbidding students from engaging in true apprenticeship and creating a complex observership process and procedural work that precludes the development of clinical acumen and “real-world know-how.” | International medical graduates often encounter significant barriers to observerships in their desired country and specialty, creating additional hurdles to selection [37]. | |
| Allowing the mythology of publication numbers to persist without any intervention to curb the extra- or co-curricular arms race for publications within the medical student population. Allowing continued abuse by faculty within certain fields to capitalize on the “publish or go unmatched” culture makes it challenging to advise students appropriately. | There is a significant shadow economy of effort that capitalizes on students’ publishing to obtain a residency position, particularly in more competitive specialties [7]. | |
| Create policies only based on standardization and conformity. | Professionalism is a competency domain used in selection (or often not to select someone) who may be difficult to train and is a construct that has created significant biases, particularly against students underrepresented in medicine [38]. | |
| Patients, Care Partners, Communities & Societal perspectives | Prioritizing the self-interests of the students or programs over those of patients or community needs, whether intentional or not. | Programs consistently (albeit likely unintentionally) prioritize their desired characteristics of student fit over the ability to perform the job they are hiring for or to meet the needs of the patients served by the program [6, 39]. |
| Selecting for a specific group results in a lack of diversity in the selected residents, so that there is little to no representation of the patients/communities they are serving. | Racial concordance between patients and physicians leads to improved communication outcomes, healthcare access by minoritized populations, and lower healthcare expenditures in minoritized populations; however, diversity is often not included or even overtly forbidden as a focus of the selection process [40, 41, 42, 43, 44]. | |
| The selection process does not take key patient care and safety measures into account, such as students seeking help when needed or following through on necessary tasks to ensure safe and effective care. | The core tenets of entrustment are important but not a focus of selection as competency measures, and patient needs are often deemphasized in selection systems [39, 45]. | |
| There is no consideration of geographic need in the selection process, resulting in physician shortages in rural or less desirable geographic areas. | Rural physician shortages are evident even during training across numerous countries, resulting in access-to-care issues for patients [46, 47, 48]. |