Table 1
In-and exclusion criteria
| Category | Criteria |
|---|---|
| Population | Studies with physiotherapists or physiotherapy students were included. |
| Studies with health professionals or health professions students were included when they practiced procedures that can be used in physiotherapy (i.e., when medical students were evaluated on their ability to perform a musculoskeletal examination) | |
| Studies with health professionals or health professions students were excluded when they practiced procedures that cannot be practiced by physiotherapists (such as surgery) | |
| Educational assessments | The assessment could be either a procedure specific measurement instrument (i.e., the assessment is designed exclusively for one procedure) or a procedure unspecific measurement (i.e., the assessment is designed to measure procedures in physiotherapy education but can be used for more than one procedure) |
| The assessment should measure procedures in reality. Assessments based on virtual reality were excluded. | |
| The assessment should be feasible in various settings. Therefore, assessments that require expensive equipment were excluded. | |
| Data must be available for a specific assessment. Studies with summary data of several assessments were excluded (e.g., summary scores of a complete OSCE). | |
| Outcome | The aim of assessment should be to measure procedural skills. Assessments of similar constructs such as clinical skills or psychomotor skills [defined as ‘... motor skill, some manipulation of material, or some act which requires a neuromuscular action’ Simpson (1966, p. 17)] were included. |
| Assessments that aimed to exclusively evaluate other outcomes such as communication skills or professionalism were excluded. | |
| When assessments were designed to measure multiple outcomes, it was evaluated whether the focus was based on procedural skills (e.g., more than 50% of the items concentrate on procedural skills). The assessments with focus on procedural skills were included. | |
| Measurement properties | Studies had to report the measurement properties of an educational assessment (e.g., reliability or validity) |
Table 2
Search strategy
| Population | Assessment | Outcome |
|---|---|---|
| medical education OR education, medical[Mesh] OR physiotherapy education OR physical therapy education OR health professions education OR healthcare education OR allied health care education | scale OR global rating scale OR GRS OR checklist | practical skill* OR psychomotor skill* OR procedural skill* OR clinical skill* |
Figure 1
Study flow.
Table 3
Characteristics of included studies and assessments.
| Study | Country | Setting | Sample | Assessed procedure | Scale and items | Duration | Patients | Assessors | Purpose |
|---|---|---|---|---|---|---|---|---|---|
| Beran et al. (2012) AMPE | USA | Orthopaedic department | 24 orthopaedic residents | PSMI: Musculoskeletal physical examination; Inspection, palpation, joint range of motion, strength testing and any special tests pertinent to the clinical scenario | Four 12-15 items checklists for clinical scenarios (upper extremity, lower extremity, trauma and spine) on dichotomous scales (yes or no). | 10 Minutes | Standardised patients are required (120 minutes training) | Pool of experienced raters | High stakes purpose |
| Boulet et al.(2004) OMT | USA | Osteopathic college | 121 osteopathic students (4th year) | PUMI: Osteopathic manipulative treatment of three clinical cases (low back pain, frozen shoulder and asthmatic with cough) | OMT (Osteopathic Manipulative Treatment) assessment tool with 15 items; Every item is scored on a 0 to 2 scale (0 = done incorrectly or not done, 1 = not performed optimally and 2 = done proficiently) | 13 minutes | Standardised patients with 8 hours of formal training | 16 osteopathic physicians (5 hours of formal training) | High stakes examination (OSCE) |
| Herbers et al. (2003) PES-C | USA | University Medical Centre | 72 internal medicine residents | PSMI: Pelvic examination | 29 item dichotomous checklist (yes = when the behaviour was observed; no = when the behaviour was not observed); Includes some items about communication skills | Not specified | Gynaecologic teaching trainer required; 1 trainer was being examined and the second trainer rated the student's skills. | Gynaecologic teaching trainer required | Not specified |
| Herbers et al. (2003) PES-R | USA | University Medical Centre | 72 internal medicine residents | PUMI: Pelvic examination | Global rating scale evaluating the overall performance of the pelvic examination (five-point ordinal scale between 1 = inadequate and 5 = excellent) | Not specified | Gynaecologic teaching trainer required; 1 trainer was being examined and the second trainer rated the student's skills | Gynaecologic teaching trainer required | Not specified |
| Ladyshewsky et al. (2000) PhyES | Australia | Physiotherapy department | 12 undergraduate physiotherapy students 4 physiotherapists (at least 2 years of experience) | PSMI: Musculoskeletal physical examination of a patient with a rotator cuff problem | Physical examination checklist (3-point scale: 0 = not done, 1 = done poorly or incompletely and 2 = done well), number of items not available | Mean 30 minutes (range: 20 - 46 minutes) | Standardised patients are required | Assessors with 30 hours of training | High stakes examination (OSCE) |
| Nothnagle et al. (2010) GPSE | USA | Family medicine department | 5 faculty members and 5 students (semi structured interviews); Focus groups: 7 experienced family medicine educators, 5 residents and 5 faculty members | PUMI: Eligible for all procedures in family medicine | Global Procedural Skills Evaluation Form, 4-point scale, amount of assistance is documented ranging from significant guidance is provided to performed independently; communication skills etc. are included; Student's self-assessment is included; Difficulty of the procedures is rated as well | Not specified | Not required | Not specified | Low stakes examination (formative feedback) |
| Swift et al. (2013)* mO-S3 | USA | Physiotherapy department | 12 undergraduate 1st year physiotherapy students | PSMI: Examination skills in musculoskeletal physiotherapy (shoulder tests) | Checklist for a musculoskeletal OSCE station; 6 items checklist (5 dichotomous items and 1 ordinal item) | 6 minutes | Simulation patients with 2 hours of supervised training and 1 week of independent training | Clinical instructors (2 - 20 years of experience) | Low stakes examination (mid-term) |
| Yudkowsky et al. (2004) HTTPE | USA | University Medical Centre | 369 medical students (2nd year) | PSMI: Head to toe physical examination | 138 item checklist; three-point scale (0 = incorrect, 1 = correct after prompt, 2 = correct without prompting); Test duration: 2 h; high stakes summative assessment or low stakes formative assessment | 2 hours (45 minutes unprompted exam, remaining 1:45 hours are used for scoring, feedback, and teaching) | Trained patient instructors with 25 hours of training | Trained patient instructors with 25 hours of training | High stakes summative assessment and low stakes formative assessment |
AMPE: Assessment of Musculoskeletal Physical Examination Skills Checklist; GPSE: Global Procedural Skills Evaluation Form; HTTPE: Head to Toe Physical Examination; mO-S3: mOSCE-Station 3 checklists; PhyES: Physical Examination Skills Checklist; PES-C: Pelvic Examination Skills Checklist; PES-R: Pelvic Examination Skill Rating Scale; PSMI: Procedure Specific Measurement Instrument; PUMI: Procedure Unspecific Measurement Instrument
Figure 2
Internal consistency estimates.
Nb. The statistical method from Boulet et al. (2004) was not available.
Figure 3
Interrater reliability estimates.
