A Behaviourally Anchored Checklist for Mental Health Occupational Therapy Intake Interviews: Development and Reliability in a Single-Station Standardised Patient Encounter

Introduction: Intake interview performance is a core competency in mental health occupational therapy education, but structured and reliable assessment tools for brief observed encounters remain limited. This cross-sectional study therefore developed a 16-item behaviourally anchored checklist/rubric for intake interviews and examined scoring reliability in a single OSCE-format standardised-patient station.

Methods: Sixty third-year occupational therapy students from two Japanese universities completed one station using one of three standardised-patient scenarios. Two licensed occupational therapists independently rated performance across three domains (attitude, interview skills, evaluation). Internal consistency was assessed using Cronbach’s alpha, inter-rater agreement using intraclass correlation coefficients (ICCs; absolute-agreement definition), and Bland–Altman analysis was used to quantify rater agreement and assess fixed or proportional bias in total scores.

Results: Scenario-specific inter-rater agreement for the total score ranged from 0.761 to 0.929 across the three scenarios. Domain-level ICCs ranged from 0.584 to 0.892 for attitude, 0.510 to 0.825 for interview skills, and 0.794 to 0.979 for evaluation. Pooled descriptive summaries showed high internal consistency for all domains (Cronbach’s α = 0.850–0.887) and for total scores (α = 0.816–0.817). Bland–Altman analysis showed a small mean difference between raters (0.18) and 95% limits of agreement from –2.35 to 2.71.

Discussion: This behaviourally anchored checklist/rubric showed high internal consistency and acceptable inter-rater agreement for scoring intake interview performance in a single-station context. By operationalising interpersonal competencies as observable behaviours, the instrument may support rater calibration and structured formative feedback, including in settings where multi-station OSCE examinations are not feasible.