Table 1
Supplemental Instruction Interview Rubric (SIIR).
| DIMENSION | INSUFFICIENT (1) | DEVELOPING (2) | PROFICIENT (3) | DISTINGUISHED (4) |
|---|---|---|---|---|
| Familiarity with SI(knows basic elements) | Misinformed about or unaware of SI elements | Mentions elements with some possible misconceptions | Describes or uses elements mostly accurately | Describes or uses elements accurately |
| Familiarity with Academic Support | Misinformed about or may not acknowledge other AS programs or mission | Mentions other AS programs or mission | Describes other programs or AS mission mostly accurately | Describes other programs or AS mission accurately |
| Communication(verbal) | May be difficult to understand, unclear response, speaks off topic | Speaks nonspecifically, may demonstrate minimal processing of ideas | Speaks to specific actions or ideas, positive tone, understandable in the group | Volume, enunciation, and demonstration of ideas are clear, concise, and effective |
| Collaboration | Dominates or unwilling to contribute to group work | Willing to contribute, but may repeat or restate others | Effectively adds value to conversation or provides new ideas | Synthesizes discussion, adds new ideas or evaluates ideas, may lead group effectively, may use SI facilitation skills |
| Decision-making | Rigid or binary thinking | Considers new ideas, but does not apply | Demonstrates flexibility in thinking, teachable | Demonstrates flexibility and application in thinking |
| Communication(nonverbal) | Misaligned body language and speech, negative affect | Generally engaged, may not be actively participating, instances of negative affect | Actively listening to facilitators, positive affect, is approachable and friendly, focused | Actively listening to facilitators AND group, positive affect with aligned body language and speech, engaged |
| Mindset of an SI Leader(understands why SI works) | Uses undesired terminology such as “teaching,” “lecturing,” “kids” (as an SI Leader), inaccurately describes elements or responsibilities | References experience with SI, describes fundamental aspects of SI accurately | Mentions self-directed learning, student leading, active learning, learning styles, group work, skill integration(may not use the exact word) | Describes self-directed learning, learning styles, active learning, group work, student leading, skill integration |
Table 2
Inter-Rater Reliability Statistics Across Rubric Dimensions.
| DIMENSION | AGREEMENT COEFFICIENT | |
|---|---|---|
| PERCENT AGREEMENT | GWET’S AC | |
| SI familiarity | 0.88 | 0.72 |
| Academic Support familiarity | 0.87 | 0.76 |
| Verbal communication | 0.89 | 0.75 |
| Collaboration | 0.92 | 0.83 |
| Decision-making | 0.89 | 0.77 |
| Nonverbal communication | 0.89 | 0.74 |
| Mindset of an SI Leader | 0.88 | 0.71 |
| Total score | 0.97 | 0.86 |
Figure 1
Various Agreement Coefficients Across Each Dimension.
Note: Kappa coefficients may be interpreted as 0.61–0.80 = substantial, 0.81–1.00 = almost perfect.