Table 1
Participant Flow and Demographics of Participating Students.
| ATTRIBUTE | 2017 | 2018 | ||
|---|---|---|---|---|
| CONTROL | SI | CONTROL | SI | |
| Enrolled Students | 42 | 38 | 38 | 41 |
| Did not consent | 3 | 2 | 1 | 5 |
| Dropped course | 6 | 2 | 1 | 0 |
| Transferred sections | 0 | 0 | 1 | 0 |
| Participating Students | 33 | 34 | 35 | 36 |
| Gender | ||||
| % Women | 83.3 | 73.7 | 88.6 | 72.2 |
| IPEDS Race and Ethnicity | ||||
| % American Indian or Alaska Native | 0 | 0 | 0 | 0 |
| % Asian | 12.1 | 17.6 | 17.1 | 5.6 |
| % Black or African American | 6.1 | 2.9 | 8.6 | 8.3 |
| % Hispanic | 15.2 | 8.8 | 5.7 | 11.1 |
| % Two or more races | 9.1 | 2.9 | 0 | 2.8 |
| % Native Hawaiian or Other Pacific Islander | 0 | 0 | 0 | 0 |
| % Nonresident | 0 | 2.9 | 2.8 | 2.8 |
| % White | 54.5 | 47.1 | 62.9 | 61.1 |
| % Unknown race | 3.1 | 17.7 | 2.8 | 2.8 |
| Years Enrolled | ||||
| % First year of college | 75.8 | 79.4 | 82.9 | 72.2 |
Table 2
Descriptive Statistics of Experimental Variables in Each Year of the Experiment.
| ATTRIBUTE | 2017 | 2018 | ||
|---|---|---|---|---|
| CONTROL n = 33 | SI n = 34 | CONTROL n = 35 | SI n = 36 | |
| URM Status | ||||
| URM | 9 | 5 | 7 | 8 |
| Non-URM | 24 | 29 | 28 | 28 |
| Motivation Score | ||||
| N | 31 | 29 | 30 | 32 |
| Mean | 15.0 | 14.9 | 15.9 | 15.4 |
| SD | 2.00 | 2.12 | 1.78 | 2.39 |
| Neighborhood Income | ||||
| N | 33 | 33 | 35 | 36 |
| Mean | $118,234 | $164,418 | $164,145 | $160,503 |
| SD | $74,263 | $125,541 | $215,720 | $154,113 |
| Math Placement | ||||
| Pre-calculus | 0 | 1 | 7 | 2 |
| Calculus 1 | 17 | 14 | 16 | 17 |
| Calculus 2 or 3 | 15 | 18 | 9 | 16 |
| No information | 1 | 1 | 3 | 1 |
| Previous Science Courses | ||||
| 0 | 26 | 28 | 32 | 29 |
| 1 | 4 | 2 | 2 | 4 |
| 2 | 2 | 4 | 1 | 2 |
| 3 | 1 | 0 | 0 | 1 |
| Years since entering college | ||||
| 0 | 25 | 27 | 29 | 26 |
| 1 | 7 | 5 | 4 | 9 |
| 2 | 1 | 0 | 0 | 1 |
| 3 | 0 | 1 | 0 | 0 |
| No data | 0 | 1 | 2 | 0 |
| Attendance | ||||
| % ≥ 1 session | 51.5% | 82.5% | 28.6% | 58.3% |
| % ≥ 3 sessions | 21.1% | 32.4% | 14.3% | 38.9% |
| Exam 1 score (±SD) | 78.1 (± 14.6) | 78.5 (± 12.2) | 77.5 (± 14.5) | 78.7 (± 13.0) |
| URM | 72.8 (±20.4) | 72.7 (±19.4) | 63.1(±9.6) | 74.1 (± 12.5) |
| Non-URM | 80.1 (±11.7) | 79.5 (±10.7) | 81.1 (±13.3) | 80.0 (±13.1) |
| Final Exam Score (±SD) | 71.1 (± 9.90) | 75.5 (± 10.8) | 79.7 (± 13.6) | 78.3 (± 11.5) |
| URM | 73.22 (±11.6) | 70.9 (±13.4) | 73.0 (±8.2) | 80.4 (±8.1) |
| Non-URM | 79.9 (±8.7) | 76.3 (±10.3) | 81.4 (±14.2) | 77.6 (±12.3) |
Table 3
ANOVA Table for Final Exam Grade in 2017.
| MODEL TERM | df | ESTIMATE | STANDARD ERROR | F | P |
|---|---|---|---|---|---|
| Exam1 | 1 | 0.426 | 0.076 | 31.2564 | <0.0001† |
| Total Visits | 1 | 0.949 | 0.375 | 5.2959 | 0.0248† |
| URM | 1 | 1.028 | 1.284 | 0.6410 | 0.4265 |
| Treatment | 1 | 3.343 | 1.276 | 2.9827 | 0.0892 |
| Visits * Treatment | 1 | –0.750 | 0.368 | 4.1534 | 0.0459† |
| Terms dropped from the final model | |||||
| Ex1 * Visits | 1 | 0.24285 | 0.6241 | ||
| Ex1 * URM | 1 | 0.00024 | 0.9893 | ||
| Ex1 * Trt | 1 | 0.35636 | 0.5529 | ||
| Visits * URM | 1 | 1.00129 | 0.3140 | ||
| URM * Trt | 1 | 0.56178 | 0.4566 | ||
| Ex1 * TV * URM | 1 | 0.18102 | 0.6722 | ||
| Ex1 * TV * Trt | 1 | 0.00421 | 0.9488 | ||
| Ex1 * URM * Trt | 1 | 0.12803 | 0.7219 | ||
| Visits * URM * Trt | 1 | 0.61707 | 0.4353 | ||
[i] 1–9 Order of terms dropped from initial model, †p < 0.05.
Note. n = 67 students.
Figure 1
The Effect of Session Attendance on Final Exam Scores in 2017.
Note. TT = traditional tutoring, SI = Supplemental Instruction. Points represent individual student scores. Lines represent predicted final exams scores from the final statistical model at three different initial exam one scores (50%, 70%, and 90%). Degree of shading indicates first exam score.
Table 4
ANOVA Table for Final Exam Grade in 2018.
| MODEL TERM | df | ESTIMATE | STANDARD ERROR | F | P |
|---|---|---|---|---|---|
| Exam1 | 1 | 0.346 | 0.1735 | 3.978† | 0.0507 |
| Total Visits | 1 | 3.037 | 1.472 | 4.254† | 0.0436† |
| URM | 1 | 69.988 | 21.216 | 10.882 | 0.0016† |
| Treatment | 1 | –10.376 | 20.512 | 0.256† | 0.6148 |
| Ex1 * URM | 1 | –0.832 | 0.272 | 9.360† | 0.0033† |
| Ex1 * Trt | 1 | 0.131 | 0.248 | 0.280† | 0.5982 |
| TV * URM | 1 | –10.695 | 2.216 | 23.302† | <0.0001† |
| TV * Trt | 1 | –2.412 | 1.716 | 1.976† | 0.1651 |
| URM * Trt | 1 | –72.525 | 27.097 | 7.164† | 0.0096† |
| Ex1 * URM * Trt | 1 | 0.942 | 0.346 | 7.408† | 0.0085† |
| Visits * URM * Trt | 1 | 10.505 | 2.463 | 18.184† | <0.0001† |
| Terms dropped from the final model | χ2 | ||||
| Ex1 * Trt * URM | 1 | 1.9701 | 0.1605 | ||
| Ex1 * TV* Section | 1 | 2.2562 | 0.1331 | ||
| Ex1 * TV | 1 | 0.5233 | 0.4696 | ||
[i] 1–3Order of terms dropped from initial model; †p < 0.05.
Note. Outliers included. n = 71 students.
Figure 2
The Effect of Session Attendance on Final Exam Scores in 2018 for Each Combination of Treatment and URM Status.
Note. TT = traditional tutoring, SI = Supplemental Instruction. Points represent individual student scores. Lines represent predicted final exams scores from the final statistical model at three different initial exam one scores (50%, 70%, and 90%). The solid and dashed lines are predicted from the models with and without the three outlier points (circled in black), respectively. Shading indicates first exam score.
Table 5
Predicted Slopes (With 95% Confidence Intervals) for the Relationship Between Exam 1 Score or Number of Tutoring Sessions Attended and Final Exam Score for the 2018 Experiment.
| SECTION | URM STATUS | EXAM 1 | SESSIONS ATTENDED |
|---|---|---|---|
| Model with outliers included | |||
| Control | Non-URM | 0.346 (–0.001,0.693)a | 3.04 (0.09,5.98)a* |
| SI | Non-URM | 0.477 (0.123,0.830)a* | 0.62 (–1.14, 2.39)a |
| Control | URM | –0.486 (–0.906, –0.067)b* | –7.66 (–10.97, –4.34)b* |
| SI | URM | 0.587 (0.345,0.829)a* | 0.43 (–0.80,1.67)a |
| Model with 3 outliers excluded | |||
| Control | Non-URM | 0.288 (0.091,0.484)a,b* | 1.94 (0.68,3.19)a* |
| SI | Non-URM | 0.292 (0.117,0.467)a,b* | 1.18 (0.59, 1.77)a* |
| Control | URM | –0.587 (–1.410, 0.237)b | –7.97 (–14.66,–1.28)b* |
| SI | URM | 0.663 (0.383,0.943)a* | 0.49 (–1.12,2.10)a,b |
[i] a, bGroups that share letters within a model and variable are not statistically different from each other, based on a Tukey multiple comparison test (p > 0.05).
*Slope is significantly different from zero, based on a t-test (p < 0.05).
Table 6
ANOVA Table for the Number of Sessions Attended.
| MODEL TERM | df | ESTIMATE | STANDARD ERROR | F | P |
|---|---|---|---|---|---|
| URM status | 1 | 0.193 | 0.111 | 3.0027 | 0.0859 |
| Motivation Score | 1 | –0.018 | 0.153 | 1.4330 | 0.2338 |
| Previous Science Courses | 1 | 0.081 | 0.051 | 2.4911 | 0.1174 |
| Treatment | 1 | –0.204 | 0.063 | 10.5364 | 0.0155† |
| Year | 1 | 0.231 | 0.072 | 10.3126 | 0.0017† |
| URM * Year | 1 | –0.343 | 0.155 | 4.9104 | 0.0288† |
| Terms dropped from the final model | |||||
| Math Placement | 1 | 0.0951 | 0.7584 | ||
| URM * Trt | 1 | 0.781511 | 0.3787 | ||
| Mot*Trt | 1 | 0.13928 | 0.7098 | ||
| Math * Trt | 1 | 0.329010 | 0.5674 | ||
| Sci * Trt | 1 | 0.07756 | 0.7813 | ||
| Year * Trt | 1 | 1.454512 | 0.2304 | ||
| Mot* Year | 1 | 0.01105 | 0.9167 | ||
| Math * Year | 1 | 0.69849 | 0.4052 | ||
| Sci * Year | 1 | 0.09907 | 0.7536 | ||
| URM * Year * Trt | 1 | 1.22354 | 0.2713 | ||
| Mo * Year * Trt | 1 | 0.53461 | 0.4664 | ||
| Math * Year * Trt | 1 | 0.73902 | 0.3921 | ||
| Sci * Year * Trt | 1 | 1.22043 | 0.2720 | ||
[i] 1–12Order of terms dropped from initial model; †p < 0.05.
Note. n = 117 students.
Figure 3
Number of Academic Support Sessions Attended by Year, Treatment, and URM Status.
Note. Marginal means (±95CI) from the statistical model are presented in the outlined bars. Points represent individual students. Points are jittered slightly along the x-axis for clarity.