Figure 1
FH2T start screen (left) and a sample problem in FH2T (right).
Figure 2
Study Design of the FH2T Efficacy Study.
Figure 3
FH2T Efficacy Study Procedure.
Table 1
Student Demographic Information by Condition (N = 4,092).
| FULL SAMPLE N = 4,092 (100%) | FH2T N = 1,649 (40.3%) | DRAGONBOX N = 854 (20.9%) | ASSISTMENTS-IMMEDIATE FEEDBACK N = 795 (19.4%) | ASSISTMENTS-ACTIVE CONTROL N = 794 (19.4%) | |
|---|---|---|---|---|---|
| Gender | |||||
| Male | 2,133 (52.1%) | 867 (52.6%) | 451 (52.8%) | 400 (50.3%) | 415 (52.3%) |
| Female | 1,954 (47.8%) | 782 (47.4%) | 401 (47.0%) | 394 (49.6%) | 377 (47.5%) |
| Not reported | 5 (0.1%) | – | 2 (0.2%) | 1 (0.1%) | 2 (0.2%) |
| Race/Ethnicity | |||||
| White | 2,088 (51.0%) | 821 (49.8%) | 431 (50.5%) | 429 (54.0%) | 407 (51.3%) |
| Asian | 1,024 (25.0%) | 410 (24.8%) | 209 (24.5%) | 197 (24.8) | 208 (26.2%) |
| Hispanic | 617 (15.1%) | 270 (16.4%) | 140 (16.4%) | 98 (12.3%) | 109 (13.7%) |
| Black | 200 (4.9%) | 83 (5.0%) | 38 (4.4%) | 44 (5.5%) | 35 (4.4%) |
| Native American | 26 (0.6%) | 11 (0.7%) | 5 (0.6%) | 3 (0.4%) | 7 (0.9%) |
| Pacific Islander | 2 (0.1%) | – | 1 (0.1%) | – | 1 (0.1%) |
| Multi-racial | 135 (3.3%) | 54 (3.3%) | 30 (3.5%) | 24 (3.0%) | 27 (3.4%) |
| Accelerated math class | 720 (17.6%) | 283 (17.2%) | 147 (17.2%) | 146 (18.4%) | 144 (18.1%) |
| Gifted | 627 (15.3%) | 262 (15.9%) | 119 (13.9%) | 122 (15.3%) | 124 (15.6%) |
| EIP | 291 (7.1%) | 114 (6.9%) | 58 (6.8%) | 59 (7.4%) | 60 (7.6%) |
| IEP | 463 (11.3%) | 188 (11.4%) | 132 (15.5%) | 76 (9.6%) | 67 (8.4%) |
| ESL | 372 (9.1%) | 164 (9.9%) | 77 (9.0%) | 57 (7.2%) | 74 (9.3%) |
| Virtual | 1,618 (39.5%) | 650 (39.4%) | 331 (38.8%) | 319 (40.1%) | 318 (40.1%) |
[i] Note: EIP = Early Intervention Program is designed for students who need extra support to meet academic grade level, IEP = Individualized Education Program is designed for students with disabilities or special health care needs, ESL = English as Second Language.
Figure 4
A sample problem and students’ actions in From Here to There!.
Figure 5
A sample problem in ASSISTments.
Figure 6
Sample problems in DragonBox (used with permission by Kahoot!).
Figure 7
Sample items in the first (a) and second (b) part of the Perceptual Math Equivalence Task.
Table 2
The list of the survey instruments and the location of the instrument files.
| MEASURES | NUMBER OF ITEMS | LINK TO THE FILES | LOCATION AND THE FILE NAME ON THE OSF PAGE |
|---|---|---|---|
| Algebraic Knowledge | 10 | https://osf.io/uenvg | 1. Problem list and assessment items/Assessment items.pdf |
| Perceptual Processing Skills | 16 | https://osf.io/r47ev | 1. Problem list and assessment items/Perceptual sensitivity task_items.pdf |
| Math anxiety and Math self-efficacy | 14 | https://osf.io/rq9d8 | 1. Problem list and assessment items/Math_Anxiety_Math Self-Efficacy_items.pdf |
Table 3
Descriptions of each table.
| CSV | CONTENTS | STRUCTURE |
|---|---|---|
| student | *Note: Fully de-identified. | |
| student_demo (student_demo.csv) | Data provided by the district on students’ demographics (race/ethnicity, gender, IEP, etc.) | one row per student |
| student_roster (student_roster.csv) | Data provided by the district on students’ school, teacher, class, impersonal or virtual status and movement between status during the year of the study. Data on students’ randomization and treatment status. | one row per student |
| student_attendance (student_attendance.csv) | Data provided by the district on students’ attendance for fifth, sixth, and seventh grade. | one row per student |
| student_fidelity (student_fidelity.csv) | Data on which of the 11 study assignments students started and completed. These data were captured by the ASSISTments system. | one row per student |
| assessment | ||
| assess_student (assess_student.csv) | Student performance data on pre, mid, post, and end-of-year assessments were captured in the ASSISTments system. Student state test scores provided by the district for fifth and seventh grade. (Note that the state test was not administered in 6th grade due to the COVID-19 pandemic.) | one row per student |
| assess_student_problem (assess_student_problem.csv) | Item level data on pre, mid, post, and end-of-year assessments captured in the ASSISTments system. | one row per student (wide) |
| From Here To There (FH2T) | ||
| fh2t_problems_meta (fh2t_problems_meta.csv) | Metadata on FH2T problems (problem equations and solutions, number of optimal steps, problem sequence, hints and instructional text, etc.) | one row per problem |
| fh2t_student_action_logs (fh2t_student_action_logs.csv) | Student action log data student (actions taken, response time, whether the action was valid mathematically or a mistake, the state of the equation before and after the action, etc.) | one row per student action (long) |
| fh2t_student_problem_attempt (fh2t_student_problem_attempt.csv) | Aggregated data on student problem attempts within FH2T (number of steps taken, number of hints requested, number of errors, etc.) | one row per student per problem attempt (long) |
| fh2t_student_problem (fh2t_student_problem.csv) | Aggregated data on student problems within FH2T including overall data on the problem (total time, number of attempts, number of replays, etc.) and data on students first, last and best attempt at the problem (number of steps taken, number of hints requested, number of errors, etc.) | one row per student per problem (long) |
| fh2t_student (fh2t_student.csv) | Aggregated data on student progress/performance within FH2T (Number of problems attempted, attempts per problem, total hints requested, etc.) | one row per student |
| ASSISTments | ||
| assist_problems_meta (assist_problems_meta.csv) | Metadata on ASSISTments problems (problem order, problem type, curriculum for which the problems were derived, etc.) | one row per problem part |
| assist_student_action_logs (assist_student_action_logs.csv) | Student action log data within ASSISTments (actions, response times, responses, etc.) | one row per student action (long) |
| assist_student_problem (assist_student_problem.csv) | Aggregated data on student problem attempts within ASSISTments (number of attempts, number of hints, accuracy, response times, etc.) | one row per student per problem (long) |
| assist_student (assist_student.csv) | Aggregated data on student progress and performance in ASSISTments (number of problems, total number of hints, accuracy, etc.) | one row per student |
| DragonBox | ||
| dragon_student (dragon_student.csv) | Aggregated data on student progress DragonBox (from devices) | one row per student |
| dragon_session (dragon_session.csv) | Self-report data on usage and progress in each session | one row per student (wide) |
Figure 8
Data structures of three educational technologies and examples of the metrics.