Humanizing Data-Driven Methods in Engineering Education Research: A Systematic Literature Review of Four Journals From 2011 to 2021

Background: Quantitative methods have been frequently used in engineering education research to investigate generalizable patterns and causal relationships in phenomena of interest to the field. With the proliferation of educational data and growing computational power, researchers are more empowered than ever to wrestle with enduring research questions using machine learning or Big Data methods. However, these techniques can create bias by depressing the voices of individuals belonging to smaller subgroups compared to the majority group in engineering.

Purpose: We examined how engineering education researchers make analytical decisions when they employ data-driven methods. More specifically, we explored how researchers adopted person-centered approaches to offset the inherent issues with data-driven methods.

Scope/Method: We conducted a systematic literature review of the data-driven methods used in quantitative and mixed-method studies published in leading engineering education journals from 2011 to 2021. We used the concepts of person- and variable-centered approaches as a guiding framework to categorize the researchers’ analytical decision-making and supplemented the review with a critical perspective using the lens of QuantCrit.

Discussion/Conclusions: Twenty-four articles qualified for the full-text review. Cluster analysis and decision tree models emerged as the sample’s two most popular data-driven methods. The findings demonstrated how engineering education researchers adopted person-centered approaches to humanize their chosen data-driven method. This involved finding latent diversity in the sample and understanding the groups formed using the constructs comprising that latent diversity. The coexistence of person- and variable-centeredness in a single study showcased the nuance of leveraging data-driven methods while ensuring the experiences of minoritized populations was not washed out in the noise.