Humanizing Data-Driven Methods in Engineering Education Research: A Systematic Literature Review of Four Journals From 2011 to 2021
By: Jiafu Niu and David Reeping
References
- Anderson, C. (2008). The end of theory: The data deluge makes the scientific method obsolete. Wired Magazine, 16(7), 16–07.
- André, Q. (2022). Outlier exclusion procedures must be blind to the researcher’s hypothesis. Journal of Experimental Psychology: General, 151(1), 213–223. 10.1037/xge0001069
- Ashabi, A., Sahibuddin, S. B., & Salkhordeh Haghighi, M. (2020, December). The systematic review of K-means clustering algorithm. 2020 The 9th International Conference on Networks, Communication and Computing (pp. 13–18). 10.1145/3447654.3447657
- Auxier, B., Rainie, L., Anderson, M., Perrin, A., Kumar, M., & Turner, E. (2019). Public knowledge and experiences with data-driven ads. Pew Research Center.
https://www.pewresearch.org/internet/2019/11/15/public-knowledge-and-experiences-with-data-driven-ads/ - Baker, R. S., & Yacef, K. (2009). The state of educational data mining in 2009: A review and future visions. Journal of Educational Data Mining, 1(1), 3–17.
- Block, J. (1971). Lives through time. Bancroft Books.
- Borrego, M., Foster, M. J., & Froyd, J. E. (2014). Systematic literature reviews in engineering education and other developing interdisciplinary fields: Systematic literature reviews in engineering education. Journal of Engineering Education, 103(1), 45–76. 10.1002/jee.20038
- Brown, K. (2016). After the “at-risk” label: Reorienting educational policy and practice. Teachers College Press.
- Cakir, V., & Gheorghe, A. (2017). Longitudinal academic performance analysis using a two-step clustering methodology. International Journal of Engineering Education, 33(1 A), 203–215.
- Castillo, W., & Gillborn, D. (2023).
How to “QuantCrit:” Practices and questions for education data researchers and users . (EdWorkingPaper: 22–546). Annenberg Institute at Brown University. 10.26300/v5kh-dd65 - Chan, C. K. Y., & Fong, E. T. Y. (2018). Disciplinary differences and implications for the development of generic skills: A study of engineering and business students’ perceptions of generic skills. European Journal of Engineering Education, 43(6), 927–949. 10.1080/03043797.2018.1462766
- Choe, N. H., & Borrego, M. (2020). Master’s and doctoral engineering students’ interest in industry, academia, and government careers. Journal of Engineering Education, 109(2), 325–346. 10.1002/jee.20317
- Creamer, E., & Edwards, C. (2019). Embedding the dialogic in mixed method approaches to theory development. International Journal of Research & Method in Education, 42(3), 239–251. 10.1080/1743727X.2019.1598357
- Cunningham-Nelson, S., Baktashmotlagh, M., & Boles, W. (2019). Visualizing student opinion through text analysis. IEEE Transactions on Education, 62(4), 305–311. 10.1109/TE.2019.2924385
- Cutumisu, M., & Guo, Q. (2019). Using topic modeling to extract pre-service teachers’ understandings of computational thinking from their coding reflections. IEEE Transactions on Education, 62(4), 325–332. 10.1109/TE.2019.2925253
- Douglas, K. A., & Purzer, Ş. (2015). Validity: Meaning and relevancy in assessment for engineering education research. Journal of Engineering Education, 104(2), 108–118. 10.1002/jee.20070
- Faber, C., & Benson, L. C. (2017). Engineering students’ epistemic cognition in the context of problem solving. Journal of Engineering Education, 106(4), 677–709. 10.1002/jee.20183
- Gallego, E., Diaz Barcos, V., Correa Hernando, E. C., Sanchez Espinosa, E., & Callejo Ramos, A. (2016). Influence of the perceived workload of students on the academic performance rates. International Journal of Engineering Education, 32(2), 670–681.
- Gillborn, D., Warmington, P., & Demack, S. (2018). QuantCrit: Education, policy, ‘Big Data’ and principles for a critical race theory of statistics. Race Ethnicity and Education, 21(2), 158–179. 10.1080/13613324.2017.1377417
- Godwin, A. (2017). Unpacking latent diversity. Paper presented at the ASEE Annual Conference and Exposition, Columbus, OH.
https://peer.asee.org/unpacking-latent-diversity . 10.18260/1-2--29062 - Godwin, A., Benedict, B., Rohde, J., Thielmeyer, A., Perkins, H., Major, J., Clements, H., & Chen, Z. (2021). New epistemological perspectives on quantitative methods: An example using topological data analysis. Studies in Engineering Education, 2(1),
16 . 10.21061/see.18 - Greene, J. C., Caracelli, V. J., & Graham, W. F. (1989). Toward a conceptual framework for mixed-method evaluation designs. Educational Evaluation and Policy Analysis, 11(3), 255–274. 10.3102/01623737011003255
- Haase, S. (2014). Engineering students’ sustainability approaches. European Journal of Engineering Education, 39(3), 247–271. 10.1080/03043797.2013.858103
- Hampton, C., & Reeping, D. (2019). Positionality: The Stories of Self that Impact Others. 2019 ASEE Annual Conference & Exposition Proceedings,
33177 . 10.18260/1-2--33177 - Hampton, C., Reeping, D., & Ozkan, D. S. (2021). Positionality statements in engineering education research: A look at the hand that guides the methodological tools. Studies in Engineering Education, 1(2), Article 2. 10.21061/see.13
- Hilbert, S., Coors, S., Kraus, E., Bischl, B., Lindl, A., Frei, M., … & Stachl, C. (2021). Machine learning for the educational sciences. Review of Education, 9(3),
e3310 . 10.1002/rev3.3310 - Hofmans, J., Wille, B., & Schreurs, B. (2020). Person-centered methods in vocational research. Journal of Vocational Behavior, 118(2020), 1–15. 10.1016/j.jvb.2020.103398
- Howard, M. C., & Hoffman, M. E. (2018). Variable-centered, person-centered, and person-specific approaches: Where theory meets the method. Organizational Research Methods, 21(4), 846–876. 10.1177/1094428117744021
- Inkelas, K. K., Maeng, J. L., Williams, A. L., & Jones, J. S. (2021). Another form of undermatching? A mixed-methods examination of first-year engineering students’ calculus placement. Journal of Engineering Education, 110(3), 594–615. 10.1002/jee.20406
- Irving, G., & Askell, A. (2019). AI safety needs social scientists. Distill, 4(2),
e14 . 10.23915/distill.00014 - Jaiswal, A., Lyon, J. A., Zhang, Y., & Magana, A. J. (2021). Supporting student reflective practices through modelling-based learning assignments. European Journal of Engineering Education, 46(6), 987–1006. 10.1080/03043797.2021.1952164
- Jo, E. S., & Gebru, T. (2020, January). Lessons from archives: Strategies for collecting sociocultural data in machine learning. Proceedings of the 2020 conference on fairness, accountability, and transparency (pp. 306–316). 10.1145/3351095.3372829
- Johri, A., Wang, G. A., Liu, X., & Madhavan, K. (2011, October). Utilizing topic modeling techniques to identify the emergence and growth of research topics in engineering education. In 2011 Frontiers in Education Conference (FIE) (pp. T2F-1).
IEEE . 10.1109/FIE.2011.6142770 - Kaleita, A. L., Forbes, G. R., Ralston, E., Compton, J. I., Wohlgemuth, D., & Raman, D. R. (2016). Pre-enrollment identification of at-risk students in a large engineering college. International Journal of Engineering Education, 32(4), 1647–1659.
- Kherif, F., & Latypova, A. (2020).
Principal component analysis . In Machine Learning (pp. 209–225). Elsevier. 10.1016/B978-0-12-815739-8.00012-2 - Kidder, L. H. (1981).
Qualitative research and quasi-experimental frameworks . In M. B. Brewer & B. E. Collins (Eds.), Scientific inquiry and the social sciences (pp. 226–256). Jossey-Bass. - Kilbertus, N., Ball, P. J., Kusner, M. J., Weller, A., & Silva, R. (2020, August).
The sensitivity of counterfactual fairness to unmeasured confounding . In Uncertainty in artificial intelligence (pp. 616–626). PMLR.https://arxiv.org/abs/1907.01040 - Ko, C.-Y., & Leu, F.-Y. (2021). Examining successful attributes for undergraduate students by applying machine learning techniques. IEEE Transactions on Education, 64(1), 50–57. 10.1109/TE.2020.3004596
- Korkmaz, C., & Correia, A. P. (2019). A review of research on machine learning in educational technology. Educational Media International, 56(3), 250–267. 10.1080/09523987.2019.1669875
- Kosinski, M., Stillwell, D., & Graepel, T. (2013). Private traits and attributes are predictable from digital records of human behavior. Proceedings of the national academy of sciences, 110(15), 5802–5805. 10.1073/pnas.1218772110
- Kotsiantis, S. B. (2012). Use of machine learning techniques for educational proposes: A decision support system for forecasting students’ grades. Artificial Intelligence Review, 37, 331–344. 10.1007/s10462-011-9234-x
- Kusner, M. J., & Loftus, J. R. (2020). The long road to fairer algorithms. Nature Publishing Group. 10.1038/d41586-020-00274-3
- Kusner, M. J., Loftus, J., Russell, C., & Silva, R. (2017). Counterfactual fairness. Advances in Neural Information Processing Systems, 30, 1–18.
https://arxiv.org/abs/1703.06856 - Lanza, S. T., Bray, B. C., & Collins, L. M. (2013). An introduction to latent class and latent transition analysis. Handbook of Psychology, 2, 691–716. 10.1002/9781118133880.hop202024
- Laursen, B. P., & Hoff, E. (2006). Person-centered and variable-centered approaches to longitudinal data. Merrill-Palmer Quarterly, 52(3), 377–389. 10.1353/mpq.2006.0029
- Lu, J., Liu, A., Dong, F., Gu, F., Gama, J., & Zhang, G. (2018). Learning under concept drift: A review. IEEE Transactions on Knowledge and Data Engineering, 31(12), 2346–2363. 10.1109/TKDE.2018.2876857
- Lund, B., & Ma, J. (2021). A review of cluster analysis techniques and their uses in library and information science research: K-means and k-medoids clustering. Performance Measurement and Metrics, 22(3), 161–173. 10.1108/PMM-05-2021-0026
- Marbouti, F., Ulas, J., & Wang, C.-H. (2021). Academic and demographic cluster analysis of engineering student success. IEEE Transactions on Education, 64(3), 261–266. 10.1109/TE.2020.3036824
- Martin, J. P., Brown, S., Miller, M. K., & Stefl, S. K. (2015). Characterizing engineering student social capital in relation to demographics. International Journal of Engineering Education, 31(4), 914–926.
- Martín, H., & Sorhaindo, C. (2019). A comparison of intrinsic and extrinsic motivational factors as predictors of civil engineering students’ academic success. International Journal of Engineering Education, 35(2), 458–472.
- McCutcheon, A. L. (1987). Latent class analysis (Vol. 64). Sage. 10.4135/9781412984713
- Morin, A. J. S., Gagne, M., & Bujacz, A. (2016). Feature topic: Person-centered methodologies in the organizational sciences. Organizational Research Methods, 19(1), 8–9.
http://journals.sagepub.com/doi/10.1177/1094428115617592 - Muthén, B., & Muthén, L. K. (2000). Integrating person-centered and variable-centered analyses: Growth mixture modeling with latent trajectory classes. Alcoholism: Clinical and Experimental Research, 24(6), 882–891. 10.1111/j.1530-0277.2000.tb02070.x
- Nelson, K. G., Shell, D. F., Husman, J., Fishman, E. J., & Soh, L.-K. (2015). Motivational and self-regulated learning profiles of students taking a foundational engineering course: Learning profiles of students in a foundational engineering course. Journal of Engineering Education, 104(1), 74–100. 10.1002/jee.20066
- Newman, I., & Benz, C. R. (1998). Qualitative-quantitative research methodology: Exploring the interactive continuum. SIU Press.
- Niu, J. (2023). Humanizing data-driven methods in engineering education research: A systematic literature review of four journals from 2011 to 2021 [Masters Thesis, University of Cincinnati]. ProQuest Dissertations & Theses Global.
http://rave.ohiolink.edu/etdc/view?acc_num=ucin1704205635164482 - Niu, J., & Reeping, D. (2023). Work in progress: A systematic literature review of person-centered approaches and data-driven methods in engineering education research. 2023 ASEE Annual Conference & Exposition. 10.18260/1-2--44164
- O’Neil, C. (2017). Weapons of math destruction: How big data increases inequality and threatens democracy. Crown.
- Osborne, J. W., & Overbay, A. (2004). The power of outliers (and why researchers should ALWAYS check for them). Practical Assessment, Research, and Evaluation, 9(1),
6 . 10.7275/QF69-7K43 - Pawley, A. L. (2017). Shifting the “default”: The case for making diversity the expected condition for engineering education and making whiteness and maleness visible. Journal of Engineering Education, 106(4), 531–533. 10.1002/jee.20181
- Pizard, S., & Vallespir, D. (2017). Towards a controlled vocabulary on software engineering education. European Journal of Engineering Education, 42(6), 927–943. 10.1080/03043797.2016.1235139
- Pluye, P., Grad, R. M., Levine, A., & Nicolau, B. (2009). Understanding divergence of quantitative and qualitative data (or results) in mixed methods studies. International Journal of Multiple Research Approaches, 3(1), 58–72. 10.5172/mra.455.3.1.58
- Qiu, L., Chan, S. H. M., & Chan, D. (2018). Big data in social and psychological science: Theoretical and methodological issues. Journal of Computational Social Science, 1(1), 59–66. 10.1007/s42001-017-0013-6
- Quan, W., Zhou, Q., Zhong, Y., & Wang, P. (2019). Predicting at-risk students using campus meal consumption records. International Journal of Engineering Education, 35(2), 563–571.
- Reeping, D., Lee, W., & London, J. (2023). Person-centered analyses in quantitative studies about broadening participation for Black engineering and computer science students. Journal of Engineering Education, 112(3), 769–795. 10.1002/jee.20530
- Reid, K., Imbrie, P. K., Lin, J. J., Reed, T., & Immekus, J. C. (2016). Psychometric properties and stability of the student attitudinal success instrument: the SASI-I. International Journal of Engineering Education, 32(6), 2470–2486.
- Ruipérez-Valiente, J. A., Muñoz-Merino, P. J., & Delgado Kloos, C. (2017). Detecting and clustering students by their gamification behavior with badges: A case study in engineering education. International Journal of Engineering Education, 33(2-B), 816–830.
- Saldaña, J. (2013). The coding manual for qualitative researchers (2nd ed). Sage.
- Scheidt, M., Godwin, A., Berger, E., Chen, J., Self, B. P., Widmann, J. M., & Gates, A. Q. (2021). Engineering students’ noncognitive and affective factors: Group differences from cluster analysis. Journal of Engineering Education, 110(2), 343–370. 10.1002/jee.20386
- Shafer, D., Mahmood, M. S., & Stelzer, T. (2021). Impact of broad categorization on statistical results: How underrepresented minority designation can mask the struggles of both Asian American and African American students. Physical Review Physics Education Research, 17(1), 1–13. 10.1103/PhysRevPhysEducRes.17.010113
- Shokri, R., Strobel, M., & Zick, Y. (2021). On the privacy risks of model explanations. In Proceedings of the 2021 AAAI/ACM Conference on AI, Ethics, and Society (AIES ‘21) (pp. 231–241).
Association for Computing Machinery . 10.1145/3461702.3462533 - Singer, G., Golan, M., Rabin, N., & Kleper, D. (2020). Evaluation of the effect of learning disabilities and accommodations on the prediction of the stability of academic behaviour of undergraduate engineering students using decision trees. European Journal of Engineering Education, 45(4), 614–630. 10.1080/03043797.2019.1677560
- Tan, L., Main, J. B., & Darolia, R. (2021). Using random forest analysis to identify student demographic and high school-level factors that predict college engineering major choice. Journal of Engineering Education, 110(3), 572–593. 10.1002/jee.20393
- Tan, P, Steinbach, M., & Kumar, V. (2005). Introduction to data mining. Pearson.
- Vermunt, J. K., & Magidson, J. (2004). Latent class analysis. The sage encyclopedia of social sciences research methods, 2, 549–553.
- Wach, K., Duong, C. D., Ejdys, J., Kazlauskaitė, R., Korzynski, P., Mazurek, G., … & Ziemba, E. (2023). The dark side of generative artificial intelligence: A critical analysis of controversies and risks of ChatGPT. Entrepreneurial Business and Economics Review, 11(2), 7–30. 10.15678/EBER.2023.110201
- Waelen, R. A. (2023). The struggle for recognition in the age of facial recognition technology. AI and Ethics, 3(1), 215–222. 10.1007/s43681-022-00146-8
- Weiss, H. M., & Rupp, D. E. (2011). Experiencing work: An essay on a person-centric work psychology. Industrial and Organizational Psychology, 4(1), 83–97. 10.1111/j.1754-9434.2010.01302.x
- Woo, S. E., Jebb, A. T., Tay, L., & Parrigon, S. (2018). Putting the “person” in the center: Review and synthesis of person-centered approaches and methods in organizational science. Organizational Research Methods, 21(4), 814–845. 10.1177/1094428117752467
- Xu, X., Wei, S., & Cao, Y. (2023). Moving beyond the “international” label: A call for the inclusion of the (in) visible international engineering students. Journal of Engineering Education, 112(2), 253–257. 10.1002/jee.20513
- Yellamraju, T., Magana, A. J., & Boutin, M. (2019). Investigating students’ habits of mind in a course on digital signal processing. IEEE Transactions on Education, 62(4), 312–324. 10.1109/TE.2019.2924610
- Zimmer, M. (2010).
“But the data is already public”: On the ethics of research in Facebook . In The ethics of information technologies (pp. 229–241). Routledge. 10.1007/s10676-010-9227-5 - Zuberi, T. (2008).
Deracializing social statistics: Problems in the quantification of race . In T. Zuberi & E. Bonilla-Silva (Eds.), White logic, White methods: Racism and methodology (pp. 127–134). Rowman & Littlefield.
DOI: https://doi.org/10.21061/see.159 | Journal eISSN: 2690-5450
Language: English
Page range: 150 - 174
Submitted on: Nov 16, 2023
Accepted on: Sep 12, 2024
Published on: Oct 25, 2024
Published by: Virginia Tech Publishing
In partnership with: Paradigm Publishing Services
© 2024 Jiafu Niu, David Reeping, published by Virginia Tech Publishing
This work is licensed under the Creative Commons Attribution 4.0 License.