Potential Societal Biases of ChatGPT in Higher Education: A Scoping Review
References
- Abd-Alrazaq, A., AlSaad, R., Alhuwail, D., Ahmed, A., Healy, P. M., Latifi, S., Aziz, S., Damseh, R., Alabed Alrazak, S., & Sheikh, J. (2023). Large Language Models in Medical Education: Opportunities, Challenges, and Future Directions. JMIR medical education, 9,
e48291 . 10.2196/48291 - Alzubaidi, L., Zhang, J., Humaidi, A. J., Al-Dujaili, A., Duan, Y., Al-Shamma, O., … & Farhan, L. (2021). Review of deep learning: Concepts, CNN architectures, challenges, applications, future directions. Journal of Big Data, 8, 1–74. 10.1186/s40537-021-00444-8
- Arksey, H., & O’Malley, L. (2005). Scoping studies: Towards a methodological framework. International Journal of Social Research Methodology, 8(1), 19–32. 10.1080/1364557032000119616
- Bannister, P., Peñalver, E. A., & Urbieta, A. S. (2023). Transnational higher education cultures and generative AI: A nominal group study for policy development in English medium instruction. Journal for Multicultural Education, 18(1/2), 173–191. 10.1108/JME-10-2023-0102
- Belgodere, B., Dognin, P., Ivankay, A., Melnyk, I., Mroueh, Y., Mojsilovic, A., … & Young, R. A. (2023). Auditing and generating synthetic data with controllable trust trade-offs. arxiv Preprint arXiv:2304.10819. 10.1109/JETCAS.2024.3477976
- Benjamin, R. (2019). Race after technology: Abolitionist tools for the new Jim Code. Cambridge and Medford: Polity Press.
- Bordia, S., & Bowman, S. R. (2019). Identifying and reducing gender bias in word-level language models. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Student Research Workshop (pp. 7–15). Minneapolis, Minnesota:
Association for Computational Linguistics . 10.48550/arXiv.1904.03035 - Bozkurt, A. (2024). GenAI et al.: Cocreation, authorship, ownership, academic ethics and integrity in a time of generative AI. Open Praxis, 16(1), 1–10. 10.55982/openpraxis.16.1.654
- Brown, T., Mann, B., Ryder, N., Subbiah, M., Kaplan, J. D., Dhariwal, P., … & Amodei, D. (2020). Language models are few-shot learners. Advances in Neural Information Processing Systems, 33, 1877–1901. Retrieved on 12th November 2024 from
https://proceedings.neurips.cc/paper/2020/hash/1457c0d6bfcb4967418bfb8ac142f64a-Abstract.html - Caines, A., Benedetto, L., Taslimipoor, S., Davis, C., Gao, Y., Andersen, O., … & Buttery, P. (2023). On the application of Large Language Models for language teaching and assessment technology. arXiv preprint arXiv:2307.08393. 10.48550/arXiv.2307.08393
- Chowdhery, A., Narang, S., Devlin, J., Bosma, M., Mishra, G., Roberts, A., … & Fiedel, N. (2022). Palm: Scaling language modeling with pathways. arXiv preprint arXiv:2204.02311. Retrieved on 12th November 2024 from
https://www.jmlr.org/papers/v24/22-1144.html - Chung, H. W., Hou, L., Longpre, S., Zoph, B., Tay, Y., Fedus, W., … & Wei, J. (2022). Scaling instruction-finetuned language models. arXiv preprint arXiv:2210.11416. Retrieved on 12th November 2024 from
https://www.jmlr.org/papers/v25/23-0870.html - Corsello, A., & Santangelo, A. (2023). May Artificial Intelligence Influence Future Pediatric Research?—The Case of ChatGPT. Children, 10(4),
757 . 10.3390/children10040757 - Crawford, K. (2017). The trouble with bias. Keynote at NeurIPS. Retrived from:
https://www.youtube.com/watch?v=ggzWIipKraM - Crenshaw, K. (1989). Demarginalizing the intersection of race and sex: A Black feminist critique of antidiscrimination doctrine, feminist theory and antiracist politics. University of Chicago Legal Forum, 1989(1), Article
8 .http://chicagounbound.uchicago.edu/uclf/vol1989/iss1/8 - Cui, Y. H., Bai, F., & Zhang, R. X. (2023). ChatGPT zai gao deng jiao yu ling yu de ying yong feng xian ji ying dui [Applications, risks, and countermeasures of ChatGPT in higher education]. Journal of Chongqing University of Technology (Social Science), 5, 16–25.
- Curtis, N. (2023). To ChatGPT or not to ChatGPT? The impact of Artificial Intelligence on academic publishing. The Pediatric Infectious Disease Journal, 42(4),
275 . 10.1097/INF.0000000000003852 - Derner, E., Kučera, D., Oliver, N., & Zahálka, J. (2023). Can ChatGPT read who you are?. arXiv preprint arXiv:2312.16070. 10.1016/j.chbah.2024.100088
- Dodge, J., Sap, M., Marasović, A., Agnew, W., Ilharco, G., Groeneveld, D., … & Gardner, M. (2021). Documenting large webtext corpora: A case study on the colossal clean crawled corpus. arXiv preprint arXiv:2104.08758. 10.48550/arXiv.2104.08758
- Dwivedi, Y. K., Kshetri, N., Hughes, L., Slade, E. L., Jeyaraj, A., Kar, A. K., … & Wright, R. (2023). “So what if ChatGPT wrote it?” Multidisciplinary perspectives on opportunities, challenges and implications of generative conversational AI for research, practice and policy. International Journal of Information Management, 71,
102642 . 10.1016/j.ijinfomgt.2023.102642 - Equality Challenge Unit. (2013). Unconscious bias and higher education. Retrieved from
https://diversity.caltech.edu/documents/19785/unconscious-bias-and-higher-education-compressed.pdf - Farrokhnia, M., Banihashem, S. K., Noroozi, O., & Wals, A. (2023). A SWOT analysis of ChatGPT: Implications for educational practice and research. Innovations in Education and Teaching International, 1–15. 10.1080/14703297.2023.2195846
- Fecher, B., Hebing, M., Laufer, M., Pohle, J., & Sofsky, F. (2023). Friend or foe? Exploring the implications of Large Language Models on the science system. arXiv preprint arXiv:2306.09928. 10.48550/arXiv.2306.09928
- Fischer, I. (2023). Evaluating the ethics of machines assessing humans. Journal of Information Technology Teaching Cases. 10.1177/20438869231178844
- Garvey, J. C., Mobley, S. D., Summerville, K. S., & Moore, G. T. (2018). Queer and trans* students of color: Navigating identity disclosure and college contexts. The Journal of Higher Education, 90(1), 150–178. 10.1080/00221546.2018.1449081
- Goyal, A., Siddique, M., Parekh, N., Schwitzky, Z., Broekaert, C., Michelotti, C., … & Kumar, N. (2023). ChatGPT and bard responses to polarizing questions. arXiv preprint arXiv:2307.12402. 10.48550/arXiv.2307.12402
- Grassini, S. (2023). Shaping the future of education: Exploring the potential and consequences of AI and ChatGPT in educational settings. Education Sciences, 13(7),
692 . 10.3390/educsci13070692 - Harrad, R., Keasley, R., & Jefferies, L. (2024). Academic integrity or academic misconduct? Conceptual difficulties in higher education and the potential contribution of student demographic factors. Higher Education Research & Development, 1–15. 10.1080/07294360.2024.2339833
- Heine, S. J., & Ruby, M. B. (2010). Cultural psychology. Wiley Interdisciplinary Reviews: Cognitive Science, 1(2), 254–266. 10.1002/wcs.7
- Hidayat-ur-Rehman, I., & Ibrahim, Y. (2023). Exploring factors influencing educators’ adoption of ChatGPT: A mixed method approach. Interactive Technology and Smart Education, (ahead-of-print). 10.1108/ITSE-07-2023-0127
- Hofstede, G. (2001). Culture’s consequences: Comparing values, behaviors, institutions, and organizations across nations (2nd ed.). Sage.
- Huang, L., Ye, Z., Qin, J., Lin, L., & Liang, X. (2020). GRADE: Automatic graph-enhanced coherence metric for evaluating open-domain dialogue systems. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP) (pp. 9230–9240). Online:
Association for Computational Linguistics . 10.18653/v1/2020.emnlp-main.742 - Hutchinson, B., Smart, A., Hanna, A., Denton, E., Greer, C., Kjartansson, O., … & Mitchell, M. (2021). Towards accountability for machine learning datasets: Practices from software engineering and infrastructure. In Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency, 560–575. 10.1145/3442188.3445918
- Jo, E. S., & Gebru, T. (2020). Lessons from archives: Strategies for collecting sociocultural data in machine learning. Proceedings of the 2020 conference on fairness, accountability, and transparency. 306–316. 10.1145/3351095.3372829
- Kalhor, G., Zeraati, T., & Bahrak, B. (2023). Diversity dilemmas: Uncovering gender and nationality biases in graduate admissions across top North American computer science programs. EPJ Data Sci., 12,
44 . 10.1140/epjds/s13688-023-00422-5 - Karabacak, M., Ozkara, B. B., Margetis, K., Wintermark, M., & Bisdas, S. (2023). The advent of generative language models in medical education. JMIR medical education, 9,
e48163 . 10.2196/48163 - Kasneci, E., Seßler, K., Küchemann, S., Bannert, M., Dementieva, D., Fischer, F., … Kasneci, G. (2023). ChatGPT for good? On opportunities and challenges of Large Language Models for education. 10.35542/osf.io/5er8f
- Katirai, A., Ide, K., & Kishimoto, A. (2023a). Seisei AI Generative AI No Rinriteki Houteki Shakai-teki Kadai ELSI Ronten No Gaikan 2023-Nen 3-Gatsu Ban [Ethical, Legal, and Social Issues (ELSI) of Generative AI and AI in general: Overview of the discussion points as of March 2023]. ELSI NOTE, 26, 1–37. 10.18910/90926
- Katirai, A., Yamamoto, B. A., Kogetsu, A., & Kato, K. (2023b). Perspectives on artificial intelligence in healthcare from a Patient and Public Involvement Panel in Japan: an exploratory study. Frontiers in Digital Health, 5,
1229308 . 10.3389/fdgth.2023.1229308 - Kooli, C. (2023). Chatbots in education and research: A critical examination of ethical implications and solutions. Sustainability, 15(7),
5614 . 10.3390/su15075614 - Latif, E., Mai, G., Nyaaba, M., Wu, X., Liu, N., Lu, G., … & Zhai, X. (2023). Artificial general intelligence (AGI) for education. arXiv preprint arXiv:2304.12479. 10.48550/arXiv.2304.12479
- Leung, T. I., Sagar, A., Shroff, S., & Henry, T. L. (2023). Can AI mitigate bias in writing letters of recommendation? JMIR Medical Education, 9,
e51494 . 10.2196/51494 - Liang, W., Yuksekgonul, M., Mao, Y., Wu, E., & Zou, J. (2023). GPT detectors are biased against non-native English writers. arXiv preprint arXiv:2304.02819. 10.1016/j.patter.2023.100779
- Li, M., Enkhtur, A., Cheng, F., & Yamamoto, B. A. (2024). Ethical implications of ChatGPT in higher education: A scoping review. Journal of Interdisciplinary Studies in Education, 13(1), 55–68. 10.32674/jise.v13i1.6072
- Lin, C. C., Akuhata-Huntington, Z., & Hsu, C. W. (2023). Comparing ChatGPT’s ability to rate the degree of stereotypes and the consistency of stereotype attribution with those of medical students in New Zealand in developing a similarity rating test: a methodological study. Journal of Educational Evaluation for Health Professions, 20(17). 10.3352/jeehp.2023.20.17
- Łodzikowski, K., Foltz, P. W., & Behrens, J. T. (2023). Generative AI and its educational implications. arXiv preprint arXiv:2401.08659. 10.48550/arXiv.2401.08659
- Masters, K. (2023). Ethical use of artificial intelligence in health professions education: AMEE Guide No. 158. Medical Teacher, 45(6), 574–584. 10.1080/0142159X.2023.2186203
- McCowan, T. (2016). Three dimensions of equity of access to higher education, Compare: A Journal of Comparative and International Education, 46(4), 645–665. 10.1080/03057925.2015.1043237
- Mohamed, A. M. (2023). Exploring the potential of an AI-based Chatbot (ChatGPT) in enhancing English as a Foreign Language (EFL) teaching: perceptions of EFL Faculty Members. Education and Information Technologies, 29, 3195–3217. 10.1007/s10639-023-11917-z
- Morley, L., & Lugg, R. (2009). Mapping meritocracy: Intersecting gender, poverty and higher educational opportunity structures. Higher Education Policy, 22, 37–60. 10.1057/hep.2008.26
- Nakane, C. (1970). Japanese society. London: Weidenfeld & Nicolson.
- Najafali, D., Hinson, C., Camacho, J. M., Galbraith, L. G., Tople, T. L., Eble, D., … & Morrison, S. D. (2023). Artificial intelligence knowledge of evidence-based recommendations in gender affirmation surgery and gender identity: is ChatGPT aware of WPATH recommendations?. European Journal of Plastic Surgery, 46(6), 1169–1176. 10.1007/s00238-023-02125-6
- Nyaaba, M., Wright, A., & Choi, G. L. (2024). Generative AI and digital neocolonialism in global education: Towards an equitable framework. arXiv preprint arXiv:2406.02966.
- Ojha, S., Narendra, A., Mohapatra, S., & Misra, I. (2023). From robots to books: An introduction to smart applications of AI in education (AIEd). arXiv preprint arXiv:2301.10026. 10.36227/techrxiv.21836610.v1
- O’Neil, C. (2016). Weapons of math destruction: How big data increases inequality and threatens democracy. Bus Econ, 52, 123–125. 10.1057/s11369-017-0027-3
- OpenAI Report. (2019). Release strategies and the social impacts of Language Models. Retrieved from
https://cdn.openai.com/GPT_2_August_Report.pdf - Organisation for Economic Co-operation and Development. (2023). Country digital education ecosystems and governance: A companion to digital education outlook 2023. OECD Publishing. 10.1787/906134d4-en
- Ouyang, L., Wu, J., Jiang, X., Almeida, D., Wainwright, C., Mishkin, P., … & Lowe, R. (2022). Training language models to follow instructions with human feedback. Advances in Neural Information Processing Systems, 35, 27730–27744.
https://proceedings.neurips.cc/paper_files/paper/2022/hash/b1efde53be364a73914f58805a001731-Abstract-Conference.html - Qadir, J. (2023). Engineering education in the era of ChatGPT: Promise and pitfalls of generative AI for education. In 2023 IEEE Global Engineering Education Conference (EDUCON) (pp. 1–9).
IEEE . 10.1109/EDUCON54358.2023.10125121 - Rasul, T., Nair, S., Kalendra, D., Robin, M., de Oliveira Santini, F., Ladeira, W. J., … & Heathcote, L. (2023). The role of ChatGPT in higher education: Benefits, challenges, and future research directions. Journal of Applied Learning and Teaching, 6(1). 10.37074/jalt.2023.6.1.29
- Régis, C., Denis, J. L., Axente, M. L., & Kishimoto, A. (2024). Human-centered AI: A multidisciplinary perspective for policy-makers, auditors, and users. Taylor & Francis. 10.1201/9781003320791
- Roberts, H., Cowls, J., Morley, J., Taddeo, M., Wang, V., & Floridi, L. (2021). The Chinese approach to artificial intelligence: an analysis of policy, ethics, and regulation (pp. 47–79). Springer International Publishing. 10.1007/978-3-030-81907-1_5
- Sharma, P., Thapa, K., Dhakal, P., Upadhaya, M. D., Adhikari, S., & Khanal, S. R. (2023). Performance of ChatGPT on USMLE: Unlocking the potential of large language models for AI-assisted medical education. arXiv preprint arXiv:2307.00112. 10.48550/arXiv.2307.00112
- Sheng, E., Chang, K. W., Natarajan, P., & Peng, N. (2021). Societal biases in language generation: Progress and challenges. arXiv preprint arXiv:2105.04054. 10.48550/arXiv.2105.04054
- Smith, A., Hachen, S., Schleifer, R., Bhugra, D., Buadze, A., & Liebrenz, M. (2023). Old dog, new tricks? Exploring the potential functionalities of ChatGPT in supporting educational methods in social psychiatry. International Journal of Social Psychiatry, 69(8), 1882–1889. 10.1177/00207640231178451
- Solaiman, I., Brundage, M., Clark, J., Askell, A., Herbert-Voss, A., Wu, J., … & Wang, J. (2019). Release strategies and the social impacts of language models. arXiv preprint arXiv:1908.09203. 10.48550/arXiv.1908.09203
- Tong, W., Guan, Y., Chen, J., Huang, X., Zhong, Y., Zhang, C., & Zhang, H. (2023). Artificial intelligence in global health equity: An evaluation and discussion on the application of ChatGPT, in the Chinese National Medical Licensing Examination. Frontiers in Medicine, 10. 10.3389/fmed.2023.1237432
- Touvron, H., Lavril, T., Izacard, G., Martinet, X., Lachaux, M. A., Lacroix, T., … & Lample, G. (2023). Llama: Open and efficient foundation language models. arXiv preprint arXiv:2302.13971. 10.48550/arXiv.2302.13971
- Tricco, A. C., Lillie, E., Zarin, W., O’brien, K., Colquhoun, H., Kastner, M., … & Straus, S. E. (2016). A scoping review on the conduct and reporting of scoping reviews. BMC medical research methodology, 16, 1–10. 10.1186/s12874-016-0116-4
- Tülübaş, T., Demirkol, M., Ozdemir, T. Y., Polat, H., Karakose, T., & Yirci, R. (2023). An interview with ChatGPT on emergency remote teaching: A comparative analysis based on human-AI collaboration. Educational Process: International Journal, 12(2), 93–110. 10.22521/edupij.2023.122.6
- UNESCO. (2022). The right to higher education: a social justice perspective. Retrieved from
https://unesdoc.unesco.org/ark:/48223/pf0000381750 - UNESCO IESALC. (2023). ChatGPT and artificial intelligence in higher education: Quick start guide. Retrieved from
https://www.iesalc.unesco.org/en/2023/04/14/chatgpt-and-artificial-intelligence-in-higher-education-quick-start-guide-and-interactive-seminar/ - Vartiainen, H., & Tedre, M. (2023). Using artificial intelligence in craft education: crafting with text-to-image generative models, Digital Creativity, 34(1), 1–21. 10.1080/14626268.2023.2174557
- Venkit, P. N., Gautam, S., Panchanadikar, R., Huang, T.-H., & Wilson, S. (2023). Nationality bias in text generation. In Proceedings of the 17th Conference of the European Chapter of the Association for Computational Linguistics (pp. 116–122). Dubrovnik, Croatia:
Association for Computational Linguistics . 10.1080/14626268.2023.2174557 - Wang, Y., Wang, D., Liang, W., & Liu, C. (2023). ChatGPT jiao yu ying yong de lun li feng xian yu gui bi jin lu [Ethical risks and avoidance approaches of ChatGPT’s educational applications]. Open Education Research, 2, 26–35. 10.18653/v1/2023.eacl-main.9
- Weidinger, L., Mellor, J., Rauh, M., Griffin, C., Uesato, J., Huang, P. S., … & Gabriel, I. (2021). Ethical and social risks of harm from language models. arXiv preprint arXiv:2112.04359. 10.48550/arXiv.2112.04359
- Witte, H., Blatter, T., Nagabhushana, P., Schär, D., Ackermann, J., Cadamuro, J., & Leichtle, A. (2023). Statistical learning and big data applications. Journal of Laboratory Medicine, 47(4), 181–186. 10.1515/labmed-2023-0037
- Xie, Q., Li, M., & Enkhtur, A. (2024). Exploring generative AI policies in higher education: A comparative perspective from China, Japan, Mongolia, and the USA. arXiv preprint arXiv:2407.08986. 10.48550/arXiv.2407.08986
- Zhou, P. (2023). Unleasing GPT on the metaverse: Savior or destroyer?. arXiv preprint arXiv:2303.13856. 10.48550/arXiv.2303.13856
- Zhu, Y., & Yang, F. (2023). ChatGPT/Chuangshengshi rengong zhineng yu jiaoyu chuangxin: Jiyu, tiaozhan yiji weilai. [ChatGPT/AIGC and Educational Innovation: Opportunities,Challenges, and the Future]. Journal of East China Normal University (Educational Sciences), 7, 1–14. 10.16382/j.cnki.1000-5560.2023.07.001
DOI: https://doi.org/10.55982/openpraxis.17.1.750 | Journal eISSN: 2304-070X
Language: English
Page range: 79 - 94
Submitted on: Sep 13, 2024
Accepted on: Nov 21, 2024
Published on: Apr 15, 2025
Published by: International Council for Open and Distance Education (ICDE)
In partnership with: Paradigm Publishing Services
© 2025 Ming Li, Ariunaa Enkhtur, Beverley Anne Yamamoto, Fei Cheng, Lilan Chen, published by International Council for Open and Distance Education (ICDE)
This work is licensed under the Creative Commons Attribution 4.0 License.