Introduction
With the public release of ChatGPT in November 2022, education encountered another significant sociotechnical moment – one that sparked both enthusiasm and unease. Within days, educators, institutions, and researchers began reacting to the implications of this generative AI tool for assessment, instruction, academic integrity, and notably, learner support. Besides the media headlines, academic researchers and institutions responded with an unprecedented volume of early publications (Weidlich et al., 2025). This flurry of activity, occurring within weeks of the tool’s public availability, offers a rare opportunity to analyze how educational research communities engage with fast moving technological shifts. As Gašević et al. (2023, p. 3) argued for higher education (HE), “the impact of universities as institutions is measured in centuries and millennia in how humanity’s knowledge is discovered and shared. As a result, universities are not measured by their rapid responses to potential trends.” Yet, advanced digital technologies also push universities and other educational institutions toward reforms in their teaching and learning practices. These trends lead stakeholders to explore and reconsider their current practices, sometimes encouraging critical reflections and more fundamental change. With the COVID-19 pandemic, for example, schools and universities were pushed to switch to emergency remote teaching (Bozkurt & Sharma, 2020; Hodges et al., 2020) and had a chance to reflect critically on their digital services. Two years later, education saw another sudden sociotechnical change with the launch of ChatGPT. Yet again, the volume and velocity of academic publications on ChatGPT suggest an emerging mode of scholarly engagement driven by immediacy, uncertainty, and visibility (Kornbluh, 2024). What does this mean for, for example, teaching practice, learner support, or academic publishing?
This paper does not evaluate ChatGPT’s technical affordances or classroom uses per se. It rather examines the early assumptions of the academic community underpinning longstanding concerns such as learner support, tutoring, and how they are reframed through the lens of new technologies. We aim to uncover the assumptions in the first wave of scholarly responses to this latest innovation.
For open, distance, and digital education (ODDE) scholars as well as critical educational technologies (edtech) scholars, who interrogate power, inequality, and the assumptions embedded in educational technology (Macgilchrist, 2021), it is well understood that technological affordances do not come without crucial constraints; thus, each novelty requires careful examination. In this paper, we look into this new development by scoping early literature on ChatGPT, the first mainstreamed large language model (LLM). While many critical voices reacted in blogs and public scholarship (e.g., Caines, 2022; Golumbia, 2022; Williamson, 2023), our goal here is to scope the early work made available through a major commercial research database and identify key themes that emerged in early research on this phenomenon.
The Role of Hype in Edtech
To interpret early academic reactions to ChatGPT, it is helpful to consider the dynamics of technological hype. Within education, particularly under neoliberal pressures, technologies often enter public consciousness through inflated expectations that exceed practical implementation (Cuban, 1986). The Gartner Hype Cycle (GHC) is one lens to work this trajectory. It outlines five phases: Innovation Trigger, Peak of Inflated Expectations, Trough of Disillusionment, Slope of Enlightenment, and Plateau of Productivity, ChatGPT’s release clearly corresponds with the early stages, where scholarly attention is high and discourse, both celebratory and skeptical, proliferates.
There have been many developments in educational technologies that, with a commercial trigger, produce hype. For example, the past trend of MOOCs mirrors a similar urgency. Although MOOCs originated in 2008, they did not receive much attention (Zawacki-Richter et al., 2018). They gained traction only after a media tipping point in 2012, spurred by features in Time Magazine and The New York Times (Webley, 2012; Pappano, 2012).
Recurring Themes in Educational Research and Edtech
Educational research is particularly vulnerable to cycles of rediscovery. Due to its social and contextual complexity, foundational questions in education are repeatedly revisited with each new generation of tools and theories. As Labaree (1998) remarked, “if Sisyphus were a scholar, his field would be education” (p. 9). This sentiment has been echoed more recently by Weller (2020), who noted that edtech often suffers from “historical amnesia” (p. 3), criticizing the field for rarely pausing to reflect critically before embracing the next wave of innovation. Prinsloo (2024) likewise warns of déjà vu in ODDE, questioning whether new tools truly shift educational paradigms or simply repackage old concerns.
The case of ChatGPT retreads this path. Among the most prominent themes in early educational responses is its potential for learner support, particularly in tutoring, writing assistance, and formative feedback. Yet these functions are not novel. From Skinner’s teaching machines (Watters, 2021) to intelligent tutoring systems like AutoTutor (Albert & Thomas, 2000; Graesser et al., 2004), AI-supported academic learner support has a long and contested history. Indeed, VanLehn (2011) found that the effect size of intelligent tutoring systems was comparable to human tutoring, both less than 1.0, respectively, 0.76 and 0.79. The renewed excitement around ChatGPT echoes earlier enthusiasm, yet also risks overlooking decades of prior insights on AI and education (Williamson et al., 2023).
Research trends in ODDE reinforce this continuity. Zawacki-Richter and Naidu (2016) charted a mid-1990s emphasis on learner support, followed by a post-2010 shift toward AI and big data. More recently, intelligent support systems have regained prominence (Bozkurt & Zawacki-Richter, 2021; Zawacki-Richter & Bozkurt, 2023). ChatGPT’s emergence, rather than breaking with tradition, aligns with this longer arc.
Thus, this moment offers a chance to document how educational research re-engages with old questions under new conditions. As digital tools become more powerful and visible, understanding how academic communities respond can illuminate broader dynamics in the field and offer space and evidence to identify the themes with which scholars navigate the hype, complexity, and the promise of yet another transformative technology.
Before turning to the conceptual framework, one caveat is in order: The role of technology in education is not straightforward. Nevertheless, simple binary categorizations have dominated discussions for decades. Complex questions are raised by the debate between “boosters”, who view new technologies as transformative, and “doomsters”, who caution against potential harms (Jensen et al., 2024; Selwyn, 2014). Our exploration of the current hype refrains from adopting a techno-determinist stance, and we recognize that a rich history of research in ODDE has shown that no single theory can meaningfully drive education forward on its own. Rather, it is the alignment of pedagogy, technology, and context that fosters impactful learning environments. Thus, our aim is not to dismiss or condemn the excitement surrounding ChatGPT but to critically examine its potential and limitations through the multiple lenses of educational research.
Conceptual Framework and Literature Review
Generative AI (GAI) and Education
AI in education is not new (Williamson & Eynon, 2020), and LLMs are part of this ongoing evolution. For example, efforts to automate essay scoring using the statistical relationships between essay features and human-assigned grades date back to Page and colleagues’ work on Project Essay Grade (Page, 1966; see Bai et al., 2022, for a review of more recent work; and Dixon-Román et al., 2020 for critique). Various natural language processing (NLP) technologies have been widely used in edtech research. For example, researchers assessing online university reviews have used topic-modelling and sentiment-analysis techniques (e.g., Srinivas & Rajendran, 2019), and other have used fine-tuned LLMs to predict student ratings of teaching quality (Rybinski & Kopciuszewska, 2021).
There are also notable earlier examples of GAI models in the educational technology and academic publishing literature. Namely, the Basic Automatic BS Essay Language (BABEL) text generator was developed by Perelman and colleagues to produce nonsense text that served as adversarial examples to fool automated essay scoring systems (see Perelman, 2020). Similarly, SCIgen was designed as a ‘tongue-in-cheek’ tool to generate nonsense text that could be used to test the review process of computer science conferences with low submission standards (see Labbé & Labbé, 2013).
In contrast to text-generators like BABEL and SCIgen, which were both deliberately designed to produce nonsense outputs, more recent LLMs like ChatGPT have attracted interest for producing “human-like” texts (Floridi & Chiriatti, 2020) that are more difficult for readers to distinguish as synthetic (e.g., Gehrmann et al., 2019). These LLMs are trained to predict sequences of strings based on statistical relationships between tokens (word parts) in massive amounts of data scraped from the internet. The scale of data required to train LLMs make them both expensive to train (in terms of money and energy consumption) and difficult to curate and document (Bender et al., 2021).
Sourcing training data from the internet comes with a number of issues. For example, there is evidence that both open-source (e.g., Gao et al., 2021) and proprietary datasets (e.g., Markov et al., 2023) can propagate existing racist and gender biases (see Bender et al., 2021, for discussion). LLMs and other GAI models have been shown to memorize and output exact or near-exact copies of their training data (e.g., Carlini et al., 2021, 2023). This raises several ethical concerns. First, biases can be encoded from the underlying training data, which reflect the historical and social context they originate from, often reinforcing harmful stereotypes (Guo et al., 2024; Gallegos et al., 2024), namely pre-existing bias (Friedman & Nissenbaum, 1996). Second, there are environmental concerns related to energy-intensive processes required to train and run these models (Macgilchrist, 2024; Wu et al., 2022). Third, issues of plagiarism arise as these models can reproduce copyrighted or proprietary content. Fourth, much of the data used for training is collected without the consent of its creators (see Jiang et al., 2023). Additionally, the labor behind AI model development, often hidden from public view, raises further ethical concerns. For instance, content moderation – often outsourced to low-paid workers in the Global South – exposes individuals to violent, graphic, and toxic material in order to train filters that prevent users from encountering harmful content (Muldoon et al., 2023).
Against this background, ChatGPT has drawn much attention within education research for its potential to aid academic dishonesty (e.g., Susnjak & McIntosh, 2024; cf. Bai et al., 2024). However, the widespread uptake of ChatGPT raises a wide range of other issues for education (Kasneci et al., 2023; Rudolph et al., 2023a) that we explore further by reviewing the initial academic publications below.
ChatGPT in education and academia: Pedagogical affordances and limitations
A growing body of systematic reviews has investigated ChatGPT’s potential benefits and limitations, with an aim to uncover the affordances of ChatGPT. Across all reviews, there is a consistent emphasis on the pedagogical potential of ChatGPT. Lo (2023) synthesized 50 articles from the first three months after ChatGPT’s release, highlighting its use in student support, assessment, tutoring, and course design. Raman et al. (2023), analyzing publications through altmetric attention scores over the first four months, found that within the education cluster, dominant themes are curriculum and assessment, personalized learning, and student engagement. Jensen et al. (2024), examining blog posts and media outlets from the same period, reported that ChatGPT was positioned both as a disruptor and a tool for learner support in writing and thinking, teacher preparation, and assessment redesign, with some suggesting that it could alter teacher roles. Ali et al. (2024), reviewing 112 articles, concluded that ChatGPT supported instructional design, content generation, feedback, and student engagement. Similarly, Zhang and Tur (2024), focusing on K-12 settings, highlighted the tool’s potential to provide students with personalized learning and assisting teachers with differentiated instruction. The review emphasized ChatGPT’s flexibility in adapting to students’ learning paces and in offering accessible forms of feedback. Montenegro-Rueda et al. (2023), synthesizing 12 articles, found ChatGPT’s impact on teaching and learning to be positive – contingent on teachers’ familiarity with the tool.
A range of limitations was also discussed. Accuracy is one of the major limitations (Lo, 2023; Cong-Lem et al., 2024). ChatGPT often produced plausible-sounding but incorrect or misleading answers. Academic integrity was another issue as students can mimic scholarly discourse with minimal effort (Ali et al., 2024; Ipek et al., 2023). This might erode critical thinking (Raman et al., 2023). Ethical and legal concerns were also a widespread issue due to data privacy, algorithmic bias, and the opacity of AI decision-making processes (Cong-Lem et al., 2024).
Baber et al. (2024) reviewed very early publications on ChatGPT from November 2022 to January 2023. They synthesized 34 articles and concluded the research is ongoing and still at the exploration phase with a focus on the GAI applications, which corresponds to phase one of the Gartner hype cycle.
This paper offers another lens on early scholarly responses to ChatGPT during the phase following the initial innovation trigger. This review does not limit its focus to ChatGPT’s affordances per se but also presents how peer-reviewed journal publications mirrored the academic community’s sense of urgency and engagement. Our review focuses on research produced during this transitional period – after the initial release of ChatGPT but while second and third waves of exploration were still unfolding. We focus on education and relevant academic research, irrespective of the subject field. Within this context, we seek to answer the following questions:
RQ1: How are initial publications on ChatGPT distributed in the education field?
RQ2: What is the initial response to ChatGPT in academic publications?
RQ2.1: What are the thematic clusters in initial publications?
RQ2.2: What are the main potential uses discussed in the initial publications?
RQ2.3: What are the harms discussed in the initial publications?
Methods
Research Method
This study employed a scoping review (Arksey & O’Malley, 2005) to explore the development of hype from an educational-technologies perspective. We employed the systematic review steps to collate relevant journal articles and then analysed the emerging themes and content around ChatGPT using social network analysis (SNA) with VOSviewer, text mining with Leximancer, and open coding for content analysis.
Research Design and Procedure
This study design consists of three phases (Figure 1), progressing from a broad overview to a more detailed examination: 1) mapping bibliometric and thematic trends through the analysis of keywords and titles (N = 217), 2) clustering emerging themes based on titles and abstracts (N = 217), and 3) conducting an in-depth analysis via content analysis of full texts (n = 149).
Figure 1
The study design.
To address the first research question, we relied on descriptive findings and conducted a bibliometric analysis (Donthu et al., 2021). For the second research question, we triangulated data analysis methods, incorporating SNA, text-mining, and open coding to provide a comprehensive perspective on the identified themes.
Bibliometric Mapping, Text-Mining, and Content Analysis
Exploring the thematic clusters of a long-established or rapidly increasing field can provide evidence for what is really being discussed in a specific scientific domain. As Krippendorff (2018) pointed out, exploring the thematic clusters based on large body of texts enables a broad understanding of trends and themes within the field under investigation. Previous studies have used automated mining tools to explore thematic clusters in educational research to demonstrate the research focus throughout the years and make evidence-based predictions for the expected research trend (e.g. Bozkurt & Zawacki-Richter, 2021; Zawacki-Richter & Latchem, 2018). As Lee et al. (2004, p. 225) assert, “understanding trends and issues in terms of topics and methods is pivotal in the advancement of research”.
Computer-assisted analyses can be subject to wide interpretation and relying on one tool for the interpretation of thematic clusters could be misleading. The SNA and text-mining tools run automated analysis; however, output from such tools “requires analytical sensitivity and judgment in its interpretation” (Harwood, et al., 2015, p. 1041). Therefore, we first applied VOSviewer based on titles and keywords of the corpus. VOSviewer is a tool to create and display co-occurrence networks of key terms derived from scientific literature (van Eck & Waltman, 2010). Second, we ran Leximancer analysis for text-mining. Leximancer is a tool for displaying the lexical semantic relation across the corpus. Via Leximancer each node is “weighted so the presence of each word in a sentence provides an appropriate contribution to the accumulated evidence for the presence of a concept” (Leximancer, 2021, p. 9). Thus, we created the thematic clusters with semantic indicators of the relationship between nodes.
Third, to analyze the thematic clusters in-depth for the new emerging themes, we retrieved full text articles and conducted thematic analysis (Braun & Clark, 2006). Semantic analysis of the content of the articles via computer-mediated analysis, based on abstracts, titles, and keywords, can yield reliable and rich data. However, they represent the explicit meanings and may omit the nuances of arguments in the body of articles. In the third step, we aimed to capture the deeper meanings in the patterns and themes by iteratively coding thematic units in the full texts to capture the deeper meanings. The first and third authors of this paper employed the six steps of Braun and Clark (2006). The coding was predominantly inductive. The texts were mainly open-coded and data-based meaning was emphasized (Braun & Clark, 2012).
These steps were taken to triangulate the data, cross-check the meanings of the nodes in the full text, and find relevant examples of practical applications, ideas, or concerns relevant to the research questions 2.2. and 2.3.
Data Curation
We created a streamlined search string to include all papers that focused on both “education” and “ChatGPT”. Terms such as “learn” and “train” were intentionally excluded, as they carry different meanings in educational versus machine learning contexts and yielded irrelevant results. After several iterations of refining the string, we conducted the search with the search string in Table 1. We specifically included “GPT-3.5” and “text-davinci” among the search terms, as these refer to earlier model versions that directly underpin ChatGPT’s public-facing capabilities, particularly prior to the release of GPT-4. We excluded earlier models such as GPT-2, which lack the interactive functionality of ChatGPT.
Table 1
Search string.
| CONCEPT | SEARCH TERMS WITH BOOLEAN OPERATORS |
|---|---|
| ChatGPT | “ChatGPT” OR “GPT-3.5” OR “text-davinci” OR “chat gpt”) |
| Education | AND (educat* OR school OR academi*) |
| Limitation | (LIMIT-TO (LANGUAGE, “English”)) AND (LIMIT- (SRCTYPE, “j”)) |
As two different software programs for automated analysis were to be used, we decided to work on one database to eliminate algorithm bias and apply the same exported data format. Scopus, launched as a discovery tool in 2004 (Schotten et al., 2017), enables a wide range of bibliometric analysis with its high-quality data source and covers curated abstract databases subject to high-quality assurance practices (Baas et al., 2020). In addition, the European-based database Scopus provides a wider scope of journals compared to other leading databases such as Web of Science (Falagas et al., 2008).
Table 2 presents the inclusion and exclusion criteria. As the aim of this paper is to capture the initial response to the hype, the criteria aimed at inclusivity with some restrictions.
Table 2
Inclusion and exclusion criteria.
| CRITERIA | INCLUSION | EXCLUSION |
|---|---|---|
| Language | English | Not in English |
| Publication Type | All document types in journals indexed in Scopus | – |
| Study scope | About education and academia | Not education or academia-related |
An exploratory search was conducted on May 23, 2023, and the final search to curate the data for analysis was conducted on July 17, 2023; this yielded 247 records subject to auto-filters of language and source type (Figure 2). Four authors conducted a title-and-abstract screening and excluded papers that were not education related. The remaining titles and abstracts were all included in the automated analysis. For qualitative content analysis of the full texts, 149 records were retrieved.
Figure 2
PRISMA diagram (slightly modified after Brunton & Thomas, 2012, p. 86; Moher et al., 2009, p. 8).
Limitations
Each research design inherently comes with certain limitations based on the decisions made. This study is limited by its use of a single database, albeit one of the largest and most reputable data curation providers. However, this database is not open-access, which poses a barrier to equitable access and raises concerns about the socially just distribution of research resources. This restriction reflects a broader issue in academia, where access limitations are not unique to this database but warrant ongoing acknowledgment in scholarly work until a more inclusive solution is developed. Additionally, because Scopus encompasses a broad range of journals in medicine, it may over-represent medical education outputs within our results. Moreover, some humanities fields that are more likely to have a broader critical approach, addressing cultural, philosophical, societal, colonialist, and intersectional issues, tend to write more in zines, books, edited volumes, and blogs than in journals.
Another limitation is the focus on articles written in English only, which introduces a language bias and excludes valuable contributions from non-English publications. Furthermore, at the time this study was conducted, other versions of the LLM model were not yet publicly available; thus, this focus on the initial model allows for an analysis of early responses to its release. Finally, some critical authors intentionally avoid terms like “generative AI” or specific brand names in their writings, which may have prevented relevant works from appearing in the search results (Tucker, 2022).
Findings and Discussion
RQ 1: Descriptive Results
To have a thorough picture of the included publications (N = 217), we described the corpus based on the following information: countries of the first authors, subject field of the publication, percentages of empirical and descriptive studies, the sub-types of empirical and descriptive work.
The geographical distribution of authors spanned 44 countries and five continents (Figure 3). Authors from the USA contributed the most (n = 51), followed by Australia (n = 24), the UK (n = 19), and China (n = 15). Notably, the African continent remains persistently underrepresented in edtech-related journals (e.g., Bond, 2024; Bozkurt et al., 2025; Zawacki-Richter & Latchem, 2018). This is a trend that continued even during a period marked by heightened urgency to publish on a highly publicized theme.
Figure 3
Geographical distribution of authors.
Table 3 presents the types of research in the corpus.
Table 3
Types of research published in the corpus (N = 217).
| TYPE OF ARTICLES | NUMBER OF ARTICLES (n) | PERCENTAGE (%) |
|---|---|---|
| Conceptual | 168 | 77.0 |
| Empirical | 49 | 23.0 |
Two third of the corpus was composed of conceptual papers with a variety of scholarly publication types (see Table 4).
Table 4
Types of conceptual research (n = 168).
| TYPE OF CONCEPTUAL RESEARCH | NUMBER OF STUDIES (n) | PERCENTAGE (%) |
|---|---|---|
| Commentary/letter to editor | 53 | 31.7 |
| Discussion | 48 | 28.7 |
| Editorial | 23 | 13.8 |
| Q&A with ChatGPT | 17 | 10.2 |
| Review | 16 | 9.6 |
| Intervention | 8 | 4.8 |
| Other | 2 | 1.2 |
The conceptual papers on ChatGPT in education showed a diverse range of document types. The largest proportion (31.5%) consisted of commentaries or letters to the editor, where educators and researchers share their opinions or reflections on ChatGPT’s role. Discussions made up almost 29% of the content, highlighting ongoing debates and conversations about its impact on educational practices, while editorials accounted for 14%. Ten percent of the papers were question and answer (Q&A) sessions with ChatGPT, showcasing interactions with the LLM. Reviews (9.5%) were diverse in their approach. Some examples are narrative overviews on the history of ChatGPT (Wu et al., 2023), development of ChatGPT and its comparison to other LLMs (Ray, 2023), comparison of affordances of LLM models (Rudolph et al., 2023b; Singh & Singh, 2023), list of the growing roles of ChatGPT concerning scientific writing (Ciaccio, 2023). There is a meta-analysis signaling improvement in students’ learning (Wu & Yu, 2024), a scoping review of the use of LLM in medicine (Kim et al., 2023), and a systematic review focusing on teachers’ role (Gentile et al., 2023). Intervention studies (4%) explored specific applications of ChatGPT to solve educational challenges, and the remaining 1% included miscellaneous content.
This range of publication types demonstrated the multifaceted ways in which ChatGPT was being explored in the educational landscape and underscores a tendency toward rapid, often opinion-driven dissemination. As shown in Tables 3 and 4, the publication trend was skewed toward brief opinion pieces or exploratory reports testing ChatGPT’s capabilities. This picture mirrors concerns raised by Weidlich et al. (2025) about the rush, where conceptual clarity and methodological rigor are often sacrificed in the urgency to contribute to a highly visible theme or tool.
Table 5 outlines the distribution of empirical research types within the corpus. The largest category was studies testing ChatGPT. Fifteen studies out of 17 compared ChatGPT with exam results. Two studies tested ChatGPT not against exams; one tested and analyzed its fake referencing (Day, 2023), and the other compared ChatGPT produced radiology articles with real articles (Ariyaratne et al., 2023). The second largest category was qualitative research, accounting for 30% of the total empirical studies (n = 15). Quantitative research represented 26% (n = 13), while mixed-method research accounted for 10% (n = 5).
Table 5
Types of Empirical Research in the corpus (n = 49).
| NUMBER OF STUDIES (n) | PERCENTAGE (%) | |
|---|---|---|
| Test ChatGPT | 17 | 34.0 |
| Qualitative | 15 | 30.0 |
| Quantitative | 13 | 26.0 |
| Mixed | 5 | 10.0 |
This breakdown highlights a balanced emphasis on ChatGPT-related comparisons and qualitative research. The initial empirical papers tried to understand the scope and capability of ChatGPT by testing them via exam questions or through interacting with it.
Table 6 presents a breakdown of various disciplines by their respective counts and percentages. Health and Medicine dominated the distribution and accounted for 31.8% with 69 articles. Education followed with 16.6%, while a notable 14.7% of cases did not mention the discipline. Business and Law; Science, Technology, Engineering, and Mathematics (STEM); and Arts, Humanities, and Social Science represented 12.4%, 9.7%, and 8.3% of the corpus, respectively. Computer and Information Science had a smaller share at 4.6%, with Library and Support services and other being minimally represented at 1.4% and 0.5%.
RQ 2.1: Thematic Clusters in Initial Publications
As titles and keywords are the most succinct and condensed sources of information from scholarly articles, the main themes in the corpus were first analyzed using social network analysis of the titles and keywords. The Scopus database also works with VOSviewer. Therefore, the initial mapping of the themes based on social network analysis between the meaningful word collection helps us see the direction of the discussions.
The results display three interconnected cluster areas (Figure 4). The first cluster is organized around academic publishing related topics such as: authorship, text bias threat, authorship and researcher principles and ethics, plagiarism, academic integrity, and content recommendation.
Figure 4
Main themes based on titles and keywords (N = 217).
The second cluster reflects on the impact of the new LLM tool on education. This cluster highlights how the corpus discusses the potential opportunities and challenges, strategies, and approaches to ChatGPT, the future of teaching, and assessment opportunities in higher education.
The third cluster concerns the capabilities of the new LLM model. The initial publications test the platform’s accuracy and reliability, utilizing question-and-response tests and exploring the ‘critical thinking’ capability. These tests are mostly performed in the field of medical education.
To analyze the labels more deeply and see their semantic relations, thematic clusters of the titles and abstracts were analyzed through text mining and a concept map of the discussed themes using the text mining tool Leximancer (see Figure 5).
Figure 5
Concept map based on titles and abstracts (N = 217).
The concept map (Figure 5) was based on semantic relations and revealed seven thematic areas: AI, education, academic, ChatGPT, questions, authorship, and medicine.
Education cluster: The potential of ChatGPT in assessment
The use of ChatGPT for assessment was one of the key foci in the education cluster (see path education – teaching – assessment). For example, Deebel and Terlecki (2023) discussed the potential use of ChatGPT for self-assessment in urology education. As ChatGPT performed well enough, the authors suggested that newer models could potentially be used by urology trainees and professors. Such potential applications also came with a discussion on the need for authentic assessments of students as traditional assessments are subject to losing their evaluation rigor (e.g., Lawrie, 2023). This also inspired calls for higher education institution leaders to revisit their existing practices and focus on continuous assessment of learners (Chaudhry et al., 2023).
ChatGPT elicited many positive responses for its potential uses, while also spurring discussion on its limitations as a replacement for human counterparts (see path potential – education – limitations – issues). Some authors expected LLMs to close the gap soon with human performance in human assessment, requiring a deep transformation in the educational approach (Khademi, 2023).
Education cluster: Critical thinking
Studies also explored the reinforcement of students’ critical thinking skills with ChatGPT (see path students – learning – critical – skills). Using ChatGPT to assist problem-solving steps could help students improve their deductive abilities and, thus, critical thinking skills, in blended learning environments (Sánchez-Ruiz et al., 2023).
Education cluster: Medicine
The education cluster also branched into medicine, which was one of the prominent fields testing ChatGPT’s capabilities. For example, Khan et al. (2023) tried creating a multiple automated assessment. Projecting its potential use for patient counselling, Friederichs et al. (2023, p. 2) considered ChatGPT a rich resource for medicine students that enables interactive access to factual information regardless of time or location. Furthermore, medical students (and patients) could use the service it provides for medical decisions in the future (Friederichs et al., 2023). On the other hand, other authors considered its capacity to interpret data as limited compared with human judgment (Abdulai & Hung, 2023; Huh, 2023).
Exploring the limits of ChatGPT
The AI cluster reveals the accuracy and performance tests of ChatGPT (see questions – testing – ChatGPT – responses – accuracy – performance). One of the highlights of the model was that it provides dialogue-based interaction, trained on vast data sources (e.g., Friederichs et al., 2023). That is why many publications first sought to test its limits, capabilities, and response accuracy (e.g., Chaudhry et al., 2023; Oh et al., 2023; Vasconcelos & dos Santon, 2023). For example, Gilson et al. (2023) tested the performance of ChatGPT on questions from the US medical licensing examination and concluded it could provide reasoning and contextual information. Chaudhry et al. (2023) tested the model by asking it to solve a variety of assignments at the undergraduate level and showed that the answers were comparable with answers from high-performing students. Howell and Potgieter (2023), on the other hand, tested its performance on a topic in telecommunication policy education. The essays written by the LLM had the low quality of poorly passing papers, and exhibited biases and inaccuracy against the rubric used.
Authorship and ChatGPT: Creativity vs Integrity
The path across AI (see ethical – AI – potential – use – tools – integrity) and its reach to the distinct thematic cluster of author touch on two primary issues concerning ChatGPT and similar LLMs: a) the model as an author itself, and b) implications for authorship/ownership and writing. Due to its content-generation capabilities, the model has garnered significant academic interest (e.g., Curtis, 2023; Gefen & Arinze, 2023), with researchers underscoring its capability to create and design new learning materials (e.g., Marquez et al., 2023; Miao & Ahn, 2023). This creative potential raised concerns around implications for authorship/ownership, particularly around concerns around dishonesty and plagiarism (see section 4.4. for more details), which urged the need to check and identify sources of information. Cingillioglu (2023), for instance, conducted an empirical study using an n-gram bag of words, discrepancy language model, and support vector machine algorithm to classify essays and distinguish human-generated essays from AI-generated ones. A further ethical concern about authorship was the dominance of biased content if ChatGPT co-authors learning materials or other texts (Fuchs, 2023).
Connected to integrity, the issue of ownership in AI-generated content was also an important theme (see path ChatGPT – models – language – writing – academic – publishing). The language models have spurred calls for policy development in academic publications. For instance, Crawford et al. (2023) responded to the new development promptly in an editorial, clarifying their journal’s stance on chatbot use in writing. Since the model cannot be held accountable for its creation, authors have to be transparent and responsible in their usage. They stated:
The Journal of University Teaching and Learning Practice promotes research and scholarship that have transparency in methods, and integrity and truth practiced by authors to ensure that quality foundations are built, on which teaching and learning can advance. For AI-informed manuscripts, this means a clear statement in the acknowledgements of AI-usage, and where used for methods, it must similarly be referred to there. (p. 7)
Garcia (2024) responded in a letter to warn the academic community to be careful about using AI tools for writing peer review reports, despite their potential benefits against time constraints. Rahimi and Abadi (2023) shared their concern about authors for failing to acknowledge or declare their use of ChatGPT.
RQ 2.2: Potential Uses of the Model in Education and Scholarship
Following the fundamental understanding of the publications via automated content analysis, we conducted a qualitative content analysis of the full texts. As a result, we identified two main roles of ChatGPT as discussed in the publications: learner support and authorship assistance, both with significant implications for education and scholarly communication (Table 7). As the corpus consisted of diverse publications, from notes to empirical studies, one representative example was selected for the table among the ones that most clearly articulated the themes and contributed substantially to the coding process.
Table 7
Potential uses of ChatGPT in education and scholarship (n = 149).
| MAIN USES | SUB-FIELDS | REPRESENTATIVE EXAMPLES |
|---|---|---|
| Learner support | Information service and source |
|
| Personalized learning |
| |
| Feedback and assessment |
| |
| Critical thinking reinforcement |
| |
| Authorship assistance | Co-authoring |
|
| Assistant bot |
| |
| Democratization of science |
|
Learner Support
The potential of ChatGPT to support learning and teaching was central in the publications, through its roles as 1) information source and service, 2) personalized learning provider, 3) feedback provider and assessor, and 4) critical thinking reinforcer. These functionalities align with previous findings (Lo, 2023), highlighting ChatGPT’s potential to create personalized educational materials, facilitate collaborative learning, and self-directed learning. For Gefen and Arinze (2023), for instance, ChatGPT is a versatile tool for accessing reliable and contextualized information, and thus a valuable resource for students.
The tailoring of educational content to individual needs was also a core topic. From this perspective, ChatGPT enables learners to access customized material that aligns with their specific learning goals and styles. This includes creating individualized lesson plans, study guides, and practice exercises (Gentile et al., 2023; Marques et al., 2023; Thakur et al., 2023). Real-time assistance and customized feedback are mentioned, through which learners can identify the areas they need more support or generate personalized learning plans (Fuchs, 2023). For some researchers, ChatGPT would eventually bring the educational transformation for personalized learning to the desired level, as it is capable of recommending learning content via varied methods for learning that suit best (Yu, 2023). This personalization goes further to assist groups that need additional support for diverse reasons. For example, Yang et al. (2023) considered that children with Down syndrome (trisomy 21) might benefit from ChatGPT for daily tasks as well as develop their literacy and numeracy skills.
A recurrent theme was ChatGPT’s potential to offer timely, detailed and constructive feedback on student work (Gentile et al., 2023; Rasul et al., 2023), which would support self-assessment and skill improvement (Deebel & Terlecki, 2023). In this sense, its capacity to evaluate responses and suggest improvements adds to its utility as a learning assistant.
Another application of ChatGPT was engaging learners in dialogue to foster analytical and critical thinking skills through dialogues. Yilmaz and Karaoglan Yilmaz (2023) observed that undergraduate students’ self-efficacy, motivation, creativity and computational thinking skills increased in a programming course that used ChatGPT in such a way.
Overall, regarding learner support, authors utilized this new technology to revitalize previous calls for a new teaching approach and philosophy, claiming that this technology yields a space to think outside of the box (Tlili et al., 2023).
Authorship assistance
Beyond its impact on education, ChatGPT, as a representative of other LLMs, also created a flurry of interest in authorship. Cotton et al. (2024) demonstrated an example of ChatGPT’s contributions to their manuscript while writing their manuscript about ChatGPT. They acknowledged how ChatGPT made a reasonable contribution to the draft that they even considered registering it as a co-author. However, it could not be held accountable for the text it generated. One of the highlights was its prospective benefit towards the democratization of science. As many scholars are facing language barriers to disseminate their knowledge or to collaborate with academics, the new model was found to be positive in overcoming these barriers. Ellaway and Tolsgaard (2023, p. 661) stated, “LLMs could be used to translate and correct manuscripts in ways that could reduce language barriers, thereby allowing scholarly work from non-native English-speaking countries to be considered on a more equal footing”.
Authorship was not limited to academic publishing, but authoring teaching materials as scholars and teachers could use ChatGPT as an assistant bot. Jeon and Lee (2023) identify four key roles of ChatGPT as a result of their qualitative study with language teachers. ChatGPT was used as an interlocutor, content provider, teaching assistant, and evaluator. These roles went beyond teaching and learning practices and foregrounded the human-AI relationship in the distribution of authorship power. The paper argues that this interaction, on the one hand, supports teachers in discovering, editing, and improving quality and accessibility by leveraging the strengths of both human insight and AI efficiency; on the other hand, it requires competence in incorporating pedagogical knowledge and using ChatGPT as an assistant chatbot (Jeon & Lee, 2023, p. 15888).
The initial phase of excitement surrounding ChatGPT, marked largely by positive responses, indicates a pervasive, cautiously optimistic hope for enhanced learner support and new paths for individualized learning opportunities. Addressing individual needs, customizing learning materials, accommodating self-paced learning designs, and supporting self-directed navigation through the learning process are challenges that are difficult to manage at scale. Thus, ChatGPT was regarded as a potentially valuable tool to support both the learners and the education system in these areas, which is particularly relevant where scale is considered necessary or unavoidable. Furthermore, its capability of generating text also raised hopes recognizable from previous edtech hype cycles to support academic work, lessen the teaching load, and increase productivity.
RQ 2.3: Harms Around the Model in Education and Scholarship
The qualitative content analysis yielded four main fields for the challenges and concerns (see Table 8). ChatGPT’s use in academic domain could exacerbate social harms caused by a lack of accessibility, accountability, and social justice. A significant concern identified in the literature was around biased knowledge. As Cingillioglu (2023, p. 262) highlighted, “these projects raise ethical concerns about predictive policing, entrenching retrospective discrimination and bias into future policing decision-making and strategies. “All these points are particularly problematic in education, where AI-generated content can reinforce pre-existing discriminatory discourses. Fuchs (2023) added to that by indicating how students relying on ChatGPT may inadvertently propagate already biased information, limiting critical thinking and diverse perspectives. Tlili et al. (2023) presented a similar concern found on diminishing critical and innovative thinking as much as its potential to foster it. Using chatbots to get an easy answer might affect critical thinking adversely.
Table 8
Harms and concerns about ChatGPT in education and scholarship (n = 149).
| HARMS/ CONCERNS | SUB-CATEGORIES | REPRESENTATIVE EXAMPLES |
|---|---|---|
| Limited knowledge production and co-construction | Biased information dissemination and reproduction |
|
| Over-reliance on biased knowledge |
| |
| Data ownership & privacy | Reliance on few original research contributions |
|
| Privacy & Security |
| |
| Accessibility, Accountability, and Social Justice | Ownership |
|
| Social justice |
| |
| Academic Integrity, Ethics, and Plagiarism | Plagiarism |
|
| Publication integrity & ethics |
|
Issues surrounding data ownership and privacy were also extensively discussed. Kasneci et al. (2023) underscored concerns about the storage and potential misuse of sensitive data generated by AI tools. Similary, Lund and Wang (2023) cautioned that AI-generated content can inadvertently reveal personal data, a risk to individuals’ privacy. Yu and Guo (2023) highlighted the absence of robust data security measures, an alarming situation about the potential exploitation of user data.
Ownership was a major issue in the early publications. Although many publishers accepted ChatGPT as a co-author, and it can indeed produce writing, ChatGPT had no authority in its produced texts and could not be held accountable for the generated content (Howell & Potgiete, 2023). Additionally, the use of untraceable data sources undermines transparency, making the verification of the information difficult or impossible (Gašević et al., 2023; Ivanov & Soliman, 2023). That would eventually affect the marginalized communities that rely on accurate data for advocacy. Affordability also poses a challenge due to subscription fees for access or integrated services. Underfunded libraries and institutions would not be able to afford it at scale, resulting in a widening of the knowledge gap (Emenike & Emenike, 2023).
Academic integrity emerged as another critical area of concern, particularly with the increasing use of ChatGPT among students. O’Connor (2023) and Perkins (2023) drew attention to the growing trend of students using AI tools to cheat that raises ethical questions about assessment practices. Fabricated referencing (Eysenbach, 2023; Howell & Potgiete, 2023), and irresponsible or uninformed use of ChatGPT by students (Masters, 2023) made plagiarism another core issue. Academic integrity also arose as an issue for publishing. Peer-review, a crucial element of the publishing process, should not be handed over to ChatGPT and editors should be careful about their policies because critical appraisal by experts in the field is fundamental (Garcia, 2024). An additional concern was that ChatGPT could be the new catalyser for predatory journals, with authors using it for fast writing and publication (Yatoo & Habib, 2023).
The initial publications about the capabilities and applications of ChatGPT demonstrated a wide spectrum of harms and concerns. The power of ChatGPT to generate content raised issues around its accountability, the resource pool it feeds itself from, and the potential misuse of it intentionally or unintentionally by students, publishers, and scholars, leading to the dissemination of unreliable information. Even if ChatGPT functioned in a reliable way, the accessibility and affordability of it by all groups remains a problem. Also, as ChatGPT is based on already biased knowledge, its use has a high potential to reproduce biased knowledge, and thus affect high-stakes decisions in social and political realms.
Conclusion, Implications, and Suggestions
This study examined the early academic response to the release of ChatGPT, capturing how educational researchers engaged with yet another wave of technological hype, analyzing publications indexed in Scopus up to July 17, 2023. Through a combination of automated and qualitative content analysis, this study shows how ChatGPT was framed as a potential transformative edtech innovation, with the discourse largely following familiar patterns seen in past hype cycles.
Our analysis revealed a swift, dynamic academic response to the launch of ChatGPT, with 217 entries in a single research database within a short timeframe, most of which are conceptual, and relatively few empirical studies of robust design. The main themes cluster around testing the capabilities of ChatGPT, its impact on academic writing, and future practices in learning contexts. Most papers avoid solely encouraging the development but include possible harms, concerns, and/or limitations of LLMs, at individual, institutional, and societal scales. Both the potential uses and harms listed concur with previous reviews (e.g., Ali et al., 2024; Lo, 2023; Zhang and Tur, 2024), highlighting the potential of ChatGPT for personalized learner support in academic writing, feedback, and individualized materials. The limitations also echoed the harms and limitations mentioned in the literature (Ipek et al., 2023; Raman et al., 2023), such as academic integrity, including both publication ethics and plagiarism in learning process, limited knowledge production, and accessibility. Nevertheless, the ideas are at a surface, listing level rather than providing a deep empirical analysis of the affordances and constraints of ChatGPT.
Despite its limitations, including reliance on a single database, an overrepresentation of contributions from the medical field, and the exclusion of less formal yet influential platforms such as blogs, this study enables evidence and space for more critical reading of what is being published on hype topics. Even within its limitations and covering an early stage, the ever-growing number of commentaries, reviews, and meta-analyses was already present in this very early stage, hinting at a huge impact of ChatGPT. All publications were published in peer-reviewed journals, thus requiring a longer time to be published. This hints at a tendency to treat the technology itself as the center of inquiry, again and again. However, education is too multi-faceted, too context-dependent, and too socio-politically embedded to be driven by techno-centric speculation alone.
The number of reviews and meta-analyses has been growing, and these publications will eventually form the decision-making processes of the institutions for adaptation, adoption, and even change. As recently highlighted by Weidlich et al. (2025), early research on ChatGPT exhibits hallmarks of fast science (Frith, 2020), characterized by methodological shortcuts. Such a tendency would eventually fail to disentangle media from method. Within this exploration of ChatGPT, this review, echoing Weller’s (2020) critique of edtech’s historical amnesia, reveals a recurrence of familiar narratives, this time framed around GAI.
Overall, the literature illustrates the hype surrounding the use of ChatGPT for learner support and for more personalized learning, without mentioning that technological support has been recognized as an ongoing research topic in the last sixty years. Compared to the more mature research traditions on intelligent tutoring systems, ChatGPT is often presented as an innovation in itself rather than one element within a pedagogical frame in the continuum of longer traditions. This tool-centric orientation holds the risk of attributing positive effects to technology, rather than as the result of underlying instructional and pedagogical methods. Such an approach holds the risk of drawing hasty and inappropriate causal conclusions.
Moreover, despite the discussion around the harms, the publication do not discuss GAI as a commercial product. Many discussions fail to critically interrogate the inherent limitations of these models, particularly the biases in their training data, their reliance on probabilistic language generation, and the broader implications of their integration into educational systems. Despite the plethora of critical voices on social media, blogs, and in public scholarship, there is a notable lack of a more cautious, deeply critical approach in these early research publications.
Seemingly, the hype has not plateaued yet. It is not hard to see at this stage that the educational field will continue to attract commercial interests. This continues to raise concerns about the potential shift from pedagogically driven approaches to data-driven, market-oriented approaches (Selwyn, 2014). While many scholars have addressed data-relevant, mass-GAI practice-relevant concerns with different focuses (e.g. Bozkurt et al., 2024; Nichols et al., 2025; Xiao et al., 2025; Williamson et al., 2023), there remains a need for stronger critical voices in leading academic publications in education.
While many see solid potential for using LLMs to support, for instance, students and learning, for this to come to fruition and become more than the ‘coulds’, ‘mights’ and ‘potentials’ that we so often read in edtech research, the use of LLMs will require reflective embedding in the technologies’ individual, interpersonal, social, cultural, economic and planetary contexts (Williamson et al., 2024). Universities worldwide should avoid hastily embracing digital transformation solely for the sake of keeping up with trends and adopting new, emergent technology. It is clear that these technologies are still following profit-based logic. Instead, a thoughtful approach, grounded in research, evidence, and contextualization, is essential to harness these technologies effectively.
Data Accessibility Statement
The datasets used and/or analysed during the current study are available from the corresponding author on reasonable request.
Sustainable Development Goals (SDGs)
This study is linked to the following SDG(s): Quality education (SDG 4).
Acknowledgements
The authors thank the editor and anonymous reviewers for their feedback.
Competing Interests
The authors have no competing interests to declare.
Author Contributions (CRediT)
BC: Conceptualization, methodology, formal analysis, investigation, data curation, visualization, writing—original draft preparation, writing—review and editing; FM: Supervision, writing—review and editing; JHYB: formal analysis; writing – reviewing & editing; HEG: formal analysis; writing – reviewing & editing; OZR: Supervision, writing – reviewing & editing; FSL: Editing. All authors have read and agreed to the published version of the manuscript.
Author Notes
Based on Academic Integrity and Transparency in AI-assisted Research and Specification Framework (Bozkurt, 2024), the authors of this paper acknowledge that World Map in this paper were initially created with the assistance of Power AI (January, 2025). These visualizations were later adjusted and finalized by the authors to accurately represent the research data and to ensure they meet academic standards. The authors also assessed and addressed potential biases inherent in the AI-generated content. The final version of the paper is the sole responsibility of the human authors.