Exploring the Impact of Artificial Intelligence in Advancing Smart Learning in Education: A Meta-Analysis with Statistical Evidence

Research Gap and Objective

Although AI is said to have an effect on student learning, the lack of a meta-analysis on the topic makes an evidence-based conclusion difficult (Dimitriadou & Lanitis, 2023). The gaps in understanding of its effects can be filled by conducting a meta-analysis that can provide robust evidence-based findings on the influence of AI on smart learning advancements. Addressing these gaps was therefore the aim of this study, which had a prime focus on identifying how AI has been adopted at each level of education and in each geographical region, and what kinds of AI tools have been used. Based on the findings, this analysis aimed to contribute to the understanding of how AI tools such as ChatGPT may be implemented in suitable educational practices for the betterment of learning outcomes. To sum up, this study addressed the research lacunae through a comprehensive meta-analysis of the impact of AI on the progress of smart learning, focusing primarily on the overall influence of the use of AI tools on educational outcomes at different educational levels and in different geographic locations.

Research Questions

The major research questions the study aimed to answer are:

1. How can ChatGPT be used to provide personalized learning experiences and help students develop their critical thinking abilities?

2. What are the possible risks and ethical considerations while using ChatGPT in an educational environment?

3. How does the use of ChatGPT in administrative tasks impact instructional quality and efficiency in education?

Literature Review

AI has become a pivotal force in the transformation of education, bringing with it innovative ways to enhance the teaching and learning processes. AI technologies such as OpenAI’s ChatGPT have garnered much attention since they can generate fluent, contextually appropriate responses, simulating human-like interactions. These technologies have been employed to personalize learning experiences, automate administrative tasks, and offer data-driven insights (Javaid et al., 2023; Tlili et al., 2023; Younas et al., 2025; Imran et al., 2025).

Personalized learning is a cornerstone of AI applications in education, tailoring educational content to meet the unique needs of each student. According to Kulik and Fletcher (2016), the existence of intelligent tutoring systems (ITS) that offer feedback features and adaptive learning materials increases the efficiency of student success. There is pacing for every student, so long as the tutor caters to their learning pace. Mhlanga (2023) emphasized that AI offers a personalized kind of learning that improves critical thinking and understanding of the matter.

Engagement is vital for effective learning, and AI technologies such as VR and AR are so created in a manner that engages the student’s mind. Cukurova et al. (2020) further elaborated on how AI-powered VR and AR make abstract concepts tangible and memorable, hence enhancing students’ understanding and retention. Haleem et al. (2022) and Hanshaw et al. (2024) stated that AI is going to change the nature of education itself through personalized learning, and that intelligent tutoring systems will become instrumental to a dynamic technology-infused learning environment.

Evidently, lifelong learning and the development of professionalism have become some of the most salient issues brought about by AI. AI-driven platforms offer personalized learning pathways, helping individuals acquire new skills throughout their careers. Bhatt and Muduli (2023) elaborated how learning becomes truly perpetual, following continuously adjusting labor market demands. This ensures that learners remain flexible in skills, keeping them competitive in the job market through emerging trends.

While there are tremendous potentials of AI in education, its implementation harbors many ethical issues. As Williamson and Eynon (2020) argued, transparency, fairness, and inclusivity in the formation of AI algorithms may prevent or lessen bias in its outputs. Ethical use of AI within an educational context is key to ensuring equal opportunities for all learners. Özer (2024) and Bashir et al. (2025) discussed both the advantages and risks of using AI in education and presents a way forward whereby education should be restructured urgently to fit into the interventions that AI is making against ethical concerns.

Search of the Literature and Screening

Screening of all literature retrieved is carried out to ensure that the study is rigorous and accurate (Di & Zheng, 2022; Ma et al., 2022; Tang et al., 2021; Zhang & Ma, 2023) (Figure 1). The screening criteria for documents are as follows:

1. Literature conducted on AI analysis of the impact of teaching on the results of learning.

2. Type of document must be for experimental research literature.

3. Experiment in the literature must include an experimental group and a control group.

4. The literature must contain data for measuring learning effects (learning performance, learning efficiency, etc.), and the effect value can be calculated. The data include sample sizes (N), mean values (M), standard deviations (SD), or values of both the experimental and control groups that are mainly needed for effect value calculation.

5. Duplicate literature should be excluded.

Figure 1

Calculation Formula

The effect size is an indicator that reflects the strength of the experimental effect or the strength of the variable association. It is not affected by the sample size (or the effect is very small). The difference in effect size was mainly analyzed to explore the differences in the impact of AI technology on learning effects under different teaching conditions. Since the number of included studies was small and it was a small sample study, Hedges’ g (hereinafter referred to as g value) was used as the effect value indicator. When calculating the effect size of each study, the selected data are mainly the mean, standard deviation (SD), and sample size (N) of the experimental group and the control group. According to scholars such as Lipsey and Wilson (2017), when a study contributes more than one effect size in the analysis, it will lead to statistical dependence, thereby causing a deviation in the overall effect. Therefore, when there were multiple independent and dependent variables in the literature to be included in the analysis, only one of them was used to calculate the effect value. For this reason, we selected learners’ learning effect as the only dependent variable.

When the study sample size is small, Cohen’s d will seriously overestimate the effect value. Hedges’ g value can be corrected by multiplying the standardized mean difference (d) by the correction factor (J).

Effect size g (Hedges’ g) calculation:

Suppose a study has two independent groups and we want to compare the means of the two groups, where μ₁ and σ₁ are the population mean and standard deviation of the first group, and μ₂ and σ₂ are the population mean and standard deviation of the second group; if we assume that the two population standard deviations are the same (as is the assumption of most parametric analysis methods), then σ₁₂ = σ₂₂, and the standardized mean difference parameter and the population standardized mean difference are defined as:

In the following section, we will describe how to estimate the value of δ for independent group design studies. Assuming that the variances are equal, σ₁₂ = σ₂₂, this allows for a simple estimate of the standard deviation without discussing the case of unequal variances. We can estimate the standardized mean difference in the group design study as

where ${\overline{X}}_1$ and ${\overline{X}}_2$ are the sample means of the two groups, and the denominator S is the within-group standard deviation, which is obtained by combining the two groups:

where n1 and n2 are the sample sizes of the two groups, and S1 and S2 are the standard deviations in the two groups. The reason for combining the two sample standard deviation estimates is that we assume that the population standard deviations are equal (σ₁ = σ₂ = σ), but the sample standard deviations S1 and S2 cannot be exactly the same. Combining the estimates of the two standard deviations can obtain a more accurate estimate of the common standard deviation.

The sample estimate of the standardized mean difference is often referred to as the Cohen’s (2013). Confusion about this terminology arises from the fact that δ, originally proposed by Cohen as a common parameter for describing effect size in statistical power analysis, is sometimes referred to as d. In this article, we use the symbol δ to refer to the effect size parameter and d to refer to the sample estimate of that parameter.

d^th variance and given by (a very good estimate):

In the formula, the first term on the right side of the equal sign reflects the uncertainty of the mean difference estimate, while the second term reflects the uncertainty value of the S within the estimate.

d the standard error is the square root of V_d

It turns out that d has a small bias; that is, it overestimates δ when the sample size is small. This bias can be corrected by a simple method to obtain an unbiased estimate of δ. This unbiased estimate is called Hedges’ g (Hedges, 1981).

To convert d to Hedges’ g, we use a correction factor, called J. Hedges (1981) proposed an exact formula for J, but in common practice, researchers use an approximation:

In the above formula, df is the degree of freedom, which is used to estimate S within, which is n₁ + n₂ –2 for two independent groups. This approximation usually has an error of less than 0.007. When df > 10 (Hedges, 1981), it is less than 0.035%. Then,

Effect size is a measure that reflects the strength of the experimental effect or the strength of the association of variables (Borenstein et al., 2009; Hwang, 2016). Sample size has no effect on it (or the effect is too small). The difference in the effect size was mainly analyzed to examine the differences in the impact of AI technology on learning effects under different teaching conditions. Since the number of studies included is small and this is a small-sample study, we use Hedges’ g as the effect value indicator hereafter (g value). The data selected when calculating the effect size of each study were primarily the mean, standard deviation (SD), and sample size (N) of the experimental group and the control group. As Lipsey and Wilson (2017) and Hedges and Olkin (2014) stated, when a study provides more than one effect size that is entered into the analysis, it will establish statistical dependence and create a divergence in the overall effect. This is the reason why, when there are several independent and dependent variables in the literature to be included in the analysis, only one is used to calculate the effect value. Accordingly, in this study, we selected only one dependent variable: learners’ learning effect.

When the study sample size is small, Cohen’s d will seriously overestimate the effect value. However, Hedges’ g value can be corrected by multiplying the standardized mean difference (d) by the correction factor (J).

Effect size g (Hedges’ g) calculation:

In the above formula, df is the degree of freedom, which is used to estimate S within, which is n₁ + n₂ –2 for two independent groups.

Data Analysis

Table 1 provides an overview of the main studies on AI applications in education from 2020 to 2023. It classifies the type of AI deployed, such as personal tutors, intelligent support for collaborative learning, or intelligent VR, by continent, subject, and educational level. Also, it lists the intervention period, user roles, and learning context in order to provide a complete picture of how AI technologies have been applied in global educational experiences. This great volume of information informs us of the different approaches and impacts of AI, which has great potential to personalize learning, streamline administrative tasks, and engage learners in all forms of settings.

Publication Bias Test

We conducted bias detection, as the phenomenon of a deviation between the published result and actual result may exist in the analysis of related literature (Duval & Tweedie, 2000; Oh, 2002). Due to the small sample size of this article, publication bias was detected by qualitative funnel plots and quantitative Begg’s tests. As Figure 2 depicts, the points on the funnel plot seemed to be more or less centrally focused to the combined effect value of 0.503 and showed a symmetrical distribution. It was preliminarily judged that publication bias was not obvious. The result of Begg’s test was Z = 1.941 < 1.96, p = 0.052 > 0.05, further indicating that publication bias was not significant. Therefore, the combined value effect obtained in this paper is very accurate and highly robust.

Figure 2

Forest Plot

The collected data were analyzed using meta-essential software, facilitating quantitative analysis. The included studies were evaluated based on heterogeneity to assess variation in the data results. In Figure 3, the forest plot shows analysis of the distribution of the data as an overall estimate. Huang et al. (2020) emphasized that a forest plot is crucial for assessing the overall impact size of a study. Therefore, forest plots were included in the analysis.

Figure 3

Summary Measure and Forest Plot for the Effect Size of Smart Learning Advancement Education Level

Meta-regression was used to analyze the possible variation, specifically in the effect size across different educational levels, including school, high school and university level (Cohen, 2013).

Surface 2 is the combined effect value of all studies. The sample heterogeneity test results showed Q = 276.208, p = 0.000 < 0.10, I2 = 83.708, which indicated that there was great heterogeneity among the samples, and hence a random effects model (Random model) had to be used. Data from Table 2 indicate that under the random effects model, the combined effect value of all studies was 0.503, which reached a statistically significant level of p = 0.000 < 0.001. This shows that AI technology-based teaching had a positive effect on students’ overall learning outcomes. We used the effect size criteria defined by Cohen, namely an effect size of less than 0.2 is considered a small effect, 0.2 to 0.8 is a medium effect, and greater than 0.8 is a large effect.

Effects on Learning Outcomes

Luckin & Holmes (2016) identified three types of AI software applications in education that are currently available: a) personal tutors; b) intelligent support for collaborative learning; c) cognitive virtual reality. As Table 3 shows: a) The combined effect value of personal tutors was 0.302, intelligent support for collaborative learning was 0.531, and the VR training system supported by AI was significant no matter combined impact was at level (p = 0.000 < 0.001), while inter-group effects were all significant (QBET = 13.002, p = 0.002 < 0.01), reaching statistical significance. It could therefore be inferred that the three types of AI technology-based teaching had a moderate statistically positive effect on students’ learning outcomes. There were statistically significant differences in the level of effectiveness across the three types. Analysis of the data showed that the promoting effect of the three types based on AI teaching can significantly improve achievement for all, in achievement and correlative learning effectiveness aspect have guiding role to c) intelligent virtual reality promoting strongly follows b) intelligent support for collaborative learning. Furthermore, a) personal tutors weaker than the rest.

Impact of AI-Based Teaching on Learning Outcomes in Different Regions

The research from various countries was divided into six regions: Europe, Asia, Africa, North America, South America and Oceania. The combined effect value according to Table 3 data for Africa was found to be 0.982, that of Asia was 0.467, Europe was 0.573, and North America was 0.235. The combined effect values of the three most important regions (i.e., Asia, Europe and North America) were all statistically significant (p = 0.000 < 0.001), and North America’s combined effect values were statistically significant (p = 0.006 < 0.01); b) The between-group adjustment chi-squared test (omnibus) showed it to be extraordinarily notable at QBET = 22.114, p = 0.000 < 0.001. These findings show that AI teaching of technology in Africa had a substantial effect on student learning outcomes. A positive impact was also noted for students from the Asian, European, and North American regions, indicating a medium positive impact. Differences in the impact on learning results were not negligible.

We also used data to analyze the performance based on AI teaching of technology in different regions, and concluded that both can significantly promote the realization of teaching objectives and learning effects. The promoting effect was significant in AI teaching of technology, and the order of the various regions from high to low is Africa, Europe, Asia, and North America.

The Impact of AI-Based Teaching on the Learning Outcomes of Different Subject Types

This article divides all research into two types based on subject type: Humanities and Social Sciences, and Natural Sciences. As the Table 3 data show, the Humanities and Social Sciences combined effect value was 0.510, while that of the Natural Sciences was 0.471. The combined effect values for both types of disciplines were all significant at the 0.001 level (p = 0.000 < 0.001); however, the between-group effect was not significant, QBET = 0.264, p = 0.608 > 0.05.

We can conclude that based on AI, the technology-based teaching effect on the learning outcomes of students in different subject types had a medium positive impact. However, there was no significant difference in the learning effect of the two types of subjects. The results of data analysis show that AI technology had a higher positive impact on the learning outcomes of Humanities and Social Sciences compared to Natural Sciences.

The Impact of AI-Based Teaching on Learning Outcomes at Different Educational Levels

This article divides all research into Higher Education, Secondary Education, Primary Education, Pre-school Education and Special Education according to the different levels at which the research was conducted.

As the data in Table 3 show, the combined effect value for Higher Education was 0.448, for Primary Education it was 0.488, and for Secondary Education it was 0.543. The effect values of the three types of education levels combined were all significant at the 0.001 level (p = 0.000 < 0.001), whereas the between-group effect was not significant: QBET = 1.254, p = 0.534 > 0.05.

The Impact of AI Teaching with Different Intervention Durations on Learning Outcomes

This paper divides all the research into one-time and long-term interventions according to the development cycle of AI teaching experiments in each study.

As the Table 3 data show, the Long-term combined effect value was 0.424, while the Short-term was 0.629, and the two types of intervention combined effect values were all significant at the 0.001 level (p = 0.000 < 0.001). The inter-group effect was significant: QBET = 7.046, p = 0.008 < 0.01.

The Impact of AI-Based Teaching on the Learning Outcomes of Different User Roles

This paper divides all the research into the following categories according to the different user roles of each research discipline: Student-led, Teacher-led, Jointly led by student & teacher.

The Table 3 data show that the Jointly led by student and teacher combined effect value was 0.512, Student-led was 0.518, and Teacher-led was 0.312, while the combined effect values of the three types of user roles were all significant at the 0.001 level (p = 0.000 < 0.001). The between-group effect was not significant: QBET = 4.745, p = 0.093 > 0.05.

The Impact of AI Teaching in Different Environments on Learning Outcomes

This paper divides all the studies into three categories based on the environment in which the AI teaching experiments were carried out in each study, namely Formal, Informal and Mixed environments.

As the Table 3 data show, the combined effect value for Formal was 0.458, for Informal it was 0.748, and for Mixed it was –0.195. The combined effect values of Formal and Informal were all significant at the 0.001 level (p = 0.000 < 0.001), but the merging effect in the Mixed environment was not significant. The between-group effect was extremely significant: QBET = 19.049, p = 0.000 < 0.001.