Introduction
While previous research has explored AI’s role in online learning broadly, there is limited empirical evidence on its direct impact on student engagement in asynchronous settings. This study aims to fill that gap by analyzing the use of AI course assistants at Los Angeles Pacific University. With a growing trend and demand for online learning, asynchronous online courses are convenient and even preferred by students in higher education institutions today. However, this does not come without challenges. According to the National Center for Educational Statistics (2023), in Fall 2021 roughly 9.4 million students, or 61 percent of all undergraduate students, were enrolled in at least one distance education course. Asynchronous courses can be characterized by a lack of meaningful interactions amongst students, and between students and their instructors (Vermeulen & Volman, 2024). This lack of belonging and interaction can lead to feelings of isolation or even indifference as students engage with course content. Imagine experiencing unfamiliar content through a disengaging medium, essentially in a vacuum, where other human-to-human interaction rarely occurs. Asynchronous online learning also presents a challenge for instructors as they must engage with students online, most often through a Learning Management System (LMS), leaving few opportunities to engage directly and assess true understanding of course content (Bergdahl, 2022). With the widespread availability of generative AI tools, instructors and students have new ways to engage with course content, including on-demand explanations, examples, and interactive practice that may complement existing online learning activities.
These AI tools are commonly used in education as assistants that provide some level of personalized learning (Lo, 2023). Depending on the type of tool, students may use it to gain course-specific knowledge, or as a means of expanding their knowledge outside of the course (Tsz Kit Ng et al., 2025). Rather than removing limits on learning, AI systems expand access to just-in-time information and feedback within the constraints of their training data, prompt design, and accuracy. In practice, common educational uses include obtaining formative feedback on drafts, clarifying assignment expectations, rehearsing domain-specific skills, and using AI as a “study buddy” to ask follow-up questions or review key concepts (UK Department for Education, 2024). At the same time, these tools have well-documented limitations, including the potential for incorrect or fabricated responses and concerns about academic integrity, which underscores the need for empirical evidence about their effects in authentic course settings, particularly in fully asynchronous online programs like LAPU.
Generative AI tools may be part of the solution to fostering a greater level of engagement with the course content because they supply personalized content, automated feedback, and simulated interactions to students. These interactions may be the driving force behind student engagement in previously disengaging asynchronous courses. Generative AI tools may also benefit instructors by automating monotonous administrative tasks, which leaves more time for instructors to engage with students (Su & Yang, 2023, p. 365).
The evidence shows an ever-changing landscape in higher education, partly due to the popularity of asynchronous online learning and new technological advances of generative AI tools. But, with changes come challenges. Student engagement is a present challenge and an area of focus in higher education. Studies indicate that students who actively participate in class (e.g., contributing to discussions, asking questions, and persisting with challenging tasks) and who report interest and investment in their coursework show greater gains in grades and standardized measures of achievement than students who are merely compliant or disengaged (Christenson et al., 2012; Fredricks et al., 2004; Trowler, 2010). Therefore, as new advancements in modality continue to arise and new technologies become part of the learning landscape, the need for continual assessment of engagement is necessary. For this reason, the purpose of our study is to explore the effects that generative AI tools have on student engagement in asynchronous online learning.
Literature
Asynchronous Online Learning
Online education can be either synchronous, taking place in real-time, or asynchronous, involving predetermined materials and activities that students engage with on their own time (Tartavulea et al., 2020). Online learning includes all educational activities that students participate in, such as lectures, seminars, and small group meetings, as well as one-on-one supervision and asynchronous activities, such as contributing to an online discussion platform, reviewing resources, or completing assignments (Vermeulen & Volman, 2024). Various types of learning activities have been evaluated for their effectiveness in asynchronous online learning. Discussion forums are notably one of the most interactive elements for the online classroom, because they create a constructive and creative dialogue between the student, their peers, and their instructor, which facilitates learning and critical thinking in the online environment (Varkey et al., 2023). In synchronous learning environments, one of the main strengths is real-time interpersonal communication, natural language, and immediate feedback–which is not typically available in asynchronous learning environments (Blau et al., 2017). However, this can be achieved with the incorporation of artificial intelligence in asynchronous courses (Hooda et al., 2022).
Los Angeles Pacific University provides asynchronous online learning through a Learning Management System (LMS), which allows for interactions between students and their instructor and classmates. Los Angeles Pacific University uses Moodle as the LMS to host its courses. Students navigate independently through their course to complete activities such as assigned reading or review materials, engaging with instructional media, completing discussions with peers, and independently completing assignments. During the 8-week term, learning content is structured into weekly modules that typically include a specific topic, review resources, and a discussion forum, with all activities due at the end of the week. In each course, there are also a minimum of 3 assignments that assess learning and understanding of the content and objectives. These asynchronous activities will be discussed and assessed further in the following paragraphs.
Independent learning is one of the main benefits of asynchronous learning for adult learners. Students are able to interact with the content of the course and their peers at the time that best suits them, given the respective deadlines of each activity. Yet this benefit also presents challenges, such as engagement with the content, ability to assess true understanding, and feelings of isolation. For this reason, our desire is to evaluate the use of LAPU’s course assistant to measure the levels of cognitive, affective, and behavioral engagement among students. This study aims to evaluate the student’s experience and use of the course assistant to assess student engagement in each of these dimensions.
Engagement
Engagement is crucial to student learning and satisfaction in online courses (Vermeulen & Volman, 2024). As we seek to find ways to positively impact student learning and satisfaction, our attention will be focused on engagement. Banna, Lin, Stewart, and Fialkowski (2015) stress that engagement is the key solution to the issues of learner isolation, dropout, retention, and graduation rate in online learning. Each of these issues are consistently relevant to higher education institutions, and must continue to be addressed and assessed. Yet, the complexity of engagement still poses a challenge and opportunity to further explore. Research presents a struggle to find a clear centralized definition of engagement for online learning (Martin & Borup, 2022), and some have proposed that engagement be clearly defined by the researcher (Christenson et al., 2012). Therefore, it is important for us to clearly define engagement for our study. Student engagement is defined as: “the student’s psychological investment in an effort directed toward learning, understanding, or mastering the knowledge, skills, or crafts that academic work is intended to promote” (Newmann, Wehlage, & Lamborn, 1992, p. 12). Therefore, we will be evaluating the academic work of students in asynchronous online learning environments in order to determine engagement.
Three Dimensions of Engagement
Student engagement is often conceptualized along three dimensions: behavioral, cognitive, and affective engagement (Fredericks et al., 2004; Bond et al., 2020). For our study, we will be assessing engagement through these dimensions. Vermeulen and Volman (2024) assessed engagement through different types of activities in the online environment in each of these three dimensions, and we will employ the same strategy to evaluate the student’s experience and use of the AI course assistant.
Behavioral engagement
Martin and Borup (2022) define behavioral engagement as the physical behaviors and energy that students demonstrate when completing learning activities because this definition focuses on both the individual (i.e., behavior and energy) as well as the other people or materials with which behavior manifests (i.e., affordances of the technology for various types of behavioral engagement) in the online learning environment. Behavioral engagement can be understood as an effort of participation and involvement in learning activities. Behavioral engagement can be challenging for students attending online classes due to the difficulty of developing a daily schedule while not being physically present at a ground campus (Vermeulen and Volman, 2024). Behavioral engagement facilitates learning at all levels, so it is extremely important that students practice good time management and access their respective course activities at appropriate times during the term. At LAPU, asynchronous learning activities include a variety of assignment types, such as discussion forums, case studies, simulations, gamified activities, and reviewing of learning media. Beyond these, students are responsible to engage with material from textbooks, reference materials, and utilize the course chatbot in order to round out their asynchronous learning experience. The presence of AI tools within the LMS can influence behavioral engagement by creating an interactive learning environment that is available for students to access at any time, which is supported by the flexibility of online courses (Garcia et al., 2025).
Cognitive engagement
According to Vermulen and Volman (2024), cognitive engagement can be measured through generated discussion and the offering of pre-recorded videos and lectures. In their research, several asynchronous activities involved personalization to stimulate students’ cognitive engagement before online lectures occurred. For instance, pre-recorded videos enabled some students to better customize their education based on their own needs and preferences (Vermeulen and Volman, 2024). Martin and Borup (2022) define cognitive engagement as the mental energy exerted toward productive involvement with course learning activities, because this definition focuses on both the individual (i.e., mental energy exerted) and the learner’s interactions with others or materials (i.e., productive involvement) in the online learning environment. We will be adopting this definition for our study. AI supports the personalization of learning and the interaction with materials by serving as an additional tool for students to access. Students may interact with various AI tools to ask questions, expand learning beyond the provided content within the course, and learn more about the content that they are passionate about. In that way, AI supports cognitive engagement by providing students with more opportunities to direct their own learning (Garcia et al., 2025).
Affective engagement
Martin and Borup (2022) define affective engagement as the student’s emotional response to learning activities, or the emotional energy that students associate with learning activities. This definition emphasizes both the individual (i.e., emotions) as well as the online environment affordances (i.e., tools for establishing social presence) for forming relationships that foster the emotional response. We will be adopting this definition of affective engagement for our study. Examples of institutional activities that promote affective engagement are supportive emails and course communications, flexibility in deadlines and schedules, accessibility to teachers through email, LMS chat features, and student access to personal contact information (Vermeulen & Volman, 2024). Specifically at LAPU, we also integrate emotional assignments, such as personal reflections, application of a Christian worldview perspective, and weekly interactions with colleagues in the discussion forums. We will be evaluating affective engagement by assessing the students’ feelings of engagement through the end of the course survey.
Taken together, existing engagement models converge on a tripartite structure—behavioral, cognitive, and affective engagement—that highlights complementary but distinct aspects of how students participate in learning (Fredricks et al., 2004; Bond et al., 2020). Behavioral approaches emphasize observable participation (e.g., attendance, time-on-task), cognitive models foreground mental effort and strategy use, and affective models focus on emotions, belonging, and perceived support. Each strand has strengths (for example, behavioral indicators are relatively easy to capture in learning analytics) but also limitations (behavioral metrics can miss deeper processing; cognitive and affective indicators often rely on self-report and are sensitive to course context), so no single model fully captures students’ experiences in asynchronous online courses.
Within this framework, course-embedded AI assistants such as Spark can be conceptualized as tools that potentially intersect with all three dimensions: they may prompt behavioral engagement (e.g., more frequent log-ins or check-ins), scaffold cognitive engagement (e.g., clarifying concepts, supporting elaboration), and influence affective engagement (e.g., perceptions of support, reduced feelings of isolation). However, prior work has rarely examined how such tools map onto established engagement models in fully asynchronous settings, and little is known about which dimensions, if any, show measurable change when AI assistants are introduced. The present study addresses this gap by operationalizing behavioral, cognitive, and affective engagement using institutionally derived measures and examining how Spark usage relates to each dimension.
While previous research extensively covers engagement strategies in online learning environments, there is limited direct research on the integration of generative AI tools in the asynchronous setting. Vermeulen and Volman’s research continues to build upon the elements that impact engagement in online asynchronous learning. More specifically, their research assesses engagement in three dimensions: behavioral, affective, and cognitive. While their research emphasizes factors of activities and measurements of engagement, it does not explicitly address generative AI tools. We will build upon those dimensions by assessing the integration of AI tools in the course activities to promote cognitive, affective, and behavioral engagement between the student and the course content. The goal is to determine how AI can be used to minimize disengagement and improve students’ outcomes as they engage in the various course activities related to these dimensions.
Generative AI Tools in Engagement
There are a multitude of generative AI tools now available for use in education, which not only expand the learning opportunities for students but also offer various levels of support (Tsz Kit Ng et al., 2025). Generative AI tools can be used to answer questions, explain topics, provide feedback, gather information, and serve as thought partners for students (Roberto, 2024). These tools can be trained to be virtual tutors that can assess the learner’s level of comprehension of a topic, and then adapt the learning content to fit the needs of the learner (Su & Yang, 2023). Hanshaw et al. (2024) argues that “the use of AI in higher education has evolved from adaptive learning platforms to more sophisticated tools like AI course assistants, which can support students by offering guidance, answering questions, and facilitating learning through Socratic methods.” Due to the Socratic methods that these tools model, deeper learning can be achieved by students as they critically consider both their inputs and outputs as they engage with the tools (Hanshaw et al., 2024).
As AI tools have become more popular and prominent in educational settings, they have changed many aspects of the student experience: including but not limited to assignment types, content development, and interaction within an LMS. The UK Department for Education (2024) has found that students report increased engagement with AI tools and content. The study also highlighted how new types of creativity are developed when students engage with AI tools. As instructors integrate new technologies into their classrooms, they also report a pleasant learning atmosphere, greater motivation among learners, and the ability to evaluate the competence of students (Nguyen et al., 2022). AI literacy is another component of education that instructors and students must develop, as AI is more heavily utilized in the classroom. In order to do so, there must be training at a high level to ensure all parties understand the use cases, abilities, and social impacts of utilizing AI in education (Walter, 2024).
Whether or not students accept and utilize AI chatbots and other tools may depend on their background. Stohr et al. (2024) found that male students in technology and engineering degree programs had more positive views of AI chatbots than their female counterparts in medical or humanities degree programs. These differences suggest that adoption patterns may be influenced not only by discipline but also by demographic factors, which can shape perceptions of chatbot utility and relevance. Despite this, over one-third of students surveyed utilize ChatGPT for educational purposes, and the use of other chatbot technologies was minimal (Stohr et al., 2024). Specifically to chatbots, Gokcearslan et al. (2024) found that students experienced an increased motivation to learn and develop language skills when interacting with the course-embedded chatbot. Furthermore, Luan & Wolff (2025) found that STEM disciplines benefit most from chatbots, and text-based interactions show the most effectiveness in enhancing learner performance. It was also found that the type of chatbot impacted the effectiveness of the tool in the learning environment (Luan & Wolff, 2025). Shoufan 2023 found that students who are not trained in evaluating the responses from chatbots are not as successful as students who learn this skill, which points to a need for more student training in the use of chatbots in education.
Concerns
Lim et al. (2023) and Lee et al. (2024) note that generative AI in the education sphere provides an incredible opportunity for educators to reimagine what learning looks like, despite the notable fears of cheating, academic integrity, and plagiarism that accompany it. Realistically, generative AI tools are not perfect. Generative AI tools have also been known to hallucinate, which involves the tool purposefully generating an incorrect or fabricated response (Tysver & Bit Law, n.d.). Generative AI tools, such as ChatGPT, can create fraudulent, self-generated articles or other sources to provide a sense of credibility for their own responses, which leads to confusion and incorrect learning (Majovsky et al., 2023).
Another major concern is whether or not students or instructors are able to use AI tools easily and ethically (Sammel et al., 2014). These tools can easily and quickly complete work for students, which can include plagiarism and is considered cheating. Questions about originality can lead to uncertainty around the assessment of mastery (Gokcearslan et al., 2024). The success of the use of AI tools in the classroom depends on a few variables, such as prior learning, AI literacy, and promotion of the tools by instructors. However, with user-friendly interfaces and streamlined deployment in the classroom, challenges can be overcome (O’Dea et al., 2024). Achieving streamlined deployment depends on the tool itself, the instructors’ support of the tool, their training, and overall support they receive from the administration of the institution. This also depends on many variables, such as funding, technology availability, and the reliability of the technology (Eiland & Todd, 2019; Brooks & Springer, 2023). Brooks and Springer (2023) also argue that there is an urgent need to develop technology policies within educational institutions to promote equitable learning and close learning gaps for students. Establishing such policies is particularly important as institutions explore ways to harness generative AI tools, ensuring that their implementation maximizes benefits while mitigating risks. Despite these valid concerns, the use of generative AI tools in education can lead to active learning and greater engagement within asynchronous courses.
Course Assistants at LAPU
In 2024, Los Angeles Pacific University partnered with Nectir AI, an educational technology company, to deploy “customizable, FERPA-compliant AI Assistants that live within the LMS” (Nectir.io, 2025). Nectir course assistants (chatbots) are unique in that they are customizable and trainable based on the content of the specified course. The chatbot will learn the course content including assignments, discussion forums, course activities, and textbook content. This enables it to answer students’ questions regarding course content. The course assistant can also access the internet, which allows them to provide more in-depth answers to the responses. Notably, they can be trained to serve as a partner or study buddy, and can be limited to providing feedback, expanding on topical knowledge, or answering questions related to the course. The course assistants are not designed to complete assignments for students, but to be used as a tool to help students expand their own learning. They can also be trained to serve a specific purpose, such as acting as a professional from the respective field for students to interview, or as a branching scenario platform. Additionally, “Nectir AI’s conversational tutors engage students in dynamic, back-and-forth dialogues, asking questions to assess their understanding and providing personalized explanations based on their responses (Ghai, 2024),” which leads to more personalized learning and greater engagement with the course content. These course assistants provide an excellent opportunity for engagement with students as they bring the content to life and lead to more engaging and personalized learning experiences.
In 2024, LAPU launched course assistants in every course. Ghai, 2024 stated, “After just one term of using Nectir AI, students and faculty at Los Angeles Pacific University reported a 20% increase in GPAs, a 13% rise in average final scores, a 36% boost in motivation, and a 13% improvement in self-efficacy”. These assistants had an incredible effect on the overall learning experience, but more specifically, on student learning outcomes, motivation, and engagement. Since then, LAPU has been able to gather more data that will be used for the purposes of our research.
Research Questions and Hypothesis
Research Questions
This study investigates how generative AI tools, specifically AI-powered course assistants, influence student engagement and academic outcomes in asynchronous online courses. Engagement is examined across three domains: cognitive, affective, and behavioral. Given the increasing integration of AI in higher education, this study aims to assess both the immediate and longitudinal impacts of AI tool usage on students’ learning experiences and performance.
Research Questions
RQ1: What is the relationship between student use of the AI course assistant (Spark) and their engagement in asynchronous online courses, as measured across cognitive, affective, and behavioral dimensions?
RQ2: How does the frequency of Spark usage relate to student academic performance, as measured by final course GPA?
RQ3: Do students who use Spark report more positive perceptions of support, motivation, and course clarity compared to those who do not?
Hypothesis
Students who engage with the generative AI course assistant (Spark) will demonstrate higher levels of engagement and academic performance than students who do not, with the strongest effects emerging over time as integration becomes more consistent and students gain familiarity with the tool.
Methods
Research Design
This study employs a mixed-methods, quasi-experimental design to evaluate the relationship between Spark usage and student engagement across cognitive, affective, and behavioral dimensions (RQ1), as well as the association between Spark usage frequency and academic performance (RQ2). We also examine whether Spark users report more positive perceptions of support and course clarity (RQ3). The study follows a quasi-experimental design, comparing student engagement metrics before and after the implementation of AI tools.
Participants
Los Angeles Pacific University (LAPU) is a completely asynchronous, fully online Christian university. Students do not have to follow the Christian faith to attend the university. LAPU serves both graduate and undergraduate students. The university is a Hispanic-serving institution (HSI) with approximately 42.6% of students identifying as Hispanic or Latino (Hispanic Association of Colleges and Universities, 2024).
The gender distribution at LAPU is notably imbalanced, with a female-to-male ratio of approximately 4.32 to 1, indicating a predominantly female student population.
Data gathering and use were approved through LAPU’s Institutional Review Board (IRB). Data was gathered from 1335 unique students and 2088 unique student-course combinations. Multiple students were enrolled in more than one course for the term of the study.
Interventions
The AI tool deployed in every LAPU course is a Nectir AI course assistant or chatbot, named Spark, which is trained specifically on the content of that course (including the course’s syllabus, assignments, discussion forums, and textbook content). The course assistant is able to answer questions, provide feedback on student work, guide the students through the course content, and answer administrative questions–such as what assignment the student should work on first per the course schedule. The course assistant is embedded at the top level of the course homepage, where students are prompted to interact with it.
Key functionalities include:
Discussion support: AI prompts students to engage in discussions, offering feedback and asking follow-up questions.
Personalized feedback: AI provides personalized feedback on assignments to support student learning.
Administrative assistance: AI assistants guide students through course materials, answering logistical and academic questions.
Interactive learning support: AI assistants act as a study partner, helping students deepen their understanding through Socratic questioning.
Data Collection
Qualitative and quantitative data were obtained from the end-of-course survey. Quantitative data regarding the student’s success (GPA in the course) was obtained from the LMS. Chatbot engagement metrics were also obtained from the LMS, which includes specific tool use counts from each student. Please note–the terms chatbot, course assistant, and Spark are used interchangeably for the purposes of this research study.
Spark was made available as an optional course assistant within all participating asynchronous online courses; students could choose whether and how often to use it, and no course policies required its use.
Because Spark was an optional course assistant rather than a mandated instructional component, students self-selected into usage based on their own preferences, needs, and comfort with technology. This voluntary adoption introduces the possibility that Spark users differed systematically from non-users (e.g., in motivation, time-on-task, or help-seeking tendencies), which may also influence engagement and performance outcomes. As a result, the observed associations between Spark usage and engagement or GPA should be interpreted as correlational rather than strictly causal.
Data Collection Procedure
Data for this study were drawn from routine institutional processes at the end of each 8-week term. First, students completed the standard end-of-course survey, which included Likert-scale items on motivation, engagement, perceived support, course organization, and clarity, as well as open-ended questions about encouragement and overall course experience. Second, final course grades were exported from the LMS and converted to GPA values on a 4.0 scale. Third, Spark usage data (e.g., message counts, conversation counts, and self-reported usage) were extracted from the LMS and vendor logs and merged with survey and GPA records at the student-course level. All data were de-identified prior to analysis in accordance with IRB approval.
Data Analysis
We are analyzing the data we obtained to see if the AI-powered chatbot deployed across all university courses has an impact on student engagement. We are assessing engagement from the dimensions of cognitive engagement, behavioral engagement, and affective engagement.
This study employed a quasi-experimental, observational design without a randomized control group, comparing students who elected to use Spark with those who did not. Accordingly, all analyses are correlational and do not support strong causal claims about the effects of Spark on engagement or performance.
Cognitive Domain
From the cognitive domain, we are assessing the correlation between a student’s expected GPA and actual GPA in the course to determine if their expectations of academic performance were influenced by using the tool through a Pearson product-moment correlation. These analyses address RQ1 by examining how Spark usage correlates with cognitive engagement as reflected in GPA alignment and perceptions of course structure.
Expected GPA was obtained from a standard end-of-course survey item asking students, “What final grade (on a 4.0 GPA scale) do you expect to earn in this course?” Students entered a single numeric value from 0.0 to 4.0, which we used as their expected GPA. Actual GPA was drawn from the LMS and recorded on the same 4.0 scale, allowing us to compute the correlation between expected and actual course GPA.
The second analysis we conducted was a Pearson correlation analysis to determine a link between perceived support and course organization with the use of the chatbot. This allows us to assess whether students access the tool more or less based on how much support they receive from other sources (instructors, success coaches, etc.), and whether or not they feel as though they are comprehending the course material. This is an essential area where the course assistant can provide support because it is trained on the course content and course activities.
Further comparing the students’ experiences, we conducted an Independent t-test to determine if there was any significance between a student’s experience in the course, and whether or not the student used the chatbot.
Four Likert-scale items (Q9–Q12) assessed perceived advisement, course organization, relevance, and clarity (1 = strongly disagree to 5 = strongly agree), and together they demonstrated good internal consistency (Cronbach’s α = 0.88), though results are reported at the item level.
Affective Domain
Open-ended responses were analyzed using TextBlob, a lightweight Python-based natural language processing library that computes polarity scores on a continuous scale from –1.0 (very negative) to 1.0 (very positive). TextBlob was selected because it provides a transparent, reproducible lexicon-based approach that can be implemented consistently across large numbers of short survey responses without requiring model training. At the same time, lexicon-based sentiment analysis has well-known limitations (e.g., reduced sensitivity to context, sarcasm, and domain-specific language), so the resulting polarity scores should be interpreted as approximate indicators of emotional valence rather than fine-grained measures of affect.
More sophisticated, transformer-based sentiment models or custom, domain-tuned classifiers could offer greater nuance in future work, but were beyond the scope and infrastructure of the present institutional study.
To address RQ1 in the affective engagement domain, we analyzed student sentiment from open-ended survey responses to evaluate whether those who used Spark reported more positive emotional experiences related to encouragement and support. From the affective domain, we conducted a sentiment analysis on a group of open-ended questions in the survey to determine student feelings of encouragement and engagement within the courses. We then compared those results, using an independent sample t-test, to determine if there was a correlation between the results and the use of the course assistant.
Behavioral Domain
To examine RQ1 from the behavioral engagement perspective, we assessed the relationship between Spark usage and Likert-scale survey items related to motivation, participation, and overall course engagement, as well as longitudinal trends in these scores across academic terms. Behavioral engagement was assessed with three Likert-scale items (Q1: “I felt motivated to engage in learning activities,” Q3: “I felt encouraged and supported in this course,” Q5: “The course kept me engaged”), each rated from 1 (strongly disagree) to 5 (strongly agree). These items demonstrated excellent internal consistency in the present sample (Cronbach’s α = 0.94). From the behavioral domain, we conducted a correlational and inferential analysis to determine if there was any correlation between the use of the course assistant and student engagement. The full text of all engagement-related survey items is provided in Appendix A, along with descriptive statistics.
Spark usage was captured in two ways. For the longitudinal engagement analyses, we used a self-report survey item (Q46) asking students how many times they used Spark during the term; responses were dichotomized into “Spark users” (Q46 > 0) and “non-users.” For the GPA tier analysis, we relied on system-logged counts of Spark messages and conversations from the LMS/vendor logs, which provided an objective indicator of usage frequency. These two measures are related but not identical, and we interpret findings from each in light of their respective strengths and limitations.
Findings and Discussion
Cognitive Domain
Correlation Between Expected and Actual GPA
A Pearson product-moment correlation was conducted comparing expected GPA and actual course GPA to examine whether students who used Spark, the AI course assistant, had more aligned expectations with their academic performance. Analyses were conducted separately for students who used Spark and those who did not.
For students who used Spark (n = 343), there was a strong, positive correlation between expected GPA and actual course GPA, r(341) = .77, p < .001. Similarly, for students who did not use Spark (n = 1745), there was also a strong, positive correlation, r(1743) = .78, p < .001. These results indicate that students, regardless of Spark usage, demonstrated a high degree of accuracy in predicting their final grades.
Correlation Between Survey Dimensions of Course Experience
To better understand how course experiences clustered together for students who used Spark versus those who did not, Pearson correlation analyses were conducted on four survey items:
Q9: I received an adequate level of academic advisement from my success coach.
Q10: The organization of the course content is intuitive.
Q11: I learned relevant information in this class that I can apply to my life.
Q12: Instructions for this course were clearly stated and easily understood.
We found no statistically significant differences between Spark users and non-users on items assessing perceived advisement, course organization, relevance, or clarity; mean scores were high in both groups. This suggests that Spark did not substantially alter students’ global perceptions of course quality, which may already have been strong.
Among students who used Spark, moderate correlations were found between the items, with the strongest correlation occurring between Q10 and Q12 (r = .69). All other relationships ranged from r = .50 to r = .65, suggesting a coherent but slightly more differentiated perception of course quality and support (see Figure 1).
Figure 1
Correlation Matrix for Students Who Used Spark.
Note. Cells display Pearson correlation coefficients among four course-experience items: Q9 (advisement), Q10 (course organization), Q11 (relevance), and Q12 (clarity of instructions). Higher values indicate stronger positive associations between items. Scores are based on a 5-point Likert scale (1 = strongly disagree, 5 = strongly agree). The full wording and descriptive statistics for these items are provided in Appendix A; together they demonstrated good internal consistency (Cronbach’s α = 0.88).
Among students who did not use Spark, correlations were consistently stronger across all items, ranging from r = .57 to r = .79. The highest correlation was again between Q10 and Q12 (r = .79), indicating that course organization and clarity of instructions were particularly interconnected for this group (see Figure 2).
Figure 2
Correlation Matrix for Students Who Did Not Use Spark.
Note. Cells display Pearson correlation coefficients among four course-experience items: Q9 (advisement), Q10 (course organization), Q11 (relevance), and Q12 (clarity of instructions). Higher values indicate stronger positive associations between items. Scores are based on a 5-point Likert scale (1 = strongly disagree, 5 = strongly agree). The full wording and descriptive statistics for these items are provided in Appendix A; together they demonstrated good internal consistency (Cronbach’s α = 0.88).
These findings suggest that students’ perceptions of advisement, organization, relevance, and clarity were more clearly associated for those who did not use Spark, while Spark users may have had a more nuanced view of their course experience.
Survey-Based Course Experience Comparison by Spark Usage
To further evaluate student engagement and course experience, responses to four key survey items were compared between students who used the AI course assistant (Spark) and those who did not. The survey items assessed perceived support (Q9), course organization (Q10), relevance of learning (Q11), and clarity of instruction (Q12). Independent samples t-tests were conducted to compare mean responses across the two groups.
There were no statistically significant differences in responses between the two groups for any of the items:
Q9 (I received an adequate level of academic advisement from my success coach): t(df ≈ 483) = –1.47, p = .141
Q10 (The organization of the course content is intuitive): t(df ≈ 483) = –0.70, p = .486
Q11 (I learned relevant information in this class that I can apply to my life): t(df ≈ 483) = –0.14, p = .892
Q12 (Instructions for this course were clearly stated and easily understood): t(df ≈ 483) = –1.43, p = .15
Mean scores for all items were high across both groups, indicating overall positive experiences. The group that did not use Spark reported slightly higher mean ratings across all four items (e.g., M = 4.48 for Q9 vs. M = 4.39 for Spark users), but these differences were not statistically significant.
These findings suggest that students’ perceptions of course support, structure, and relevance were consistently high regardless of Spark usage, indicating that Spark integration did not negatively impact traditional areas of student satisfaction.
Affective Domain
To further examine student perceptions of the course experience, a sentiment analysis was conducted on three open-ended survey questions (Questions 2, 4, and 6). These items prompted students to reflect on sources of encouragement and overall course support. Responses were concatenated into a single text entry per student and were analyzed using TextBlob, a natural language processing tool that computes sentiment polarity scores ranging from –1.0 (very negative) to +1.0 (very positive).
Students who used the Spark AI course assistant demonstrated slightly more positive sentiment (M = 0.038, SD = 0.13) than students who did not use the assistant (M = 0.019, SD = 0.12). An independent-sample t-test indicated that this difference was statistically significant, t(432.6) = 2.88, p = .004. The sentiment analysis revealed a statistically significant but very small difference in polarity scores between Spark users (M = 0.038) and non-users (M = 0.019). Given the modest absolute difference in means, this effect is unlikely to represent a large practical shift in students’ emotional experience and should be interpreted as a subtle, directional indication of more positive language among Spark users rather than a substantial change in affect.
These findings suggest that students who engaged with Spark were more likely to describe their course experience in emotionally positive terms. This aligns with the hypothesis that generative AI tools can serve as a source of academic encouragement, and may help foster a more supportive learning environment in asynchronous courses.
Behavioral Domain
Quantitative Analysis of Likert-Scale Survey Items and Spark Usage
To assess the relationship between student engagement and use of Spark, we conducted a correlational and inferential analysis using three Likert-scale survey items (Questions 1, 3, and 5). These questions were presumed to reflect cognitive and affective aspects of student engagement.
First, Pearson correlation coefficients were computed among the three items and Spark usage (coded as 1 = used Spark, 0 = did not use Spark). Results indicated strong positive correlations among the survey items: Question 1 and Question 3 (r = .81), Question 1 and Question 5 (r = .87), and Question 3 and Question 5 (r = .81), suggesting that the items are measuring related constructs. In contrast, correlations between Spark usage and the survey items were weak and negative: Question 1 (r = –.06), Question 3 (r = –.07), and Question 5 (r = –.05), indicating no linear association between Spark use and engagement ratings.
To further examine group differences, independent-sample t-tests were conducted to compare mean Likert responses between students who used Spark and those who did not. The results revealed a statistically significant difference for Question 3, t(396.2) = –2.09, p = .037, with Spark users reporting slightly lower scores. No significant differences were observed for Question 1, t(410.4) = –1.78, p = .076, or Question 5, t(425.8) = –1.45, p = .149.
These findings suggest that while the items themselves are closely related and appear to reflect a coherent dimension of engagement, Spark usage was not positively associated with higher engagement ratings on these items. The slight negative relationship on Question 3 warrants further investigation, and may be influenced by contextual or instructional variables not captured in this analysis.
Amount of Spark Use
To compare overall academic performance between Spark users and non-users, we conducted an independent-samples t-test on course GPA. Spark users achieved a statistically significantly higher mean GPA (M = 3.32, SD = 1.05, n = 341) than non-users (M = 3.04, SD = 1.23, n = 1,743), t = 4.39, p < .001, with a small effect size (Cohen’s d = 0.23; mean difference = 0.28 GPA points on a 4.0 scale). These findings indicate a modest GPA advantage for students who used Spark at least once during the term.
To address RQ2, we examined the relationship between the frequency of Spark use and final course GPA using both continuous and categorical indicators. Initial correlation analyses with continuous usage metrics (message count and conversation count) revealed weak linear relationships with GPA (r = .03 and r = .01, respectively). For exploratory purposes, we also segmented students into Low, Medium, and High engagement tiers based on tertiles of their Spark message counts. A one way ANOVA comparing GPA across these tiers showed no statistically significant differences, F(2,409) = 1.41,p = .24,η2 = .01, although mean GPAs followed a modest upward trend from the Low tier (M = 3.15, n = 137) to the Medium tier (M = 3.18, n = 138) and the High tier (M = 3.36, n = 137). Given that the tiers are sample specific and data driven, we interpret these findings as exploratory and descriptive rather than as evidence of a strong, categorical effect of Spark usage on academic performance. Taken together with the non-significant ANOVA across Low, Medium, and High usage tiers, these results suggest that simply being a Spark user is associated with a small GPA advantage, whereas increasing the amount of use does not yield a statistically reliable additional benefit in academic performance.
It is important to note that our longitudinal engagement analysis used self-reported Spark usage, whereas the tiered GPA analysis used system-logged message counts. Self-report may be subject to recall or social-desirability bias, while log data provide more precise frequency estimates but do not capture students’ perceptions of their own usage. These differences in measurement may contribute to discrepancies in the observed patterns across analyses and should be considered when interpreting the robustness of our findings.
While differences between the Low and Medium tiers were marginal, students with sustained interaction with Spark, both in message volume and conversational depth, demonstrated a clearer advantage in academic outcomes. This suggests that frequent and consistent engagement with Spark may play a role in reinforcing understanding and supporting higher achievement, especially when students are active participants in these AI-mediated learning interactions. Although mean GPAs were highest in the High-usage tier, the omnibus ANOVA was not statistically significant and the effect size was small, so the tiered analysis should be interpreted as an exploratory pattern rather than a definitive impact of Spark on grades (see Figure 3).
Figure 3
Average GPA by Spark Engagement Tier.
Note. Low, Medium, and High Spark-usage tiers are based on tertiles of Spark message counts; bars display mean course GPA on a 4.0 scale, and error bars represent standard errors. A one-way ANOVA indicated no statistically significant differences in GPA across tiers, F(2, 409) = 1.41, p = .24, η2 = .01, although mean GPAs increased from the Low (M = 3.15, n = 137) and Medium (M = 3.18, n = 138) tiers to the High tier (M = 3.36, n = 137). These tier-based results are interpreted as exploratory and descriptive rather than as evidence of a strong categorical effect of Spark usage on performance.
Furthermore, an analysis across courses revealed that the average GPA and Spark usage varied widely by discipline–indicating potential differences in how effectively Spark was integrated or utilized in each course. Some courses with higher Spark engagement corresponded with higher GPAs, while in others, higher usage may have been a compensatory response to course difficulty.
These findings point to the importance of both implementation strategy and student engagement behavior when deploying AI tools in asynchronous online courses. While the mere availability of Spark does not guarantee improved outcomes, strategic use–especially through multiple interactions over time–appears to contribute meaningfully to student performance.
Temporal Shifts in Engagement Across Terms
To examine how student engagement evolved over time with Spark usage, we conducted a longitudinal analysis of three Likert-scale engagement items—Question 1 (“I felt motivated to engage in learning activities”), Question 3 (“I felt encouraged and supported in this course”), and Question 5 (“The course kept me engaged”)—across seven academic terms from Spring 2024 through Spring 2025. Students were grouped based on self-reported Spark usage (Question 46), with Spark users defined as those reporting one or more uses.
As shown in Figure 4, during the initial phases of Spark deployment (Spring and Summer 2024), there were minimal differences in engagement scores between Spark users and non-users. In some cases (e.g., Q5 in SP22024), non-users reported slightly higher scores. However, beginning in Fall 2024 (FA12024 and FA22024), students who used Spark began to report consistently higher engagement scores across all three items.
Figure 4
Average Engagement Scores (Q1, Q3, Q5) by Term and Spark Use.
Note. Scores are based on a 5-point Likert scale (1 = strongly disagree, 5 = strongly agree). Spark usage is self-reported (Q46 > 0). The data show a growing divergence over time, with Spark users reporting consistently higher engagement starting in Fall 2024.
This trend continued into Spring 2025, where Spark users reported their highest overall scores in the longitudinal dataset, particularly on Question 3, which asks students to rate how supported they felt in the course. The increasing spread between groups over time suggests that Spark’s impact on behavioral and affective engagement intensified as its integration matured. These results imply a temporal effect: Spark’s influence was not immediate. Rather, results accumulated as instructors iteratively refined its use, students became more familiar with its capabilities, and the institution aligned Spark usage more closely with course design and support structures. What began as a minimally viable feature became a pedagogically impactful tool associated with higher engagement scores by Spring 2025.
Discussion
Interpretation of Findings
Cognitive Domain
In the cognitive domain, students who used Spark and those who did not both showed strong correlations between expected and actual GPA, indicating accurate self-assessment across groups. Importantly, Spark usage did not confer an additional advantage in this alignment, suggesting that students’ grade expectations were well calibrated regardless of whether they engaged with the AI assistant.
To build upon this finding, we analyzed student perceptions of engagement and support correlated to the use of the course assistant. The moderate correlation between the use of the tool and lack of support suggests that students who have feelings of confusion about course organization and instructions, disengagement, or lack of support may engage with Spark more often. This pattern is consistent with the idea that students who feel less supported or more confused about course organization may be more likely to turn to Spark for clarification and guidance, although our correlational design does not allow us to determine whether Spark directly resolves these challenges.
Sinatra et al. (2015) argued that cognitive engagement requires students to go beyond the requirement of the activity (p. 3). These definitions provide important insights, but the qualifiers that the learner comprehends “complex ideas” and goes “beyond the minimal requirements” limits possible manifestations of cognitive engagement, making such efforts dependent on both the type of learning activity students are assigned and the expectations that instructors place on their work (Sinatra et al., 2015). This suggests that in order for students to reach cognitive engagement, they must comprehensively understand the requirements of the course activities. AI tools can serve as a means to obtain that clarification. According to this study, when students needed further clarity on instructions or assignments, they accessed the course assistant for help. This is important to note, because it reinforces the need for additional support tools for students in the asynchronous online learning environment. For these measures, there is a higher degree of connectivity between a student’s need for additional support and their decision to utilize the generative AI tool for further assistance.
Affective Domain
The sentiment analysis of student responses signals a small but significant difference in sentiment scores between Spark users (M = 0.038) and non-users (M = 0.019), with Spark users expressing a slightly more positive sentiment. While the effect size is small, it supports the hypothesis that generative AI tools increase students’ feelings of engagement, support, and positivity when engaging with the course content. Additionally, the real-time responses of the course assistant lead to immediate feedback and personalized learning, which in turn can create greater engagement in asynchronous courses where those elements may be lacking.
Our study suggests that students who engaged with Spark were more likely to describe their course experience in emotionally positive terms. This aligns with the hypothesis that generative AI tools can serve as a source of academic encouragement, and may help foster a more supportive learning environment in asynchronous courses. Vermeulen and Volman (2024) found positive measures of affective engagement through the mechanisms of promoting a group feeling, interaction, and creating a sense of empathy and trust among classmates and the instructor. Martin and Borup (2022) found that affective engagement can increase when others in a student’s course community facilitate communication, develop relationships, and instill an excitement for learning. Both of these findings support the results of our study. According to our findings, we can suggest that students engaged with a generative AI course assistant have a positive impact on the students’ affective engagement and feelings of belonging in the course.
Behavioral Domain
The analysis of the Likert-scale survey responses regarding the use of Spark showed no significant association between usage of the tool and feelings of engagement. Although the survey responses regarding engagement all showed positive results, Spark usage did not seem to have an effect on these feelings. However, when examining the data longitudinally across seven academic terms, a clear trend emerged. Beginning in Fall 2024 and continuing through Spring 2025, Spark users reported consistently higher engagement scores across all three behavioral items (Q1, Q3, and Q5) compared to non-users. This temporal shift suggests that the impact of Spark may accumulate over time, particularly as students and instructors become more familiar with its use and as implementation becomes more intentional. Initial implementation effects were limited, but a clear temporal trend of increasing engagement among Spark users emerged as usage matured institutionally. This suggests that AI tools may follow a delayed efficacy curve, where meaningful outcomes only emerge after sustained exposure, refinement, and integration into instructional workflows.
Interestingly, students in the high engagement tier who utilized Spark frequently throughout the term achieved slightly higher GPAs (M = 3.36 GPA) than the students in the Medium (M = 3.18) or Low (M = 3.15) engagement tiers. Although overall engagement survey scores were not significantly impacted by the use of the tool, it indicates that higher levels of interactions may still lead to improved academic outcomes and GPAs.
Several alternative explanations may help account for the mixed and, at times, null findings in the behavioral engagement measures. One possibility is a measurement mismatch: our behavioral indicators (e.g., self-reported survey items and LMS activity proxies) may not have captured the specific micro-behaviors that Spark most strongly influences, such as just-in-time help-seeking or brief check-ins while working on assignments. It is also possible that students primarily used Spark to clarify concepts or receive feedback without substantially changing the frequency or timing of their log-ins and submissions, which would dampen observable differences in behavioral metrics. In addition, inconsistent instructor messaging and variability in how strongly Spark was integrated into course activities across sections may have led to uneven adoption, thereby diluting potential effects at the aggregate level. Finally, some students may have perceived Spark as a compensatory tool to “catch up” when they fell behind, which could offset potential gains in behavioral engagement when comparing users and non-users.
Impact of Generative AI on Active Learning
Based on the data obtained from our study, Generative AI has an overall positive effect on active learning, specifically in the asynchronous, online environment. As students utilize the AI tool, in our case a course assistant chatbot, they are interacting with a tool that provides more personalized learning opportunities. This ranges from an avenue to ask questions about the content or the course administration, or in more sophisticated interactions even engaging in branching scenarios or interviews. The integration of generative AI in the classroom has overall positive effects on engagement, feelings of motivation and confidence, feelings of course accessibility and organization, and confidence in mastery of the course content. This is further supported by the longitudinal trend observed in our study, where Spark users in later terms (Fall 2024 onward) demonstrated greater gains in behavioral engagement than students in the early phases of deployment. These findings indicate that Spark’s pedagogical potential may not be immediate, but rather intensifies as its use becomes more deeply embedded in course activities and student workflows. While the correlation was significant in these areas, more research is needed to determine the best ways to design and deploy the tool so that students are successful.
Limitations
A key limitation of this study is that Spark usage was voluntary, meaning students chose whether and how frequently to use the assistant. This self-selection may reflect underlying characteristics such as higher motivation, stronger self-regulation, or greater openness to technology, which could also contribute to higher engagement and performance. Consequently, while our findings show meaningful associations between Spark usage and both GPA and affective engagement, they cannot definitively establish that Spark alone caused these improvements. We are limited to the data that we have collected internally. Since LAPU is an online school, we are limited to data from students who experience an asynchronous, online environment. The study could be expanded to include synchronous and in-person environments. Additionally, some courses at LAPU require the use of the course assistant for specific activities (as part of an assignment or other activity). In these instances, students may be more apt to utilize the tool since they are already familiar with it. Another consideration is that instructors who promote the use of the tool may have students who are more willing to use it.
Another consideration of this study is that the level of engagement is based on self-reported data from students. Question 3 of the end-of-course survey is “I felt engaged throughout this course.” Students then respond with a number 1 through 5, where 1 signifies they do not agree, and 5 that they completely agree. Based on the survey data, Question 3 had an average response of 4.55 out of 5. This suggests a high level of engagement in the courses, without incorporating the analysis of the course assistant use. This is important to note because, on average, students feel as though they are highly engaged in their courses. To further determine the impact of course assistants, data could be compared historically to determine student engagement across multiple terms with and without the tools.
Future Research
We suggest that further research be conducted on generative AI in the asynchronous online learning environment, specifically on course assistants as their capabilities evolve. Students will learn to interact with these tools in new ways as the technology becomes more advanced. Further research could also explore students’ willingness to engage with the tool based on different instructional strategies or instructor promotion.
Conclusion
This study examined the role of Spark, an AI-powered course assistant, in asynchronous online courses at LAPU. Overall, differences between Spark users and non-users on GPA and survey-based engagement were small and often non-significant, indicating that Spark should be viewed as a complementary support rather than a primary driver of student outcomes. Overall, differences between Spark users and non-users on GPA and survey-based engagement were small and often non-significant, indicating that Spark should be viewed as a complementary support rather than a primary driver of student outcomes.
Quantitative analyses showed that students who used Spark tended to have slightly higher GPAs and, in later terms as implementation matured, reported somewhat greater behavioral engagement, though effect sizes were modest and the tiered GPA differences were not statistically significant. Sentiment analysis provided converging, but subtle, evidence that Spark users described their course experiences in more positive emotional terms, suggesting a small directional association with perceived support and motivation rather than a large practical shift in affect.
These findings imply that course-embedded AI assistants like Spark may offer incremental benefits to students’ academic and emotional experiences in asynchronous courses when thoughtfully integrated into course design, but stronger claims about educational impact are not warranted from the present data. As students engaged with Spark, they used it for clarification, feedback, and administrative questions—functions that can help address some challenges of asynchronous learning, even if their measurable effects remain limited in size. Looking ahead, instructors, instructional designers, and institutions will need to experiment with more intentional use cases and continue longitudinal evaluation, ideally with validated engagement measures and more rigorous designs, to better understand when and for whom AI course assistants meaningfully enhance learning and engagement.
Appendices
Appendix A
| COLUMN | QUESTION |
|---|---|
| Question 1 | I felt encouraged throughout the course. |
| Question 2 | I felt encouraged throughout the course. - <b>What made you feel encouraged or a lack of encouragement?</b> |
| Question 3 | I felt engaged throughout the course. |
| Question 4 | I felt engaged throughout the course. - <b>What made you feel engaged or a lack of engagement?</b> |
| Question 5 | I felt supported throughout the course. |
| Question 6 | I felt supported throughout the course. - <b>What made you feel supported or a lack of support? </b> |
| Question 7 | Student support services enabled me to successfully complete the course. |
| Question 8 | Student support services enabled me to successfully complete the course. - Comments: |
| Question 9 | I received an adequate level of academic advisement from my success coach. |
| Question 10 | The organization of the course content is intuitive. |
| Question 11 | I learned relevant information in this class that I can apply to my life. |
| Question 12 | Instructions for this course were clearly stated and easily understood. |
| Question 13 | Instructions for this course were clearly stated and easily understood. - Comments: |
| Question 14 | This course helped me better understand the relationship of a Christian Worldview to the content area of the course. |
| Question 15 | This course helped me better understand the relationship of a Christian Worldview to the content area of the course. - Comments: |
| Question 16 | This course helped me better understand the relationship of a Christian Worldview to my life and work in the world. |
| Question 17 | This course helped me better understand the relationship of a Christian Worldview to my life and work in the world. - Comments: |
| Question 18 | The learning activities and assignments promoted the achievement of the stated learning objectives. |
| Question 19 | The learning activities and assignments promoted the achievement of the stated learning objectives. - Comments: |
| Question 20 | On average, how many hours did you spend on this course (online and offline) per week? |
| Question 21 | Did you adopt an eTextbook for this course? |
| Question 22 | The tools and media supported the course learning objectives. |
| Question 23 | The tools and media supported the course learning objectives. - Comments: |
| Question 24 | My instructor used the grading criteria (rubrics) as a basis for evaluating my work. |
| Question 25 | My instructor used the grading criteria (rubrics) as a basis for evaluating my work. - Comments: |
| Question 26 | My instructor’s interactions made the course material interesting and relevant. |
| Question 27 | My instructor’s interactions made the course material interesting and relevant. - Comments: |
| Question 28 | My instructor’s feedback helped promote active learning. |
| Question 29 | My instructor’s feedback helped promote active learning. - Comments: |
| Question 30 | My instructor responded to my requests for assistance within one business day. |
| Question 31 | My instructor provided feedback on assignments within 5 business days. |
| Question 32 | My instructor demonstrated respect for the students. |
| Question 33 | My instructor demonstrated respect for the students. - Comments: |
| Question 34 | I was able to use the course communication tools effectively to communicate with my instructor. |
| Question 35 | Having the ability to schedule office hours with my instructor made me feel more supported. |
| Question 36 | How likely are you to take advantage of LAPU’s Inspire 10% tuition discount by referring a friend to study at LAPU? |
| Question 37 | Pay it forward. What advice would you give a future student to help them succeed in this course? (optional) |
| Question 38 | Pay it forward. If you found resources that particularly helped you, share them here so we can pass them along. (optional) |
| Question 39 | Additional comments you would like to provide. (optional) |
| Question 40 | Did you use Spark, the AI Course Assistant, while participating in your course? |
| Question 41 | What is the primary reason(s) you did not use Spark within your course? |
| Question 42 | Effectiveness of AI Assistant: The AI course assistant helped me understand course material and concepts. |
| Question 43 | Availability and Responsiveness: I am satisfied with the availability and responsiveness of the AI course assistant within the course. |
| Question 44 | Support and Clarification: The AI course assistant supported me in clarifying doubts and answering questions related to the coursework. |
| Question 45 | Integration with Learning Experience: The AI course assistant enhanced my overall learning experience in my course. |
| Question 46 | Approximately how many times did you utilize the AI course assistant within your course. |
| Question 47 | How many times did you connect with your Success Coach? |
| Question 48 | What is your expected grade outcome for the course? |
| Question 49 | Suggestions for Improvement: What improvements or additional features would you suggest for the AI assistant to support your learning experience better? (optional) |
Data Accessibility Statement
The datasets generated and/or analysed during the current study are available in the OSF repository: https://osf.io/vazfu/?view_only=773c19f7f9ca4b47bdeb0e8711d2f00b.
Sustainable Development Goals (SDGs)
This study is linked to SDG 4: Quality education.
Ethics and Consent
Ethical approval was obtained through the Los Angeles Pacific University IRB.
Acknowledgements
Kristen Denlinger for her insight into the project.
Author Contributions (CRediT)
Frans Flores: Conceptualization, investigation, project administration, methodology, writing-original draft, writing-review and editing, Kenna Norman: Conceptualization, formal analysis, methodology, investigation, writing – original draft, writing review and editing. George Hanshaw: Conceptualization, data curation, formal analysis methodology, writing-original draft, writing – review and editing. All authors have read and agreed to the published version of the manuscript.
Author Notes
Based on Academic Integrity and Transparency in AI-assisted Research and Specification Framework (Bozkurt, 2024), the authors of this paper acknowledges that the paper was reviewed for grammar with the assistance of Grammarly 14.1116.0 complementing the human editorial process. Data analysis in this work were assisted by ChatGPT4o (June 2025). These analyses were later reviewed, revised, and finalized by the authors to accurately represent and report the research data. The human authors critically assessed and validated the content to maintain academic rigor. The authors also assessed and addressed potential biases inherent in the AI-generated content. The final version of the paper is the sole responsibility of the human authors.