The Effects of a Teacher-Curated LMOOC on EFL Students: A Case Study

From MOOCs to Language MOOCs

Massive Open Online Courses (MOOCs) offer researchers an opportunity to contemplate and rethink our educational history and speculate on what the future may entail (Lin & Chang 2014). Originally developed based on Connectivism Theory (Siemens 2005), MOOCs have since incorporated several other learning theories and pedagogical approaches into their design and delivery. Hence, MOOCs are no longer a singular concept. The most common categorisation is, perhaps, to classify them into cMOOCs (Connectivist MOOCs) and xMOOCs (Edx or Extended MOOCs). While cMOOCs adopt a learner-centred pedagogy and emphasise independent navigation of learning resources and creation of personal learning materials, xMOOCs are more structured and content-focused and emphasise knowledge transmission. Pedagogically speaking, xMOOCs hinge on a more teacher-centric philosophy and are able to accommodate a larger number of students. This type of MOOC is now more popular and it is the predominant type offered on major MOOC platforms (Godwin-Jones 2014). However, as technology and pedagogy develop, it is now more difficult to put a given MOOC into a particular classification; therefore, I believe that MOOCs should be positioned as a continuum in a pedagogical dichotomy, rather than viewed as a dichotomy.

In the past decade, the proliferation of MOOCs has significantly increased, prompting extensive discussions on both theoretical and pedagogical aspects of the concept. To this end, various fields of education have recognised the potential of this educational model and harnessed different features of MOOCs to augment teaching and learning. One field that has seen an increase in the use of MOOCs is language education. The term Language Massive Open Online Courses (Language MOOCs or LMOOCs) was coined and defined as “dedicated web-based online courses for second languages with unrestricted access and potentially unlimited participation” (Barcena & Martin-Monje 2014: 1). There are several benefits that MOOCs can offer language learning. First, LMOOCs give access to learners in the contexts where exposure to the target language is limited such as in English as a foreign language (EFL) contexts. This also means that students can have opportunities to interact with learners from all around the world using the target language they are learning. In addition, the flexible nature of MOOCs allows them to be used to serve various purposes by both teachers and learners. Learners can use LMOOCs as independent self-learning materials or as professional development, while teachers can incorporate LMOOCs into their courses or even the whole curriculum.

Despite these benefits, however, there are some notable challenges with using MOOCs for language learning. MOOCs have sometimes been seen as “problematic” for language learning (Barcena & Martin-Monje 2014; Sokolik 2014). This is because the nature of MOOCs is more suited for knowledge transmission and receptive learning. Language learning is, nevertheless, skill-based and requires not only input, but also opportunities to practice in an authentic and meaningful way, something current LMOOCs have not yet successfully provided. Although there was initial hype around the fact that having access to a massive community would encourage learners to interact with one another, the level of interaction in LMOOCs has been quite low, mainly due to the heterogeneous demography of learners and the impersonal nature of the model (Jitpaisarnwattana, Reinders & Darasawang 2021; Martin-Monje, Castrillo & Mañana-Rodriguez 2018). In addition, from a teaching perspective, teachers considering teaching in an LMOOC may be required to change their teaching philosophy and possess facilitation skills, something not many language teachers have (Castrillo 2014; Read & Barcena 2013). The final challenge is related to feedback. It is an arduous task to provide feedback to a large number of students. Although automated feedback to close-ended questions may suffice for other subjects, language development requires comprehensive feedback and ongoing scaffolding.

Using LMOOCs as a Part of a Language Course

Despite early adoption of MOOCs and LMOOCs as stand-alone courses, the last decade has seen a more integrative approach in using LMOOCs as part of a language course/programme. From a pedagogical perspective, LMOOCs are very versatile as they can be used not only as a supplementary for a course, but also to replace a part of a course or even the whole course itself. One way that LMOOCs can be used is through a flipped learning model. Zhang (2017) used self-developed LMOOCs as a flipped learning component of an English course in China. The students were asked to study in LMOOCs prior to coming to the face-to-face classes, where collaborative activities and discussion were emphasised. Using LMOOCs in this way means that they, to a certain degree, serve as a replacement for course content that would traditionally be taught in the class.

In addition to flipped learning, LMOOCs can also be used as a complementary/supplementary element of a given course. Teachers can require their students to take part in an LMOOC as a part of course completion. The degree to which LMOOCs are used in a given course can vary from using them as a participation component to asking students to work with LMOOCs as an assessed assignment. This type of integration is, however, more suitable for a formative or learning-oriented assessment (assessment for learning), rather than a summative one. At the less integrative end of the spectrum, LMOOCs can simply be used as an adjunct to a language course. One or more LMOOCs can be presented to students as additional learning resources that they can study independently. This means that participating in LMOOCs is not a requirement of the language course, rather learners are recommended to explore these LMOOCs for self-learning and development.

Empirical Studies on LMOOCs

In the last decade, there has been a growing number of empirical studies on LMOOCs especially since the Covid-19 pandemic. From a pedagogical perspective, research on LMOOCs has been catagorised into two lines: LMOOCs as an alternative/a replacement of a traditional language course and LMOOCs as an integrative part of a language course (Jitpaisarnwattana, Reinders & Darasawang 2019). In the first line of research, a number of learning-related constructs have been examined. Friðriksdóttir (2018, 2021) investigated retention and engagement patterns in several LMOOCs on the Icelandic language and found that retention rate was relatively low in these courses, with a sharp drop-out rate at the beginning of the course. Time constraint was an important factor that forced many students to drop-out, while gradual and scaffolded presentation of input had a positive impact on retention. The motivation to register for, participate in and complete a MOOC is another important topic of research on LMOOCs. Students enrolled in LMOOCs both for intrinsic (interested in the content) and extrinsic (wanting to develop professionally) reasons (Beaven, Codreanu & Creuze 2014). MOOCs were viewed as useful resources that could equip students with skills necessary for social, economic and geographical mobility. However, a lack of language learning support in MOOCs, especially for those who are not native speakers of the target language, was an important factor that demotivated students (Uchidiuno et al. 2017). Rubio (2014) compared gains in comprehensibility of learners studying Spanish in two modes of delivery: traditional face-to-face and an LMOOC. It was found that although both groups significantly improved their pronunciation comprehensibility, greater gains were evident among the LMOOC learners.

Research looking at the effects of integrating LMOOCs into traditional L2 courses is much more limited. More often than not, the integration is associated with a blended/flipped learning approach. Wang and Wright (2018) studied the use of an LMOOC as a flipped component in a Chinese language course for adult learners. The flipped instruction adopted in the study was able to enhance oral proficiency among the students and led them to have overall positive attitudes toward the course. Similar findings were reported in Zhang (2017) who found that using an LMOOC as a flipped learning component could result in students having positive attitudes toward the course. They also held a perception that their reading and writing skills had improved as a result of engaging with the LMOOC content. Despite such a perception, however, language development was not empirically measured in the study. Luo (2020) integrated a MOOC on learning skills as part of a Spanish course at a university in Taiwan and found a positive change in students’ learning behaviours after participating in the MOOC. More recently, Jitpaisarnwattana and Chalmers (2022) integrated an LMOOC focusing on English grammar into a fundamental English course at a university in Thailand. The LMOOC was utilised as a platform for remedial practice for students who had previously failed their midterm exam. Although the results showed that the majority of the students who took the LMOOC still failed the final exam, their scores statistically significantly improved. The students showed overall positive attitudes toward the LMOOC and believed that their scores had improved as a result of, albeit not entirely, using the LMOOC.

These research findings illustrate the potential of LMOOCs as both a complement to or a replacement for L2 classes and as an integrative technology in these classes. However, most of the studies were exploratory and only a handful of these looked at the actual effects of participating in LMOOCs on language development. I believe this is the area of research that should be substantiated empirically and the current study aims to shed more light on it.

Participants and Research Context

The participants were from a cohort of first-year students at a nursing college in Thailand who were studying a compulsory English course called “English for Daily Use”. The course ran for one semester (15 weeks). There were 179 students in the cohort and all of them agreed to take part in the study. The demographic data collected at the beginning of the course showed that 169 of the participants were female and 10 were male. Their age ranged from 18–20 years old. Most of them had been learning English for more than 10 years prior to taking the course. As for their English language proficiency, 40 students were A1, 105 were A2, 25 were B1 and 9 were B2 based on the internal English proficiency test used across nursing colleges in Thailand (see Table 1). It should be noted that the college did not use proficiency to allocate students into the class; hence, a very diverse range of proficiency among the students.

English for Daily Use is a compulsory course for all first-year students at the college. The course offers a total of 4 hours of class time per week over a 15-week period. The course is divided into two components: lecture (2 hours) and self-study practice (2 hours). The course content is based on a commercial textbook, EVOLVE 2 (Cambridge University Press, A2 level). Each unit in the textbook covers all four language skills with specific grammar points and topic-related vocabulary. Supplementary materials are usually provided. The course assessment comprises four parts: midterm exam (35%), final exam (35%), attendance (10%) and assignments (20%).

The Intervention: The Teacher-Curated LMOOCs

The LMOOC used in this study was curated by the course instructor. Five lessons from four different LMOOCs on ThaiMOOC Platform (https://lms.thaimooc.org/) were selected and curated into an LMOOC task for the students. The reason for using a teacher-curated LMOOC rather than assigning the students to take one LMOOC was to ascertain that the content of the LMOOC was relevant to the content covered in the course in terms of grammar, vocabulary and reading materials. As the term has not been used elsewhere, it is important to clarify it. The term teacher-curated LMOOC refers to the practice of selecting different learning units from different LMOOCs and curating them into a coherent lesson for a specific learning purpose. Although all of the chosen lessons were about English language learning, the lessons were either conducted in Thai or contained Thai subtitles. This was because previous research has shown that, for low proficiency students, first language support is essential in order for them to benefit from taking the LMOOC (Jitpaisarnwattana & Chalmers 2022; Roussel et al. 2017). The curated LMOOC is summarised in Table 2 below:

The LMOOC intervention took place in the second half of the semester after the students had taken the midterm exam. The intervention lasted for 5 weeks (Week 9 to Week 13). Prior to the start of the intervention, the course instructor conducted an introductory session with the students to provide instructions and help them with registration. The students were asked to use the self-study practice (2 hours) to complete the LMOOC lesson. At the end of each week, the students were asked to screen-capture confirmation that they had completed the LMOOC lesson for that week and submit it in the course learning management system. It should be noted that, in the first half of the semester, the students were also assigned to complete several assignments including grammar exercises and vocabulary worksheets and external reading exercises.

The Follow-Up Attitudes Study

At the end of the semester, the attitude questionnaire was sent to all of the students via Google Form. Of the 179 students, 123 students responded to the questionnaire; therefore, only 123 students were included in the follow-up study. The questionnaire consisted of 12 five-point Likert scale items in three categories: English language learning, usefulness for their learning in the course, and learning experience. Three open-ended questions were also included to allow the students to provide more insight into the questions being asked. To confirm the content validity of the questionnaire, it was checked by three specialists in online language learning for item objective congruence (IOC). The analysis yielded a value of 0.929, suggesting acceptable validity.

Five students were invited to attend the semi-structured interviews with the researcher. The selection was based on their responses in the questionnaire to include participants with both positive attitudes and negative attitudes toward the LMOOC intervention. The interviews were conducted online and in Thai. The interviews were transcribed and translated by the researcher. To check for accuracy, the transcripts were back translated by another lecturer at the nursing college.

Data Collection and Analysis

Data from the midterm and final examinations were collected through score reports from the college. Both examinations followed the same test structure, consisting of 70 multiple-choice items and covering three areas of the course content: grammar, vocabulary and reading. The examinations consisted of 40 grammar items, 14 vocabulary items and 16 reading items. The course instructor (who is also the researcher) wrote the test item and each examination was approved by the academic committee at the college. The scores obtained were analysed using the Statistical Package for Social Sciences 28.0 (SPSS). The overall scores and the scores in each component were compared using paired sample t-tests to find out the differences between scores in the midterm and final.

The attitude questionnaire was sent to the students at the end of the course via Google Forms. Of the 179 students, 123 students responded to the questionnaire. Five students were invited to do semi-structured interviews with the researcher. Data obtained from the questionnaire were analysed using fundamental descriptive statistics (frequency, mean scores and standard deviations). The data from the open-ended questions and interview scripts were coded using thematic content analysis. To ensure the reliability of the coding process, two individuals independently coded the responses, the researcher and another English lecturer at the nursing college.

Effects of the Teacher-curated LMOOC on the Students’ Performance

In order to see the effects of the curated LMOOC on the students’ performance, the scores from the midterm and final exams were analysed using paired sample t-tests. The results regarding the overall scores are shown in Table 3 below.

As shown in Table 3, the students had significantly higher overall scores in the final (M = 39.87, SD = 11.14) than in the midterm (M = 35.04, SD = 10.37), with a significant difference at the 0.05 probability level (t (178) = 10.49, p = .000). Therefore, it can be assumed that, after participating in the LMOOC, the students showed an overall better performance in the final exam.

In addition, the scores in the grammar component (40 items) from the midterm and final were analysed using paired sample t-tests. The analysis is shown in Table 4 below:

The students scored significantly better in the grammar component of the final (M = 21.10, SD = 7.29) than in the midterm (M = 17.47, SD = 5.92), with a significant difference at the 0.05 probability level (t (178) = 10.33, p = .000). This indicated statistically different scores between the midterm and the final, suggesting that, after the intervention, the students’ performance regarding grammar improved in the final.

Also, the scores regarding vocabulary (14 items) from the midterm and the final were analysed using paired sample t-tests. Table 5 illustrates the results below:

It can be seen that the students had significantly higher scores on vocabulary in the final (M = 9.87, SD = 2.74) than in the midterm (M = 8.97, SD = 2.54), with a significant difference at the 0.05 probability level (t (178) = 3.95, p = .000). This showed that the score differences were statistically significant, suggesting that the students showed improvements in terms of vocabulary in the final after engaging with the LMOOC.

Finally, the scores concerning reading comprehension (16 items) were analysed using paired sample t-tests. The analysis is shown in Table 6 below:

As indicated, although the scores regarding reading comprehension were higher in the final (M = 8.91, SD = 2.96) than in the midterm (M = 8.61, SD = 3.72), the gain was not statistically significant at the 0.05 probability level (t (178) = 1.14, p = .255). This means that after taking part in the LMOOC, the students’ performance related to reading only marginally improved without having a statistically significant difference.

Effects of the Teacher-curated LMOOC on Different Groups of Students

To examine the effects of the curated LMOOC on different groups, the students were divided into two groups: those who failed the midterm (N = 101) and those who passed (N = 78). The scores from each group were then analysed using paired sample t-tests. The analysis regarding the overall scores of the students who failed the midterm is shown in Table 7:

As seen, the failing students did significantly better in the final (M = 32.66, SD = 6.87) than in the midterm (M = 27.56, SD = 4.71), with a significant difference at the 0.05 probability level (t (100) = 7.54, p = .000). This means that the LMOOC intervention had a positive effect on the lower proficiency students. Also, although many of the students still failed the final exam, they were much closer to the passing baseline (35 points).

Moreover, the overall scores of the students who passed the midterm were analysed using paired sample t-tests. The results of the analysis are illustrated in Table 8:

It can be seen that the passing students also performed much better in the final (M = 49.21, SD = 8.32) than in the midterm (M = 44.73, SD = 7.15), with a significant difference at the 0.05 probability level (t (100) = 7.54, p = .000). Despite the score difference being smaller than the failing group, the difference was still statistically significant. This suggests that participating in the LMOOC intervention had positive effects on both groups of students.

Students’ Attitudes towards the Teacher-curated LMOOC

This section analyses the students’ attitudes toward the curated LMOOC. The data were analysed according to three categories: English language learning, usefulness for their learning in the course, and learning experience.

Four questionnaire items explored the students’ attitudes toward English language learning. As shown in Table 9, the students generally agreed that studying in the LMOOC was beneficial for their English language learning. Most of the students (81.3%) agreed that the curated LMOOC helped them improve their English. A slightly lower proportion of the students (78.9%) reported that they were more confident in their English language learning. A strong level of agreement was also reported regarding the students having better knowledge of English and MOOCs being a good learning tool at 82.9% and 86.2% respectively.

The qualitative data lends support to the quantitative data, in that studying in the curated LMOOC allowed them to know their weaknesses and make necessary improvements. This view was reported in many answers to the open-ended question. For instance, two students mentioned that:

In addition, many of the students felt that the LMOOC was a good tool for learning English, as stated by one student:

Four items in the questionnaire looked at the usefulness of the curated LMOOC for the students’ learning. As seen in Table 10, the majority of the students (79.7%) strongly agreed that participating in the LMOOC was helpful in preparing for their final exam. Participating in the LMOOC was also seen as beneficial for understanding and catching up with the course content, as evident by a similar level of agreement at 83% and 82.1%, respectively. In addition, the students also felt that studying in the MOOC format helped them better understand the content presented to them.

In answers to the open-ended questions and in the interview, one clear theme reported by several students was that participating in the LMOOC helped them catch up with the content they did not understand in the class, as one student said:

Moreover, many students thought that participating in the LMOOC was very useful for preparing for the final exam. For example, two students reported:

Four questionnaire items probed into understanding the students’ learning experience in the curated LMOOC. According to the descriptive statistics shown in Table 11, nearly 75% of the students thought that studying in the LMOOC was interesting. A similar percentage of the students (76.5%) also reported having a good experience learning in the LMOOC. An even greater proportion of the students (87.8%) viewed the content of the curated LMOOC as interesting. However, although many of the students (65.9%) thought that the LMOOC could motivate them to learn English, just over a third of them (34.1%) did not see it as a motivational tool to learn English.

As for the qualitative data, many students felt that the content in the LMOOC was interesting and that the LMOOC was a motivational tool for learning English, as two students stated:

In summary, the students viewed the curated LMOOC as an effective tool for them to improve their English. It was also seen as a useful resource for the students to review what they did not fully understand in the class, as well as to prepare for their final exam. Finally, they expressed an overall positive experience in learning in the LMOOC and thought that learning English in an LMOOC format was interesting and motivating.