Perceived Impact of a Teacher Development MOOC on Self-Determined IEP Implementation

Continuing professional development for teachers

If society aims to generate a diverse multitude of positive outcomes, both for teachers and students alike, teacher CPD would appear to be an imperative driving force in achieving that objective. CPD provides teachers with a chance to explore new methods, to try out innovative approaches and to use more effective classroom strategies, thereby contributing to an ongoing developmental progression in professional practices (Basma & Savage, 2018; Darling-Hammond et al., 2017; Gracheva et al., 2020; Lynch et al., 2019). Furthermore, literature suggests that involvement in the CPD process tends to increase teachers’ sense of self-efficacy (Omare, 2021; Saadati et al., 2023) and have positive effects on teacher retention (Coldwell, 2017; Scott et al., 2021; Smet, 2021; Toropova et al., 2021). Teachers’ partaking in CPD translates into a richer and more stimulating educational experience for students, leading to more positive academic outcomes, higher motivation, greater engagement and hence, overall, a more successful schooling experience (Basma & Savage, 2018; Fletcher-Wood & Zuccollo, 2020; Kraft et al., 2018; Lynch et al., 2019; Yoon et al., 2007).

A plethora of continuing professional development activities are available in a range of formats – continuing education or university credit courses; seminars, conferences, and workshops; mentoring and professional learning communities – providing teachers with plenty of opportunities to further their job competency levels (Boulay, 2021; Gaudreau et al., 2021). Each of these CPD delivery methods nonetheless comes with a set of obstacles. Meta-analysis would seem to indicate that there are considerable discrepancies in CPD outcomes, dependent upon the way CPD is designed and delivered (Basma & Savage, 2018; Didion et al., 2020; Kraft et al., 2018; Lynch et al., 2019). Obstacles that reportedly hinder full participation centre around resources teachers tend to lack: time, proximity, substitute teachers, organizational support, and financial resources (Buczynski & Hansen, 2010; Darling-Hammond et al., 2017; Opfer & Pedder, 2011; Tooley & Connally, 2016). This speaks to the importance of more consciously devising CPD exercises that align with the professional realities experienced out in the field, in order to remove some known barriers and achieve higher attendance.

Massive open online courses: a novel approach to CPD

In light of all the aforementioned, accessible training delivery methods – such as massive open online courses (MOOCs) – have surfaced as promising avenues for teacher CPD. A MOOC allows teachers to attend high-quality training sessions which, for the most part, are typically free-of-charge, designed by academic experts, and much easier to fit into a teacher’s demanding work schedules (Misra, 2018; Tømte, 2019; Yıldırım, 2022).

It so happens that a handful of research studies have already recognized the potential of MOOCs for teacher CPD delivery, remarking on the flexibility and autonomy that they grant end-users (Duchaine et al., 2024; Misra, 2018; Tømte, 2019; Yıldırım, 2022). That said, few studies have focused on the question of the outcomes yielded from teacher enrolment in MOOCs, whether it be on shifts in knowledge, shifts in beliefs, shifts in practices, attitudes or student engagement. Among the rare studies that address the various outcomes, Gaudreau and Duchaine (2021) and Misra (2018) can be cited as one that examines the potential of MOOCs as a teacher CPD delivery method. However, given the combination of their exploratory study design, their impact being measured at only one time point immediately after training delivery, and the fact that teachers had not had time to implement knowledge or apply it in practice, more extensive study of outcomes is warranted.

With this in mind, our study aims to fill the aforementioned outcomes void by documenting the perceived impact of a MOOC in order to gain a more comprehensive understanding of its potential as a teacher CPD delivery system. To date, no studies would seem to have examined a MOOC through the lens of a training evaluation model, making this a novel approach for research in the education field. Accordingly, this study should make a significant contribution to the body of research by adopting a structured assessment methodology.

Description of variables

It is important to state from the outset that this article solely addresses a portion of the research findings resulting from the aforementioned study that sought to assess the perceived impact of a MOOC through the lens of Coldwell and Simkins’ (2011) training evaluation model. Herein, the focus remains on four of the six variables in our modified Coldwell and Simkins (2011) model, those being Knowledge and Attitude Shifts, Behavioural Shifts, Teacher Occupational Wellness, and Student Outcomes – all of which are more fully defined below. Participant Response is addressed in an article by Duchaine et al. (2024) that reports high satisfaction levels among MOOC attendees, further supporting its potential as an effective teacher CPD avenue.

The Knowledge and Attitude Shifts variable refers to any formal learning that takes place along with shifts in perspectives and mindsets that result from CPD endeavours. For the purposes of this study, we were expressly documenting teachers’ behaviour-based attitudes towards inclusive education for socially and behaviourally challenged children as an indicator of whether teachers seemed open to adjusting their workplace practices to meet such students’ needs. As for knowledge, our aim was to document any increased know-how regarding self-determination and Individualized Education Plan implementation. Coldwell and Simkins (2011) stress that, when measuring formal learning, criteria should align directly with the explicit training program objectives. This can be assessed immediately upon CPD completion (Kirkpatrick & Kirkpatrick, 2016).

The Behavioural Shifts (in Teacher Practices) variable refers to any formal knowledge (acquired during CPD) which teachers then translate into real-life practical application. By mesuring shifts in teachers’ workplace behaviours and practices, it becomes obvious whether the reported formal learning outcomes have in fact achieved the desired transformational effect on professional practice. In this study, the practices we were documenting specifically concerned the application of self-determined Individualized Education Plans (IEPs). It is worth mentioning that, as it is understood that shifts occur gradually and progressively as teachers experiment and adopt change (Coldwell, 2019), the behavioural shifts were measured across time (three to six months after the MOOC).

Occupational wellness encompasses the overall sense of satisfaction with one’s work situation, stemming from an ability to take appropriate action and make sound choices whilst accounting for social, environmental and individual factors (Laguardia & Ryan, 2000). Said otherwise, to attain a sense of workplace well-being, a person must achieve goals and meet standards that satisfy one of the three fundamental psychological needs (autonomy, competence or relatedness) (Laguardia & Ryan, 2000). For the purpose of this study, the Teacher Occupational Wellness (Sense of Self-Efficacy) variable refers to a teacher’s belief in their own ability to effectively work with socially and behaviourally challenged children.

The Student Outcomes (Self-Determination) variable refers to children’s tendency to use self-determined behaviours at school. Self-determination is defined as a child’s ability to invest in their own future, to regulate their own emotions, and to exercise freedom according to their own will (Field et al., 1998). Generally speaking, fostering self-determination involves: fostering a student’s self-awareness and self-worth; helping children feel effective and in control of life; defending and advocating for children’s rights; encouraging children to set and work towards goals, to make choices and decisions, and to solve problems; and, to aid children in developing their own individual perspectives and internal dialogue (Wehmeyer & Field, 2007). From the standpoint of students being successful at school, there is much evidence pointing to the fact that teachers should undeniably be directing their efforts towards building student self-determination levels as it lowers absenteeism and drop-out rates among students who adopt it (Blanchard et al., 2004). Futhermore, those with higher self-determination levels are more likely to adopt proactive behaviours, to steer their own educational course, and to persevere when facing obstacles (Field et al., 1998).

Context

Study participants and survey procedure

The study’s target population consisted of teachers attending an Individualized Education Plan Massive Open Online Course (IEP MOOC) in the Fall of 2022. Sampling was based on non-probability convenience sampling. The core inclusion criteria were 1) to be registered to attend the MOOC in question, entitled The Self-Determined IEP: Supporting Success for Students with Behavioural and Adjustment Difficulties in Fall 2022; and 2) to be a teacher employed on a temporary or permanent basis in mainstream and/or special education classrooms.

Having obtained ethical approval from the principal investigator’s university, an invitation was sent by email, describing the nature of the research and its implications, to everybody who had registered (N = 6998) for the MOOC (among them teachers, support staff, school principals and specialists) in order to recruit study participants. The email emphasized that participation in the study was voluntary and an entirely separate endeavour from attending the training itself.

Teachers who had elected to participate in the study (n = 72) were invited on a voluntary basis to complete the first online questionnaire (T0) prior to attending the MOOC training session (pretest). A week after attending, another email questionnaire (T1) was sent only to those who had completed the first one. Then, three months after the IEP MOOC was held, those who had completed both the pretest (T0) and the first post-test (n = 61) received another online questionnaire by email to complete a second post-test (T2), once again on a voluntary basis. Lastly, a year after attending the MOOC, those who had completed T0, T1 and T2 surveys (n = 58) were invited to complete post-test T3. In the end, 55 teachers had completed surveys at all four time points.

It is worth noting that only study participants who had been working with an IEP in the current school year were asked to answer certain questions which documented practices associated with self-determined IEP implementation and the perceived impact on students. Respondents were asked: “During the current school year, have you had to set up an IEP for a student?”. For those who answered “no”, the questionnaire ended promptly. In all, 26 teachers filled out the questions which documented IEP implementation practices at T1 and 10 teachers at T3. For the questions which documented perceived impact on students, 26 teachers responded at T1 and 20 teachers at T2. Table 1 outlines characteristics for the final 55 participants.

Twenty-seven percent of study participants stated they had attended the training with coworkers, providing further valuable opportunities for them to share and discuss the content. With respect to course completion mode, 71% of study participants had signed up for the purely theoretical program and 29% for the combined theory-and-practice program with 86% completing the training in under 15 weeks across both completion modes.

Survey instruments

An online questionnaire at T0 documented teachers’ prior knowledge, attitudes, beliefs and practices. The T1 online questionnaire documented those same variables in addition to the CPD’s perceived impact on students. Then, the T2 online questionnaire documented solely the perceived impact on students, and T3 solely teacher IEP practices.

The online questionnaire documenting teacher knowledge insofar as IEP implementation and self-determination comprised 17 questions (Cronbach’s α at T0 = .68; at T1 = .58). Respondents answered multiple-choice questions related to content broached during the IEP MOOC (e.g., Who can potentially run or lead an IEP Meeting?). Note that, for this sub-section, the Means shown in the findings should be understood as a test score (/17).

The online questionnaire documenting teachers’ attitudes towards inclusive education for socially and behaviourally challenged children was inspired by Massé et al.’s (2018) French-language adaptation of Mahat’s (2008) Multidimensional Attitudes Toward Inclusive Education Scale and comprised seven items (Cronbach’s α at T0 = .81; at T1 = .93). A six-point Likert scale ranging from 1 (completely disagree) to 6 (entirely agree) was used to ask respondents how strongly they agreed or disagreed with the seven statements (e.g., I’m willing to modify my practices to work with children experiencing maladjustments in order to foster their inclusion in mainstream classrooms).

The online questionnaire documenting teachers’ beliefs in their own ability (self-efficacy) to effectively work with socially and behaviourally challenged children comprised 10 items which were modified from Zhang et al.’s (2018) Student Teachers’ Efficacy Scale for Teaching Students with Disabilities. We utilized three of the scale’s four subscales with a six-point response framework ranging from 1 (zero competence) to 6 (very competent) to collect respondents’ own perceived competency level for each item. Table 2 contains our scale sub-sections, number of items, sample items and internal consistency over two points in time.

The online questionnaire documenting practices associated with self-determined IEP implementation were taken from the Inventaire des pratiques relatives à l’établissement des PIs pour les élèves présentant des difficultés d’adaptation [List of Practices Used in IEP Design for Children Experiencing Adjustment Challenges] by Gaudreau et al. (in preparation) which is a unidimensional scale comprised of 19 items (Cronbach’s α at T0 = .85; at T1 = .84; at T3 = .93). Respondents stated the frequency with which they adopted each practice listed (e.g., I ask for an IEP to be created when I notice a child persistently struggles) on a scale ranging from 1 (never) to 6 (always).

The online questionnaire documenting the observable impact of self-determination on children using IEPs was further modified from Massé et al.’s (2005) already modified version of the French-language adaptation of Kazdin et al.’s (1992) unidimensional scale Therapist’s Evaluation of Child Treatment comprising nine items (Cronbach’s α at T1 = .98; at T2 = .97). A six-point Likert scale ranging from 1 (completely disagree) to 6 (entirely agree) was used to ask respondants how strongly they agreed or disagreed with the seven statements (e.g., The practices I adopted as a result of the MOOC IEP training had a positive impact on autonomy for the targeted students).

Data analysis

Quantitative data analysis was conducted using IBM SPSS Statistics v28 software. Firstly, descriptive analyses were conducted (means and standard deviation). Then, paired sample t tests were carried out to calculate the variable discrepancies at different times. A one-tailed test was used, as there was an underlying assumption in this study that each variable would either remain stable or improve subsequent to teachers’ MOOC attendance. No Bonferroni correction was employed due to the relatively small sample size. Nonetheless, to mitigate this as a potential limitation, nonparametric testing (Wilcoxon signed rank test) was done and corroborates our findings.

The IEP Mooc’s Perceived Impact on Teacher Knowledge

Paired sample t-test results show statistically significant growth in teacher knowledge between T0 and T1 (see Table 3).

Among all items, the greatest knowledge shifts seem to centre around a student’s role during the IEP Meeting and a teacher’s role in fostering a child’s self-determination on the behavioural level.

The IEP Mooc’s Perceived Impact on Teacher Attitudes

Paired sample t-test results were inconclusive on whether there was a shift in teacher attitudes between T0 and T1 (see Table 4).

The IEP Mooc’s Perceived Impact on Teachers’ Sense of Self-efficacy

Paired sample t-test results show statistically significant increase in teachers’ perceived sense of self-efficacy between pretest T0 and post-test T1 on all three scale sub-sections (see Table 5).

Among all scale items, the greatest perceived shifts in self-efficacy between T0 and T1 appear to be the teachers’ belief in their ability to assess student self-determination levels and then tailor an IEP Meeting to match a student’s self-determination levels.

The IEP Mooc’s Perceived Impact on Teacher Practices

Paired sample t-test results show that, in statistical terms, teachers self-reported that there were significantly higher adoption rates for the targeted teacher practices at T1 compared with T0 (see Table 6).

Among all items, the greatest perceived shifts in practices that emerge between T0 and T1 concern the level of support given to a student during an IEP Meeting in order to foster more active student involvement and in helping students reflect upon their own personal priorities.

To investigate the sustainability of reported post-training practice shifts over time, a fourth time point (T3) was included. With only 10 teachers responding to this last survey, the t-test results are shown for informational purposes only. As Table 7 shows, descriptive analyses would seem to indicate that T1 findings held true.

Perceived Impact on Students

At post-test T1, according to teachers, the intervention methods which they adopted subsequent to the training sessions most substantially contributed to students having a more solid grasp of the IEP process insofar as how it works and its usefulness, along with greater overall self-awareness (own needs, strengths, challenges faced).

To investigate sustainability over time, a second round of data (T2) was collected for the perceived impact on students. With only 20 teachers responding to this survey, t-test results are shown for informational purposes only. Findings show no difference at T2 compared with T1 which seems to indicate that the reported positive effects hold true over time (see Table 8).