Introduction
Teachers’ professional knowledge and skills are widely recognized as crucial factors in successful integrated STEM education (Lin & Williams, 2016; Yu et al., 2021), which has led to a growing emphasis on enhancing teachers’ STEM teaching capabilities. Research has shown that teachers’ self-belief in carrying out classroom practices can significantly influence their STEM teaching intentions (Kelley et al., 2020; Margot & Kettler, 2019), as well as students’ STEM performance and career interests (Han et al., 2021). Furthermore, professional development has been identified as a key prerequisite for effective integrated STEM teaching (Brophy et al., 2008; Zhong et al., 2021). However, teachers working independently may struggle to incorporate content knowledge from disciplines beyond their area of expertise (Love & Wells, 2018). Consequently, those who lack confidence or have only a vague grasp of interdisciplinary integration face added challenges in delivering STEM lessons. This concern is underscored by Margot and Kettler (2019), whose study found that teachers often report low confidence in various aspects of integrated STEM instruction, including pedagogical methods and lesson planning.
Linked to the issues mentioned above, models and frameworks were proposed in order to provide educators with a clear view of how to teach STEM content to their students (Bybee, 2010; Corlu et al., 2014; Fan & Yu, 2016; Glancy & Moore, 2013; Kelley & Knowles, 2016; Thibaut et al., 2018; Wells, 2016). These models and frameworks have differences, while they share some similarities in essential features that enable teachers to acquire the knowledge or tend to have a positive perception of integrated STEM education value. On the other hand, for teachers with less experience, it is essential to provide instruction about what to do and how to do it; otherwise, they might still feel confused about where to start or what should be done when teaching integrated STEM education. Since most in-service teachers were not prepared for teaching in interdisciplinary contexts, guidance that gives teachers explicit instructions to teach integrated STEM education is urgent.
An instructional design (ID) model is a term to describe a conceptual model, usually created for teachers or professional developers to improve the efficiency of instructional development and the learning effect (Anderson & Goodson, 1980; Edmonds et al., 1994; Magliaro & Shambaugh, 2006). It is a process-based guideline that follows a series of steps to transform teaching principles into a systematic reflection process in education (Aldoobie, 2015; Smith & Ragan, 2004). While existing instructional design (ID) models assist educators in the step-by-step development and implementation of activities, many were originally crafted for single-discipline contexts and often neglect the complexities of interdisciplinary teaching. This oversight creates challenges when applied to STEM education. To address this gap, this study aims to develop an instructional design model explicitly designed for integrated STEM education. Grounded in a thorough analysis of relevant literature and a conceptual framework that emphasizes the critical features of interdisciplinary teaching, the model outlines the essential elements that educators should consider throughout the instructional process. By providing systematic guidance, the model aims to empower teachers to create comprehensive and cohesive STEM curricula, ultimately enriching the interdisciplinary learning environments for their students.
Related literature
Instructional Design Models
Instructional design models, generally speaking, are created to assist teachers to complete tasks in their teaching processes. There are several well-known ID models have been developed and applied; for instance, the ADDIE model (Analyze, Design, Develop, Implement, Evaluate) is one of the most frequently mentioned ID models in the education field (Budoya et al., 2019; Ismail et al., 2018; Nichols Hess & Greer, 2016). Instructional design using the ADDIE model is conducted in five sequential phases: analysis, design, development, implementation, and evaluation (Branch, 2009), which can be enacted repeatedly if necessary (Grafinger, 1988; Trust & Pektas, 2018). The ADDIE model has been practiced widely because it presents concise directions in each phase, allowing teachers to follow when generating an activity. The phases of the ADDIE model are described as follows: (1) Analysis requires teachers to analyze and determine goals, confirm the target audience, and identify resources; (2) Design requires teachers to determine learning objectives, design teaching activities and tasks, and decide on teaching strategies as well as evaluation; (3) Development asks teachers to develop course contents, prepare media and materials, and develop guidance; (4) Implementation covers preparing the environment, implementing instruction, and engaging student; and finally (5) Evaluation asks teachers to adopt an evaluation instrument they use to reflect or summarize any instructional modifications (Branch, 2009; Budoya et al., 2019). The five phases are presented as a sequential model, but teachers can move between phases if they are attempting to make adjustments during teaching, which shows the flexibility of the ADDIE model.
While the ADDIE model provides teachers with a structured approach to the design and implementation of teaching activities, it has faced several criticisms. Researchers have pointed out that the theoretical foundation of the ADDIE model is difficult to trace (Molenda, 2003), and there are gaps in its practical application, as the actual practice is often not considered (Gordon & Zemke, 2000; Schwier et al., 2004). To eliminate the weaknesses, Budoya et al. (2019) enriched the ADDIE model by combining ADDIE with the Feature Driven Development Process and stated that the improved model would satisfy practical needs. This work sheds light on expanding the ADDIE model to overcome the challenge of failing to connect theory and practice. In other words, an ID model could be practical for teachers to follow if integrated STEM education teaching practices have been considered.
Integrated STEM Education Teaching Practices
To develop an instructional design (ID) model that accounts for teaching practices, it is essential to first identify the characteristics of integrated STEM education teaching. Various models and frameworks offer diverse perspectives on integrated STEM education. Bybee (2010) used life and work situations to connect the four disciplines and proposed a framework for units in integrated STEM education. Corlu et al. (2014) framed an integrated STEM education model that emphasized interactions between mathematics and science. Similarly, Fan and Yu (2016) proposed an integrated STEM curriculum design model focusing on technology and engineering contexts, engineering design processes, science inquiry, technology use, and mathematical analysis. Concurrently, Glancy and Moore (2013) utilized engineering to connect with the other three disciplines in an integrated STEM translated model. Moreover, the integrated STEM learning conceptual framework proposed by Kelley and Knowles (2016) emphasized that learning must be grounded in situational contexts with the combination of scientific inquiry, technological literacy, mathematical thinking, and engineering design. Results from a study by Thibaut et al. (2018) summarized a theoretical framework for instructional practices in integrated STEM based on social constructivism, emphasizing five instructional elements. The PIRPOSAL (Problem Identification, Ideation, Research, Potential Solutions, Optimization, Solution Evaluation, Alterations, Learned Outcomes) pedagogical model for integrative STEM for education was centered on question-posing as the teaching strategy educators follow when guiding student learning during each phase of the engineering design process (Wells, 2016).
Integrative STEM Education utilizes technological and engineering design-based approaches to intentionally teach the principles and practices of science and mathematics alongside technology and engineering education (Wells & Ernst, 2012 as cited in Wells, 2013). This learner-centered process fosters interdisciplinary learning by embedding STEM subjects within authentic, design-driven problem-solving contexts. In this study, integrated STEM education emphasized the importance of teacher collaboration for effective STEM integration. Integrated STEM requires teacher teams to work together, drawing on their expertise to design interdisciplinary curricula and determine instructional approaches. This collaborative process ensures that STEM disciplines are meaningfully integrated within the teaching process. In summary, integrated STEM emphasizes collaborative instructional planning, where teachers work together to develop effective methods for integrating STEM content.
Although the abovementioned models and frameworks look distinct, some similar components are continually mentioned, which shows the uniqueness of integrated STEM education. Firstly, collaboration between teachers is emphasized when designing and teaching an integrated STEM education activity. Second, due to the complexity of integrated STEM education activities, teachers who work together should consider and discuss about target students’ prior knowledge in every discipline. Third, setting up a proper context for the target students is also essential since it could relate to their daily life, learning motivation and interests. Besides, the role of each discipline and connections among them in a STEM activity need to be identified, which means teachers must clarify their responsibility as experts in their own fields. Finally, it is an important skill for teachers to determine which teaching strategy suits the designed STEM activity. In light of the abovementioned information, an ID model for integrated STEM education should include general elements from the teaching process and the common features abstracted from existing STEM models and frameworks, such as team teaching, students’ knowledge in each discipline, authentic and complex learning contexts, roles of each discipline, and proper pedagogy, thereby emphasize characteristics of teaching in integrated STEM education.
While existing models and frameworks enhance teachers’ conceptual understanding of integrated STEM education, they often fall short in providing the clear, step-by-step guidance necessary for effective collaborative instruction. As a result, even after examining relevant research, teachers may still feel uncertain about successfully implementing STEM activities with their colleagues. To address this gap, this study abstracts key characteristics of integrated STEM education and expands upon the ADDIE model, presenting a theorized instructional design framework. In doing so, it aims to equip teachers with concrete directions that reduce challenges and promote more cohesive, collaborative STEM teaching.
Theorizing Integrated STEM Instructional Design Model
To establish the theoretical premise underpinning this study, this section substantiates an integrated STEM education instructional design model by analyzing relative works and improving the ADDIE model to overcome the obstacle between theory and practical needs.
Given research has demonstrated that teacher collaboration is vital for the success of STEM initiatives (Al Salami et al., 2017; Herro & Quigley, 2017; Margot & Kettler, 2019), assembling a strong team is an essential first step in integrated STEM education. It is crucial to have dedicated members who are enthusiastic about STEM education in place before teachers begin their work. Building a strong sense of teamwork early on fosters communication and confidence, enabling teachers to exchange new ideas, pose questions, and reflect on practices (Nolan & Molla, 2017). To achieve this, teachers must first identify colleagues committed to enhancing STEM instruction and form teams that include representatives from multiple disciplines. A foundation of shared professional knowledge is also vital, with science teachers, for example, understanding the fundamentals of engineering and engineering teachers learning about science inquiry. Finally, assembling a diverse, well-informed team aims to leverage each teacher’s professional expertise to deliver high-quality, integrated STEM education (Chamberlin & Pereira, 2017; Eckman et al., 2016).
In collaborative STEM teaching, the Analysis phase in the ADDIE model is crucial for teacher teams as they work together to identify students’ cultural capital, available teaching resources, and each discipline’s role within an integrated approach. By examining students’ prior knowledge, attitudes, and skills – that Williams et al. (2015) describe as cultural capital – teachers can collectively pinpoint potential gaps and develop support strategies for activities requiring specialized competencies, such as ICT skills. Working together to determine the knowledge, attitudes, and skills to be assessed ensures that goals are defined across all disciplines. Environment analysis is also essential (Wang et al., 2011), prompting teacher teams to consider factors like classroom configuration and technological tools so they can effectively align resources. Moreover, selecting authentic STEM contexts that connect with students’ experiences and tackle authentic problems (Kelley and Knowles, 2016) reinforces interdisciplinary problem-solving. By sharing insights and expertise during this analysis phase, teacher teams can design cohesive, relevant, and engaging STEM instruction.
During the Design phase, teacher teams must collaborate to establish clear learning objectives, identify relevant disciplines, determine authentic contexts, outline suitable pedagogical approaches, create interdisciplinary teaching activities, and set assessment criteria. This collaboration is vital in integrated STEM education, as it requires a comprehensive understanding of how various disciplines intersect (English, 2016). By integrating these disciplines and recognizing each contribution to activities, teachers ensure that learning is cohesive and meaningful. Teacher teams also develop the instructional context and pedagogical strategies for interdisciplinary teaching (Dow, 2006) and design a series of student-centered activities (Moore et al., 2014). Ultimately, they align assessment criteria with the defined learning objectives, capturing multiple dimensions of learning—cognitive, affective, and skill-based (Gao et al., 2020). Through this collaborative design process, teacher teams create a robust framework that promotes deeper engagement and understanding in STEM learning.
Following the Design phase, teacher teams turn to a detailed planning process that outlines all implementation aspects. Unlike the original Development phase in the ADDIE model which involves creating course content, preparing media, and developing guidance, this Planning stage goes beyond content creation to include the distribution of workloads, scheduling, and coordination among team members. Learning content should be grounded in authentic contexts, highlighting the connections between various disciplines. It is equally important to create tasks that challenge students’ existing knowledge and motivate them to draw on their own experiences (Moore et al., 2014). Teacher teams must also plan a supportive learning environment where students receive assistance as they engage in STEM activities. Additionally, discussions around in-class assignments – whether conducted through co-teaching or individual instruction – should align with models summarized by Kelley et al. (2021), such as single-teacher delivery, multiple teachers across classrooms, or a community-based model. Finally, involving STEM professionals as guest speakers or consultants can enrich student learning by providing practical insights.
Once the planning is complete, the Implementation phase becomes critical for teacher teams to ensure students achieve the intended learning goals in integrated STEM education. Integrated STEM education presents exciting opportunities to tackle complex, open-ended problems that encourage critical thinking and creativity, offering a refreshing contrast to traditional single-discipline instruction. By embracing challenges that may not have a definitive solution, students can develop essential problem-solving skills and innovative thinking. Consequently, effective implementation requires teachers to facilitate collaborative problem-solving, guide students using targeted questioning (Fan et al., 2021; Tang, 2017; Wells, 2016; Yeh et al., 2015), and provide scaffolded support in a design challenge to encourage students to identify problems and propose effective solutions (Belland, 2017; Hughes & Denson, 2021). Questioning techniques, frequently employed in scientific inquiry, encourage students to explore multiple solutions by prompting deeper thinking (Krajcik & Czerniak, 2018). Throughout this process, assessment should be ongoing: teacher teams can use both formative and summative methods, alongside portfolios, to document students’ progress. This continuous feedback mechanism enables teachers to adjust instruction as needed, ensuring that implementation remains aligned with interdisciplinary STEM learning goals.
In the final step, the Evaluation stage involves the instruments teacher teams use to assess students’ learning performance in integrated STEM education. Because this approach encourages students to generate solutions based on their ideas, they often draw on diverse knowledge and skills. As a result, a variety of assessment methods is necessary – both during and after the activities – to provide a comprehensive view of student progress. This iterative process facilitates ongoing evaluation during instruction, allowing teachers to assess the effectiveness of their teaching strategies and gain a deeper understanding of student outcomes. For instance, having students compile portfolios promotes self-reflection and gives teachers insight into learners’ development by the end of the STEM unit (Cunningham, 2017). It is equally important for teacher teams to review students’ challenges and solutions. These discussions should be used to identify improvements for future lessons. Ultimately, this collaborative Evaluation process offers valuable feedback that guides subsequent integrated STEM education activities, ensuring continual refinement of teaching practices.
To conclude, teaching integrated STEM education requires the teacher team to spend time communicating and collaborating, and thus, a preparation stage before starting to design an activity is identified as necessary. Besides, the Development phase in the ADDIE model used to focus on preparing contents, media, and materials; however, integrated STEM education activities emphasize providing students opportunities to reflect on their experiences while learning (Moore et al., 2014; Schön, 1991), showing the importance of whether teachers can plan meaningful learning experiences for students. As a result, a planning stage is used as a term to replace Development to reveal the feature. Finally, the integrated STEM education instructional design model (integrated STEM education ID model) was theorized with six stages: preparation, analysis, design, planning, implementation, and evaluation (PADPIE). The six stages were formulated as a conceptual model which expanded the ADDIE model under the concerns of practical teaching needs and emphasized integrated STEM characteristics.
Method
To confirm this provisional ID model of integrated STEM education, the Delphi consensus-building method was employed to achieve general agreement of opinions among a group of experts through iterative questionnaires. The Delphi method generally preserves participants’ anonymity using structured questionnaires to collect data. Considering time, cost, and panel attrition, two or three rounds of the Delphi technique are typically adequate for most studies (Hasson et al., 2000; McMillan et al., 2016).
In preparation for conducting a Delphi study, literature on the ADDIE model was reviewed, as well as on STEM models and frameworks, which yielded a six-stage ID model that conveys an integrated STEM education teaching process. The study originally outlined the definitions of each stage and the recommended tasks for teachers in tables included within the design questionnaires. These details were presented to a group of experts during pilot investigation to obtain preliminary feedback on the model. Subsequently, a two-round Delphi research was conducted, with questionnaires distributed via email to an international panel of STEM education scholars to achieve consensus on the model’s components. Additionally, insights from the experts’ discussions were analyzed to further refine the model, resulting in clearer definitions for each stage and the creation of a supplemental checklist tool to assist teachers in implementing integrated STEM education.
Panelists
The present study was conducted in two phases – a pilot investigation followed by the Delphi research; therefore, two panelists were recruited for each of the two phases. First, a pilot investigation was done to confirm the propriety of the integrated STEM education ID model with a group of experts, and the second phase was the Delphi study, which involved another, different group of panelists.
The Pilot Investigation Panelists
A convenience sampling technique was conducted in the present study to identify and recruit panelists. In order to validate the integrated STEM education ID model with multiple perspectives, the researchers invited contacts in their networks, such as members of the International Conference on Technology Education (ICTE) and the International Technology and Engineering Educators Association (ITEEA), to collect insights from different regions. For the pilot investigation, nine experts (Table 1) with a specialty in developing instructional design models were recruited to confirm the propriety of the integrated STEM education ID model, involving the stage definitions and the tasks assigned in each stage. The members of the panel was comprised of one from Australia, one from Hong Kong, one from Korea, one from the Netherlands, two from New Zealand, one from Japan, one from Taiwan, and one from the United States.
Table 1
Panelist Information for Pilot Investigation.
| Panelists | Country | Profession/expertise |
| 1 | Australia | STEM, PCK |
| 2 | Hong Kong | STEM, technology education |
| 3 | Korea | STEM, teaching strategies |
| 4 | Netherlands | STEM, the pedagogy of technology and design |
| 5 | New Zealand | STEM, innovation in teaching |
| 6 | New Zealand | STEM, teaching and learning strategies |
| 7 | Japan | STEM, teaching and learning strategies |
| 8 | Taiwan | STEM, teaching methods |
| 9 | United States | STEM, pedagogical practices among STEM education areas |
The Delphi Research Panelists
The Delphi research is designed to elicit experts’ opinions and comments; therefore, implementing rigorous panelists selection criteria is critical. In the current study, the experts who were invited to participate in the focus groups in the Delphi research must meet the following criteria: (1) an expert who has more than 5 years of teaching experience in the field of science, technology, engineering, or mathematics at a higher educational level; (2) an expert who has more than 5 years of teaching experience in STEM education; and (3) an expert who has research specialty in STEM education, such as articles or books publishing.
Based on findings from previous studies, a panel of 10 experts is an acceptable minimum size for valid Delphi research (Linstone & Turoff, 1975; Ziglio, 1996). As such, an invitation was sent to 21 experts who met the selection criteria of the study. The study eventually received 12 responses from individuals in Australia, Hong Kong, New Zealand, Japan, Taiwan, and the United States. Of these 12 individuals, three were in science education, four in technology education, three in engineering education, and two in mathematics education (Table 2).
Table 2
Panelist Information for the Delphi Research.
| Panelists | Country | Profession/expertise |
| 1 | Australia | Technology education, STEM, mentoring beginning teachers |
| 2 | Hong Kong | Science education, STEM, professional development |
| 3 | New Zealand | Technology education, STEM, teaching approaches |
| 4 | Japan | Engineering education, STEM, creativity and design thinking |
| 5 | Taiwan | Engineering education, STEM, emerging technology in education |
| 6 | Taiwan | Engineering education, STEM, teacher education |
| 7 | Taiwan | Mathematics education, STEM, teaching/learning, pedagogy |
| 8 | Taiwan | Mathematics education, STEM, teaching/learning |
| 9 | Taiwan | Science education, STEM, teacher education |
| 10 | United States | Science education, STEM, teaching/learning |
| 11 | United States | Technology education, STEM, teaching/learning |
| 12 | United States | Technology education, STEM, teacher education |
Instrument
Two questionnaires were developed as the instruments used in the pilot investigation and the Delphi study.
Instrument in the Pilot Investigation
The questionnaire for the pilot investigation, which was developed based on the initial six-stage model, consisted of three parts: stage definition, tasks in every stage, and review comments. The panelists were asked to read through the description and tasks, and then express their opinion by choosing “agree, pending adjustment” or “recommended deletion” on a task. In addition to giving opinions on whether the tasks were suitable, they were also encouraged to provide comments and suggestions on definitions or tasks. Table 3 displays the stage definitions and 15 tasks (two in preparation, four in analysis, two in design, two in planning, two in implementation, and three in evaluation) that were reviewed in the pilot investigation.
Table 3
The Pilot Investigation Questionnaire.
| Stages | Definitions and Tasks |
| Preparation | |
In the preparation step, teachers are going to organize a team of STEM teaching to discuss and prepare courses together. The STEM teaching team should include teachers from different disciplinary; in this case, they can use their profound knowledge to teach the content professionally and enable students to solve real-world problems relying on the knowledge during the courses.
| |
| Analysis | |
In the analysis step, teachers should analyze students’ culture capital and teaching resources, such as students’ computers and other technology tools, before starting to design activities. Moreover, the most important thing in the analysis step is that the teacher must define the value of the STEM course first. For example, the purpose of the STEM course is to cultivate students’ transformative competencies in solving complex problem, which will be the main value of this STEM course; therefore, teachers should discuss how to design activities and decide the learning objectives to achieve this goal.
| |
| Designing | |
In the designing step, teachers should arrange a context in the STEM activities to provide students an opportunity to utilize the science and math concepts and to learn how to solve real-world problems.
| |
| Planning | |
There are gaps with teachers’ expectation when students follow the problem-solving process or engineering design process during the courses. For example, when students encounter problems, they may try to avoid failure, and this could cause losing learning opportunities and experiences. According to Moore et al. (2014), it is necessary of STEM education to provide students with opportunities to learn from their mistakes and deepen what they have learned. This can be tracked back to the theory of learn-by-doing (Dewey, 1938), which not only emphasizes hands-on, inquiry, and practical experience, but also believes that the reflective experience is also necessary (Schön, 1991). Therefore, in the planning step, teachers should plan to arrange hands-on, practical experience, and reflective experience for students to gain meaningful experiences.
| |
| Implementation | |
The engineering design process is regarded as a gateway for students to convert science and math knowledge into usable abilities. Even though the engineering design process can be a specific procedure that students follow, teachers should still provide guidance and assistance properly to ensure students will learn from their experience and will not encounter unsolvable difficulties. Thus, in the implementation step, teachers should implement STEM activities through engineering design process to give a specific procedure for students when they are trying to solve ill-defined problems, and teachers should be good guides and facilitators.
| |
| Evaluation | |
Dewey (1938) stated that, experience involves continuity, which means experiences will influence one’s future experiences and decisions. With this principle, in the evaluation step, teachers should understand students’ learning achievement through using assessment tools and help students accumulate continuity experiences of interdisciplinary. The evaluation results will be fed back to all previous steps to provide evidence, which can help teachers reschedule the course next time.
|
Instrument in the Delphi Study
Two researchers developed the Round 1 questionnaire based on feedback from the pilot investigation. The questionnaire consisted of twenty-one tasks, categorized across six stages: three in preparation, two in analysis, five in design, three in planning, five in implementation, and three in evaluation.
In the Delphi study, panelists were asked to rate each task’s importance using a 5-point Likert scale (1 = not at all important to 5 = very important). In addition, panelists could provide free-text comments offering suggestions for improvement. For example, in Round 1, some experts highlighted that the term consensus in Pre2 might be confusing. They suggested revising the item to clarify that teachers should aim for a shared understanding of STEM teaching by discussing strategies rather than simply seeking agreement. After reviewing the panelists’ feedback, Pre2 was revised to provide more explicit guidance within the integrated STEM education instructional design (ID) model.
Following a review of the results from Round 1, the questionnaire for Round 2 was revised to include twenty-five tasks: three in preparation, three in analysis, six in design, four in planning, five in implementation, and four in evaluation. Both rounds were conducted via Microsoft Word documents sent by email. Table 4 shows the comparisons between tasks in Rounds 1 and 2.
Table 4
The Delphi Study Questionnaires.
| Stages | In round 1 | In round 2 |
| Preparation | ||
| Pre1 | Organize a team of teachers who are competent at interdisciplinary STEM integration as a consulting group. | Organize a team of teachers, including teachers from different disciplines, as a consulting group (at least one for each discipline). |
| Pre2 | Reach consensus on teaching to make sure the teachers who involved in are familiar with STEM education. | including teachers from different disciplines, as a consulting group (at least one for each discipline). Share STEM education teaching strategies to ensure the STEM Teacher Team has a common understanding of interdisciplinary integration. |
| Pre3 | Schedule a sequence of meeting times for joint discussion of planning STEM activities. | Schedule a sequence of meeting times for joint discussion of planning STEM activities. |
| Analysis | ||
| A1 | Analyze students’ prior and pre-requisite knowledge and skills. | Analyze students’ prior and pre-requisite knowledge, attitude, and skills which needed in STEM activities. |
| A2 | Analyze the environment, including teaching resources (i.e. classroom setting and technology devices or tools) and features of the school. | Analyze the environment, including teaching resources (i.e., classroom setting and technology devices or tools) and the features of schools to better know what context, activities, or materials can be used while teaching STEM. |
| A3 | - | Analyze what is authentic to students to solve or address. |
| Design | ||
| D1 | Determine the learning objectives and teaching strategies. | Determine the learning objectives. |
| D2 | - | Determine the pedagogical approaches (how teachers are going to integrate their different philosophies and teaching styles to support students’ learning). |
| D3 | Design the criteria and assessment tools (formative/summative) for a particular activity for assessing students’ competencies. | Design the criteria and assessment tools (formative and summative) or have students design a portfolio for assessing their knowledge, attitude, and skills. |
| D4 | Analyze the role of science, mathematics, technology, and engineering, and identify the relationship of each subject or how to bring them together. | Analyze the role of science, technology, engineering, and mathematics and their contribution to the activities, and identify the relationship of each discipline or how to bring them together. |
| D5 | Identify and design a student-centered context which is authentic to students and relate to their life experience. | Identify and design a student-centered context which is authentic to students and relate to their life experience. |
| D6 | Design an activity based on the context to encourage students to integrate and learn new knowledge and skills. | Design a series of projected-based/inquiry-based/problem-based activities based on the context to force students to integrate and learn new knowledge and skills. |
| Planning | ||
| P1 | Develop learning contents based on the activity and the relationship among the subjects should be emphasized. | Develop learning contents based on the learning objectives, context and the relationship among the disciplines should be emphasized. |
| P2 | Design and plan learning tasks which can guide students to utilize different knowledge and competency of STEM subjects in STEM activity. | Arrange structured learning tasks which are able to allow or challenge students to utilize and develop knowledge and competency of the different disciplines in STEM activities. |
| P3 | Propose a sequential teaching process, including lecturing, demonstrating, and evaluation, etc. | Create an environment where students are supported with necessary learning resources, including a sequential teaching process, necessary equipment, materials, manuals or guidance, etc. |
| P4 | - | Decide work assignment and teaching schedule for the members in the STEM Teacher Team. |
| Implementation | ||
| I1 | Follow the teaching process determined in previous steps to deliver courses. | Follow the pedagogical approach determined in previous steps to deliver courses. |
| I2 | Introduce different design processes as guidance for students to construct a framework, picture or flow when solving the ill-defined problems. | Provide students opportunities to design their own learning sequence. |
| I3 | Use questioning techniques to stimulate students to find out the responses to the challenges they faced. | Prompt students to think for themselves by the use of questioning skills. |
| I4 | Provide timely feedback to students during the design process. | Provide scaffolding for students who have problems while learning. |
| I5 | Implement assessment designed for the activity and provide opportunity for students to do reflection. | Evaluate students with formative and summative assessment or have them design and make a portfolio to allow reflection to happen. |
| Evaluation | ||
| E1a | Assess students with designed criteria. | - |
| E1b | - | The evaluation step should be iterative in the teaching process. |
| E2 | - | Include multiple types of assessment. |
| E3 | Analyze and discuss the challenges and progress in the learning process with students to accumulate continuity of interdisciplinary utilizing. | The evaluation needs to reflect students’ learning outcomes to allow them to know what they learned. |
| E4 | Review the steps of the PADPIE instructional design through the evaluation results and make suggestions for revision. | Through the evaluation, teachers are able to propose adjustments for future STEM teaching. |
Research Procedure
Phase 1: Establishment of the Theoretical Basis
Phase 1 was designed to establish the initial integrated model and identify stages by researching relevant fields. An integrated STEM education ID model was proposed, and the stages and tasks were formulated into a pilot investigation questionnaire and distributed to the pilot investigation panelists.
Phase 2: Pilot Investigation
The pilot investigation phase confirmed the definitions of the six stages and determined the propriety of the tasks. A consultation questionnaire was sent by e-mail, and nine responses were collected during a one-month period. The stage definitions and tasks were then analyzed and revised by two researchers, which were then formulated as the Round 1 Delphi questionnaire.
Phase 3: Round 1 Delphi
Twenty-one potential international experts were invited via e-mail to participate in the Delphi study. Twelve agreed to participate and completed Round 1 of the Delphi. The responses were analyzed and summarized to create the Round 2 Delphi questionnaire.
Phase 4: Round 2 Delphi
In Round 2 the experts were asked to re-rate the revised tasks, as well as newly added tasks generated through Round 1. The Round 2 questionnaire presented panelists with the descriptive statistics and comparisons between the original and revised tasks. The descriptive statistics showed the consensus results from Round 1, and the comparison table showed the differences between the original and the revised tasks. Any modifications, including stage definitions, revised tasks, and newly added tasks, were marked in RED. Information such as previous personal ratings, median ratings, interquartile range (IQR), and expected probability (EP) were provided. When all tasks reached consensus, the Delphi concluded (Figure 1).
Figure 1
Research Implementation Phases.
Data Analysis
Various measures can be used to analyze data to show a consensus of rating scores in the Delphi process. The median value measures central tendency (Hsu & Standford, 2007). A median score ≥ 4 on a 5-point scale is taken as the acceptable criterion to assess the central tendency of a task. To test for dispersion, standard deviation (SD) and IQR were used to demonstrate the collective judgments of the experts for each item (Keeney et al., 2010). The IQR is calculated as the absolute score of differences between the 25th and 75th percentiles. A minor score of IQR and SD represents a slight variance in the ratings and a lower distribution of the scores, respectively, showing higher agreement between the panelists; however, the SD and IQR values must not be higher than 1.0 (value = 1.0 is acceptable) (Raskin, 1994; Rayens & Hahn, 2000; Williams & Webb, 1994). In addition, we provided EP values as the rating frequency for a task using percentage for the panelists during the second round of the Delphi process (Corbin & Strauss, 1990).
We used Kendall’s coefficient of concordance (Kendall’s W) and non-parametric Spearman rank correlation coefficient analysis for the levels of agreement and stability as the criteria to end the Delphi process. Kendall’s W tests inter-rater reliability with W > 0.7: strong consensus, W > 0.5: moderate consensus, and W < 0.3: weak consensus. A p-value < 0.05 is considered statistically significant (Habibi et al., 2014; Schmidt, 1997; Schmidt et al., 2001; Siegel, 1956). Spearman’s correlation analysis was used to assess the correlation of ratings between two rounds as a measure lacking a normality requirement. Spearman’s scores < 0.4 represent weak correlation, scores between 0.4 and 0.6 represent moderate correlation, and scores > 0.6 show a strong correlation between rounds (Mukaka, 2012).
Results and Discussion
Consensus and Stability
The Delphi technique is usually applied to identify consensus on a particular topic presented to the panel of selected experts using a series of questionnaires. Kendall’s coefficient of concordance and Spearman’s correlation analysis were conducted using SPSS 23.0 to ensure consensus and overall stability. Kendall’s W was used to determine the level of agreement between the panelists for each round, with values in both rounds meeting the agreement level criteria (p < 0.01) across all panelist ratings. Furthermore, we calculated Spearman’s correlation score to demonstrate the stability of the panelists’ ratings between both rounds. Our results show a low but significant correlation (Spearman’s Rho = 0.349, p < 0.01) between the panelists’ opinions, indicating good stability.
A possible reason for the low Spearman’s Rho’s value could be a relationship between the ratings in both rounds. The improved task statements based on the experts’ opinions from Round 1 led to increased ratings for some tasks in Round 2. For example, after revising the P1 statement, five experts expressed a higher level of agreement on P1 (two experts raised the rating from 3 to 5, and the others changed the rating from 4 to 5 in Round 2). The SD and IQR results were improved in P1; therefore, even if the Spearman’s Rho showed only a low correlation between both rounds, the agreement levels were increased.
Item Modifications
Table 5 provides the rating consensus from the Delphi process. In Round 1, the importance of tasks was ranked within median scores ranging from 4 to 5. The SD was 0.29–1.14, and IQR values were 0.75–2.00. Overall, 19 out of 21 tasks achieved consensus. The statistical results for Pre2 in the preparation stage (SD = 1.14 > 1.00) and I2 in the implementation stage (IQR = 2.00 > 1.00) show that two tasks failed to meet the consensus criteria. Other than these two tasks, the panelists strongly agreed on the importance of the other tasks assigned at each stage.
Table 5
Delphi Process Results: Rating Consensus.
| Stage/Task | Round 1 | Round 2 | ||||||
| M | Mdn | SD | IQR | M | Mdn | SD | IQR | |
| Preparation | Preparation | |||||||
| Pre1 | 4.17 | 4.00 | 0.94 | 1.00 | 4.50 | 4.50 | 0.52 | 1.00 |
| Pre2 | 4.25 | 5.00 | 1.14* | 1.00 | 4.92 | 5.00 | 0.29 | 0.00 |
| Pre3 | 4.17 | 4.00 | 0.94 | 1.00 | 4.33 | 5.00 | 0.98 | 1.00 |
| Analysis | ||||||||
| A1 | 4.67 | 5.00 | 0.49 | 1.00 | 4.75 | 5.00 | 0.45 | 0.75 |
| A2 | 4.42 | 5.00 | 0.79 | 1.00 | 4.83 | 5.00 | 0.39 | 0.00 |
| A3 | – | – | – | – | 4.33 | 4.00 | 0.65 | 1.00 |
| Design | ||||||||
| D1 | 4.83 | 5.00 | 0.39 | 0.00 | 5.00 | 5.00 | 0.00 | 0.00 |
| D2 | 4.92 | 5.00 | 0.29 | 0.00 | 4.92 | 5.00 | 0.29 | 0.00 |
| D3 | 4.75 | 5.00 | 0.45 | 0.75 | 4.67 | 5.00 | 0.49 | 1.00 |
| D4 | 4.92 | 5.00 | 0.29 | 0.00 | 5.00 | 5.00 | 0.00 | 0.00 |
| D5 | – | – | – | – | 4.50 | 5.00 | 0.67 | 1.00 |
| D6 | 4.83 | 5.00 | 0.39 | 0.00 | 4.92 | 5.00 | 0.29 | 0.00 |
| Planning | ||||||||
| P1 | 4.33 | 4.50 | 0.78 | 1.00 | 4.83 | 5.00 | 0.39 | 0.00 |
| P2 | 4.83 | 5.00 | 0.39 | 0.00 | 4.75 | 5.00 | 0.62 | 0.00 |
| P3 | 4.33 | 4.00 | 0.65 | 1.00 | 4.58 | 5.00 | 0.51 | 1.00 |
| P4 | – | – | – | – | 4.67 | 5.00 | 0.49 | 1.00 |
| Implementation | ||||||||
| I1 | 4.00 | 4.00 | 0.85 | 0.75 | 4.25 | 4.00 | 0.87 | 1.00 |
| I2 | 3.92 | 4.00 | 1.00 | 2.00* | 4.50 | 5.00 | 0.67 | 1.00 |
| I3 | 4.50 | 5.00 | 0.67 | 1.00 | 4.83 | 5.00 | 0.39 | 0.00 |
| I4 | 4.33 | 5.00 | 0.98 | 1.00 | 4.83 | 5.00 | 0.58 | 0.00 |
| I5 | 4.58 | 5.00 | 0.67 | 1.00 | 4.92 | 5.00 | 0.29 | 0.00 |
| E1a | 4.50 | 5.00 | 0.67 | 1.00 | – | – | – | – |
| E1b | – | – | – | – | 4.58 | 5.00 | 0.51 | 1.00 |
| E2 | – | – | – | – | 4.58 | 5.00 | 0.79 | 0.75 |
| E3 | 4.58 | 5.00 | 0.67 | 1.00 | 4.83 | 5.00 | 0.39 | 0.00 |
| E4 | 4.67 | 5.00 | 0.49 | 1.00 | 4.83 | 5.00 | 0.39 | 0.00 |
| Kendall’s W | 0.326 (p<.01) | 0.180 (p<.01) | ||||||
[i] Note1. E1a was combined with D6 in Round 2; E1b was a new task added in Round 2
Note2. * Did not meet the criteria
Several experts among the Delphi panelists raised a concern on Pre2; for example, one of them suggested that “the Pre2 task needs expanding to show teachers what should they prepare for”, and others expressed the concern that “teachers in a team need to have a common minimal understanding and background knowledge about STEM or share view among members.” Accordingly, the task Pre2 in the Preparation stage was revised from Reach consensus on teaching to make sure the teachers involved are familiar with STEM education to Share STEM education teaching strategies to ensure the STEM Teacher Team has a common understanding of interdisciplinary integration (Table 4). To summarize the opinions of the expert panelists for task I2 in the Implementation stage, emphasis was placed on teachers as facilitators and consultants during the whole process instead of offering answers or design processes and asking students to obey them. Several design processes can be applied to various activities to instruct students’ learning, but in integrated STEM education activities, it is more valuable to have students explore every possibility. For this reason, task I2 was revised from Introduce different design processes as guidance for students to construct a framework, picture or flow when solving the ill-defined problems to Allow students to determine their next action (dynamic and iterative process) based on their findings and progress (Table 4).
Most tasks achieved high importance ratings from the panelist; however, combinations, deletions, or clarifications were made following the experts’ free-text suggestions and comments. Some had only minor changes by adding words while some were rewriting to clarify the tasks such as A2, P2, P3, I4, and E4 (Table 4). Apart from that, task D1 Determine the learning objectives and teaching strategies, was divided into D1, Determine the learning objectives, and D2, Determine the pedagogical approaches (how teachers are going to integrate their different philosophies and teaching styles to support students’ learning) in Round 2 based on the panelist’s recommendations since the original statement contained more than one mission. In addition, some tasks were added in certain stages following the panelist’s recommendations, such as A3, P4, E1b, and E2 (Table 4), to elaborate on the teaching process in integrated STEM education.
Discussion around practical challenges comprised a significant portion of Round 2. The six-stage integrated STEM education instructional design model resulting from this research, aims to provide direction for teachers who rarely have teaching experiences in integrated STEM education. The panelist recommended that the tasks be written briefly; otherwise, teachers may feel confused and challenged to apply the model. Some ranges were thus refined or reorganized based on the experts’ viewpoints. Below are the critical points of the feedback.
Teacher team organizing in the Preparation stage: Despite all the experts agreeing on the importance of managing a STEM teacher team in the Preparation stage, two of them questioned if it is possible to include teachers in all different disciplines. One expert from the U.S. noted, In reality it may not always be possible to have all four disciplines in a STEM teacher team. Depending on the grade level, you might have the S and M teacher, and either the T and E teacher. In response to this challenge, others emphasized the significance of discussions on teachers’ roles and work allocation, which impacted tasks Pre2 and Pre3 if Pre1 cannot be matched in some schools.
Pedagogical approaches determining in the Design stage: Some of the experts in the panelist debated whether pedagogical approaches can be developed until the nature of the activity is decided upon (an educator from Australia), revealing that Pedagogical approaches should arise from the nature of the activity, not from the teachers preferred teaching styles. Another two participants expressed the same idea commenting that Students are the core of the teaching and learning process, the teaching team should adjust the teaching method and/or styles according to the needs of students (an educator from Taiwan) and I think the teaching approach will emerge from the rest of the planning (an educator from the U.S.). However, some experts have made a defense by emphasizing the importance of teachers being aware of their teaching philosophies and pedagogical approaches. An educator from Taiwan suggested that the researchers should comprehensively consider both sides’ comments and specify the details of task D2.
The wording and timing issues in the Evaluation stage: A debate occurred on the wording of assessment and evaluation in the stage of Evaluation. Following the original definition in the ADDIE model, the use of assessment refers to no comparative data or value judgment; while evaluation involves the description of criteria and decision-making (Branch, 2009). As a result, the tasks in this stage were revised using the term assessment, and the term evaluation was only used for the stage title. The evaluation stage used to be at the end of the teaching process; however, several participants expressed concerns about whether that is where it actually occurs. An expert from New Zealand indicated that Ongoing assessment and evaluation is critical and should occur in every one of the above steps. It should not be a task in itself but should occur naturally. Other comments also expressed the same viewpoint; for example, I believe the evaluation stage is iterative and although this model is cyclical, I think there needs to be a lot of movement between all stages from two different members. In addition, the expert from Hong Kong further recommended that evaluation should be the assessment of both the instructional model and the overall students’ learning outcomes. In summary, conversations across the panelist shed light on the form of a conceptual model, revealing the lens of the non-linear instructional design model.
Overall, all the tasks achieved consensus from the panelists in Round 2 with median scores > 4.0 (range 4.0–5.0), SD was < 1 (range 0–0.98), and IQR was between 0 and 1, indicating a high level of consensus amongst the expert ratings. Kendall’s W and Spearman’s Rho attained a statistically significant threshold (p-value < 0.01). These results brought the Delphi study to a conclusion, and where Round 2 became the final round. The panelists identified a total of 25 tasks (three in preparation, three in analysis, six in design, four in planning, five in implementation, and four in evaluation) (See Table 6).
Table 6
Delphi Process Results: Task Consensus
| Stages | Tasks |
| Stage 1: Preparation | |
| Organize a STEM Teacher Team, including teachers from different disciplines, as a consulting group (at least one for each discipline). Share STEM education teaching strategies to ensure the STEM Teacher Team has a common understanding of interdisciplinary integration. Identify the role of each teacher and schedule a sequence of meeting times for joint discussion of planning STEM activities. | |
| Stage 2: Analysis | |
| Analyze students’ prior and pre-requisite knowledge, attitude, and skills which are targeted for assessment in the planned STEM activities. Analyze the environment, including teaching resources (e.g., classroom setting and technology devices or tools) and the features of schools to better know what context, activities, or materials can be used while teaching STEM. Analyze what the students would consider to be an authentic problem. | |
| Stage 3: Design | |
| Determine the learning objectives. Analyze the role of science, technology, engineering, and mathematics and their contribution to the activities, and identify the integration of the disciplines. Design a student-centered context which is authentic to students and relate to their life experience. Design a series of project-based/problem-based/design-based/inquiry-based activities to facilitate integration of new knowledge and skills. Determine the pedagogical approaches arise from the nature of the activity (how teachers are going to integrate their different philosophies and teaching styles to support students’ learning). Design the assessment criteria and tools (formative and summative) or have students develop a portfolio for assessing their knowledge, attitude, and skills. | |
| Stage 4: Planning | |
| Develop learning contents based on the learning objectives, context (problems that students need to solve) and the relationship between the disciplines should be emphasized. Arrange structured learning tasks which are able to allow or challenge students to utilize and develop knowledge and competency of the different disciplines in STEM activities. Create an environment where students are supported with necessary assistance, including a sequential teaching process, necessary equipment, materials, manuals or guidance, etc. Decide work assignment and teaching schedule for STEM Teacher Team members. | |
| Stage 5: Implementation | |
| Follow the pedagogical approach determined in previous stages to deliver courses. Allow students to determine their next action (dynamic and iterative process) based on their findings and progress. Prompt students to think for themselves by the use of questioning skills. Provide scaffolding for students who have problems while learning. Evaluate students with formative and summative assessment and have them develop a portfolio to record their reflection. | |
| Stage 6: Evaluation | |
| Repeat over time in a teaching process to provide feedback for other stages. Include multiple types of assessment to fit the various learning objectives. Provide students an opportunity to reflect their learning outcomes and allow them to know what they learned. Propose adjustments for future STEM teaching. | |
Integrated STEM Education ID Model
The integrated STEM Education ID model (Figure 2) is presented in a clockwise procedure, representing the order of the six stages; however, teachers can go back to any previous stages if needed.
Figure 2
The Six-Stage Integrated STEM Education Instructional Design Model.
Small arrows inside the circular sector embodies each stage’s progress related to what is determined in the previous stage. Large arrows pointing clockwise along the top symbolize the characteristics of each stage. Team stands for a teacher team; Background refers to the conditions that must be considered; Context refers to authentic contexts; Experience indicates learning tasks that enable students to learn from their experience; Facilitator means the teachers’ roles; and Continuity highlights the features of the evaluation stage in which the results will feedback to previous stages.
Given this study aimed to propose guidelines for teachers to deliver integrated STEM education activities, the 25 tasks were organized as a checklist (Figure 3) that teacher teams should perform. In addition, teachers can also use the checklist to record their teaching process; in this case, the checklist could serve as one of the pieces of evidence in the evaluation stage when discussing improvement for the next activity.
Figure 3
Checklist: Integrated STEM Instructional Design
Conclusion
This study proposed an integrated STEM education model to provide teachers of integrated STEM education with precise guidance. The original model was initially theorized through an extensive literature review, followed by a pilot test conducted to confirm the contents of the model. Based on the results of the pilot test, a two-round Delphi study was conducted to attain agreement among a group of experts on stage definitions and appropriateness of tasks. In order to generate a model that can be applied in different cultural backgrounds, experts from different countries were invited to participate as panelists. The study resulted in an integrated STEM education ID model consisting of six stages and 25 tasks. Using the model, teachers can follow the six stages, including preparation, analysis, design, planning, implementation, and evaluation, to deliver an integrated STEM education learning activity. Moreover, the integrated STEM education ID checklist with the assigned tasks can provide teachers with instructions to assess the teaching process and address the crucial elements in each stage.
While the model offers a comprehensive framework for integrated STEM instruction, we acknowledge concerns raised in the past several years regarding safety issues in hands-on STEM education activities. Some unqualified teachers may hesitate to engage students in practical tasks due to low confidence in teaching technology and engineering content (Love & Love, 2023). Although our study did not specifically address safety considerations, we recognize its significance in STEM education. We suggest future research be conducted to explore this aspect further as a means of ensuring that integrated STEM education instructional design enhances interdisciplinary learning and incorporates best practices for maintaining a safe and effective learning environment.
Although the reliability and validity of the Delphi technique has been critiqued (Goodman, 1987; Williams & Webb, 1994; Walker & Selfe, 1996), efforts have been made to improve credibility using stability (Habibi, 2014; Mukaka, 2012) and face validity results (Al-araibi et al., 2019; Eubank et al., 2016). We conducted the pilot investigation to confirm the contents of the proposed model, and Kendall’s W and Spearman’s Rho were used to indicate the stability of the Delphi process. The present study contributes to interdisciplinary education by constructing an Instructional Design Model with potential stages and tasks that teachers might follow when conducting integrated STEM education in their classroom. In the future, researchers can further improve on the integrated STEM education ID model to meet practical situations by analyzing the checklist data collected from teachers.