Table 1
Panelist Information for Pilot Investigation.
| Panelists | Country | Profession/expertise |
| 1 | Australia | STEM, PCK |
| 2 | Hong Kong | STEM, technology education |
| 3 | Korea | STEM, teaching strategies |
| 4 | Netherlands | STEM, the pedagogy of technology and design |
| 5 | New Zealand | STEM, innovation in teaching |
| 6 | New Zealand | STEM, teaching and learning strategies |
| 7 | Japan | STEM, teaching and learning strategies |
| 8 | Taiwan | STEM, teaching methods |
| 9 | United States | STEM, pedagogical practices among STEM education areas |
Table 2
Panelist Information for the Delphi Research.
| Panelists | Country | Profession/expertise |
| 1 | Australia | Technology education, STEM, mentoring beginning teachers |
| 2 | Hong Kong | Science education, STEM, professional development |
| 3 | New Zealand | Technology education, STEM, teaching approaches |
| 4 | Japan | Engineering education, STEM, creativity and design thinking |
| 5 | Taiwan | Engineering education, STEM, emerging technology in education |
| 6 | Taiwan | Engineering education, STEM, teacher education |
| 7 | Taiwan | Mathematics education, STEM, teaching/learning, pedagogy |
| 8 | Taiwan | Mathematics education, STEM, teaching/learning |
| 9 | Taiwan | Science education, STEM, teacher education |
| 10 | United States | Science education, STEM, teaching/learning |
| 11 | United States | Technology education, STEM, teaching/learning |
| 12 | United States | Technology education, STEM, teacher education |
Table 3
The Pilot Investigation Questionnaire.
| Stages | Definitions and Tasks |
| Preparation | |
In the preparation step, teachers are going to organize a team of STEM teaching to discuss and prepare courses together. The STEM teaching team should include teachers from different disciplinary; in this case, they can use their profound knowledge to teach the content professionally and enable students to solve real-world problems relying on the knowledge during the courses.
| |
| Analysis | |
In the analysis step, teachers should analyze students’ culture capital and teaching resources, such as students’ computers and other technology tools, before starting to design activities. Moreover, the most important thing in the analysis step is that the teacher must define the value of the STEM course first. For example, the purpose of the STEM course is to cultivate students’ transformative competencies in solving complex problem, which will be the main value of this STEM course; therefore, teachers should discuss how to design activities and decide the learning objectives to achieve this goal.
| |
| Designing | |
In the designing step, teachers should arrange a context in the STEM activities to provide students an opportunity to utilize the science and math concepts and to learn how to solve real-world problems.
| |
| Planning | |
There are gaps with teachers’ expectation when students follow the problem-solving process or engineering design process during the courses. For example, when students encounter problems, they may try to avoid failure, and this could cause losing learning opportunities and experiences. According to Moore et al. (2014), it is necessary of STEM education to provide students with opportunities to learn from their mistakes and deepen what they have learned. This can be tracked back to the theory of learn-by-doing (Dewey, 1938), which not only emphasizes hands-on, inquiry, and practical experience, but also believes that the reflective experience is also necessary (Schön, 1991). Therefore, in the planning step, teachers should plan to arrange hands-on, practical experience, and reflective experience for students to gain meaningful experiences.
| |
| Implementation | |
The engineering design process is regarded as a gateway for students to convert science and math knowledge into usable abilities. Even though the engineering design process can be a specific procedure that students follow, teachers should still provide guidance and assistance properly to ensure students will learn from their experience and will not encounter unsolvable difficulties. Thus, in the implementation step, teachers should implement STEM activities through engineering design process to give a specific procedure for students when they are trying to solve ill-defined problems, and teachers should be good guides and facilitators.
| |
| Evaluation | |
Dewey (1938) stated that, experience involves continuity, which means experiences will influence one’s future experiences and decisions. With this principle, in the evaluation step, teachers should understand students’ learning achievement through using assessment tools and help students accumulate continuity experiences of interdisciplinary. The evaluation results will be fed back to all previous steps to provide evidence, which can help teachers reschedule the course next time.
|
Table 4
The Delphi Study Questionnaires.
| Stages | In round 1 | In round 2 |
| Preparation | ||
| Pre1 | Organize a team of teachers who are competent at interdisciplinary STEM integration as a consulting group. | Organize a team of teachers, including teachers from different disciplines, as a consulting group (at least one for each discipline). |
| Pre2 | Reach consensus on teaching to make sure the teachers who involved in are familiar with STEM education. | including teachers from different disciplines, as a consulting group (at least one for each discipline). Share STEM education teaching strategies to ensure the STEM Teacher Team has a common understanding of interdisciplinary integration. |
| Pre3 | Schedule a sequence of meeting times for joint discussion of planning STEM activities. | Schedule a sequence of meeting times for joint discussion of planning STEM activities. |
| Analysis | ||
| A1 | Analyze students’ prior and pre-requisite knowledge and skills. | Analyze students’ prior and pre-requisite knowledge, attitude, and skills which needed in STEM activities. |
| A2 | Analyze the environment, including teaching resources (i.e. classroom setting and technology devices or tools) and features of the school. | Analyze the environment, including teaching resources (i.e., classroom setting and technology devices or tools) and the features of schools to better know what context, activities, or materials can be used while teaching STEM. |
| A3 | - | Analyze what is authentic to students to solve or address. |
| Design | ||
| D1 | Determine the learning objectives and teaching strategies. | Determine the learning objectives. |
| D2 | - | Determine the pedagogical approaches (how teachers are going to integrate their different philosophies and teaching styles to support students’ learning). |
| D3 | Design the criteria and assessment tools (formative/summative) for a particular activity for assessing students’ competencies. | Design the criteria and assessment tools (formative and summative) or have students design a portfolio for assessing their knowledge, attitude, and skills. |
| D4 | Analyze the role of science, mathematics, technology, and engineering, and identify the relationship of each subject or how to bring them together. | Analyze the role of science, technology, engineering, and mathematics and their contribution to the activities, and identify the relationship of each discipline or how to bring them together. |
| D5 | Identify and design a student-centered context which is authentic to students and relate to their life experience. | Identify and design a student-centered context which is authentic to students and relate to their life experience. |
| D6 | Design an activity based on the context to encourage students to integrate and learn new knowledge and skills. | Design a series of projected-based/inquiry-based/problem-based activities based on the context to force students to integrate and learn new knowledge and skills. |
| Planning | ||
| P1 | Develop learning contents based on the activity and the relationship among the subjects should be emphasized. | Develop learning contents based on the learning objectives, context and the relationship among the disciplines should be emphasized. |
| P2 | Design and plan learning tasks which can guide students to utilize different knowledge and competency of STEM subjects in STEM activity. | Arrange structured learning tasks which are able to allow or challenge students to utilize and develop knowledge and competency of the different disciplines in STEM activities. |
| P3 | Propose a sequential teaching process, including lecturing, demonstrating, and evaluation, etc. | Create an environment where students are supported with necessary learning resources, including a sequential teaching process, necessary equipment, materials, manuals or guidance, etc. |
| P4 | - | Decide work assignment and teaching schedule for the members in the STEM Teacher Team. |
| Implementation | ||
| I1 | Follow the teaching process determined in previous steps to deliver courses. | Follow the pedagogical approach determined in previous steps to deliver courses. |
| I2 | Introduce different design processes as guidance for students to construct a framework, picture or flow when solving the ill-defined problems. | Provide students opportunities to design their own learning sequence. |
| I3 | Use questioning techniques to stimulate students to find out the responses to the challenges they faced. | Prompt students to think for themselves by the use of questioning skills. |
| I4 | Provide timely feedback to students during the design process. | Provide scaffolding for students who have problems while learning. |
| I5 | Implement assessment designed for the activity and provide opportunity for students to do reflection. | Evaluate students with formative and summative assessment or have them design and make a portfolio to allow reflection to happen. |
| Evaluation | ||
| E1a | Assess students with designed criteria. | - |
| E1b | - | The evaluation step should be iterative in the teaching process. |
| E2 | - | Include multiple types of assessment. |
| E3 | Analyze and discuss the challenges and progress in the learning process with students to accumulate continuity of interdisciplinary utilizing. | The evaluation needs to reflect students’ learning outcomes to allow them to know what they learned. |
| E4 | Review the steps of the PADPIE instructional design through the evaluation results and make suggestions for revision. | Through the evaluation, teachers are able to propose adjustments for future STEM teaching. |
Figure 1
Research Implementation Phases.
Table 5
Delphi Process Results: Rating Consensus.
| Stage/Task | Round 1 | Round 2 | ||||||
| M | Mdn | SD | IQR | M | Mdn | SD | IQR | |
| Preparation | Preparation | |||||||
| Pre1 | 4.17 | 4.00 | 0.94 | 1.00 | 4.50 | 4.50 | 0.52 | 1.00 |
| Pre2 | 4.25 | 5.00 | 1.14* | 1.00 | 4.92 | 5.00 | 0.29 | 0.00 |
| Pre3 | 4.17 | 4.00 | 0.94 | 1.00 | 4.33 | 5.00 | 0.98 | 1.00 |
| Analysis | ||||||||
| A1 | 4.67 | 5.00 | 0.49 | 1.00 | 4.75 | 5.00 | 0.45 | 0.75 |
| A2 | 4.42 | 5.00 | 0.79 | 1.00 | 4.83 | 5.00 | 0.39 | 0.00 |
| A3 | – | – | – | – | 4.33 | 4.00 | 0.65 | 1.00 |
| Design | ||||||||
| D1 | 4.83 | 5.00 | 0.39 | 0.00 | 5.00 | 5.00 | 0.00 | 0.00 |
| D2 | 4.92 | 5.00 | 0.29 | 0.00 | 4.92 | 5.00 | 0.29 | 0.00 |
| D3 | 4.75 | 5.00 | 0.45 | 0.75 | 4.67 | 5.00 | 0.49 | 1.00 |
| D4 | 4.92 | 5.00 | 0.29 | 0.00 | 5.00 | 5.00 | 0.00 | 0.00 |
| D5 | – | – | – | – | 4.50 | 5.00 | 0.67 | 1.00 |
| D6 | 4.83 | 5.00 | 0.39 | 0.00 | 4.92 | 5.00 | 0.29 | 0.00 |
| Planning | ||||||||
| P1 | 4.33 | 4.50 | 0.78 | 1.00 | 4.83 | 5.00 | 0.39 | 0.00 |
| P2 | 4.83 | 5.00 | 0.39 | 0.00 | 4.75 | 5.00 | 0.62 | 0.00 |
| P3 | 4.33 | 4.00 | 0.65 | 1.00 | 4.58 | 5.00 | 0.51 | 1.00 |
| P4 | – | – | – | – | 4.67 | 5.00 | 0.49 | 1.00 |
| Implementation | ||||||||
| I1 | 4.00 | 4.00 | 0.85 | 0.75 | 4.25 | 4.00 | 0.87 | 1.00 |
| I2 | 3.92 | 4.00 | 1.00 | 2.00* | 4.50 | 5.00 | 0.67 | 1.00 |
| I3 | 4.50 | 5.00 | 0.67 | 1.00 | 4.83 | 5.00 | 0.39 | 0.00 |
| I4 | 4.33 | 5.00 | 0.98 | 1.00 | 4.83 | 5.00 | 0.58 | 0.00 |
| I5 | 4.58 | 5.00 | 0.67 | 1.00 | 4.92 | 5.00 | 0.29 | 0.00 |
| E1a | 4.50 | 5.00 | 0.67 | 1.00 | – | – | – | – |
| E1b | – | – | – | – | 4.58 | 5.00 | 0.51 | 1.00 |
| E2 | – | – | – | – | 4.58 | 5.00 | 0.79 | 0.75 |
| E3 | 4.58 | 5.00 | 0.67 | 1.00 | 4.83 | 5.00 | 0.39 | 0.00 |
| E4 | 4.67 | 5.00 | 0.49 | 1.00 | 4.83 | 5.00 | 0.39 | 0.00 |
| Kendall’s W | 0.326 (p<.01) | 0.180 (p<.01) | ||||||
[i] Note1. E1a was combined with D6 in Round 2; E1b was a new task added in Round 2
Note2. * Did not meet the criteria
Table 6
Delphi Process Results: Task Consensus
| Stages | Tasks |
| Stage 1: Preparation | |
| Organize a STEM Teacher Team, including teachers from different disciplines, as a consulting group (at least one for each discipline). Share STEM education teaching strategies to ensure the STEM Teacher Team has a common understanding of interdisciplinary integration. Identify the role of each teacher and schedule a sequence of meeting times for joint discussion of planning STEM activities. | |
| Stage 2: Analysis | |
| Analyze students’ prior and pre-requisite knowledge, attitude, and skills which are targeted for assessment in the planned STEM activities. Analyze the environment, including teaching resources (e.g., classroom setting and technology devices or tools) and the features of schools to better know what context, activities, or materials can be used while teaching STEM. Analyze what the students would consider to be an authentic problem. | |
| Stage 3: Design | |
| Determine the learning objectives. Analyze the role of science, technology, engineering, and mathematics and their contribution to the activities, and identify the integration of the disciplines. Design a student-centered context which is authentic to students and relate to their life experience. Design a series of project-based/problem-based/design-based/inquiry-based activities to facilitate integration of new knowledge and skills. Determine the pedagogical approaches arise from the nature of the activity (how teachers are going to integrate their different philosophies and teaching styles to support students’ learning). Design the assessment criteria and tools (formative and summative) or have students develop a portfolio for assessing their knowledge, attitude, and skills. | |
| Stage 4: Planning | |
| Develop learning contents based on the learning objectives, context (problems that students need to solve) and the relationship between the disciplines should be emphasized. Arrange structured learning tasks which are able to allow or challenge students to utilize and develop knowledge and competency of the different disciplines in STEM activities. Create an environment where students are supported with necessary assistance, including a sequential teaching process, necessary equipment, materials, manuals or guidance, etc. Decide work assignment and teaching schedule for STEM Teacher Team members. | |
| Stage 5: Implementation | |
| Follow the pedagogical approach determined in previous stages to deliver courses. Allow students to determine their next action (dynamic and iterative process) based on their findings and progress. Prompt students to think for themselves by the use of questioning skills. Provide scaffolding for students who have problems while learning. Evaluate students with formative and summative assessment and have them develop a portfolio to record their reflection. | |
| Stage 6: Evaluation | |
| Repeat over time in a teaching process to provide feedback for other stages. Include multiple types of assessment to fit the various learning objectives. Provide students an opportunity to reflect their learning outcomes and allow them to know what they learned. Propose adjustments for future STEM teaching. | |
Figure 2
The Six-Stage Integrated STEM Education Instructional Design Model.
Figure 3
Checklist: Integrated STEM Instructional Design