Formal Assessment in STEM Higher Education: A General Analysis and Recommendations for Improvement

Introduction

A recent report by the National Science Foundation (NSF) promulgated the importance of developing classroom-based STEM assessments (Harris et al. 2023). This report highlighted the challenges that STEM educators, particularly those at the tertiary level, face when seeking to introduce STEM-based activities into their classrooms. Assessment is seen by educators, and researchers alike, as one of the main characteristics that define an educational system (Grangeat et al. 2021). So as the field of STEM education looks to map out a path forward it is incumbent upon educators, teachers, and researchers alike to help advance knowledge on effective assessment in the classroom. Due to the inherent interdisciplinary nature of STEM education, assessment in this area offers unique challenges that teachers of monodisciplinary subjects may not face. With this stated, our focus in this paper will be to discuss the role and importance of assessment in STEM classrooms and address the challenges of assessment when taking an interdisciplinary approach to STEM education (Gao 2020). Additionally, our focus will be on the formal assessment of STEM education at the tertiary level.

Formal assessment is viewed as an integral part of improving teaching and learning and serves as a key cog in the learning process (Idin 2020). Distinctly separate from summative assessment, formative assessment provides check-ins on students during the learning process and offers instructors agency in adjusting course content and instructional strategies (Idin 2020).

‘Assessment is designed to elicit students’ behavior to produce evidence that can be used to draw reasonable inferences about what students know’ (Harris et al. 2023, p. 13). Generally, teachers prefer authentic assessments in all types of instruction (including STEM) that can more accurately measure the learning process, including observation, performance assessment, self assessment, writing samples, feedback, checklists, and portfolios (McNair et al. 2003). ‘Identifying proper methods of formal assessment is key to enabling teachers’ ability to critically review students’ performance and make the necessary adjustments’ (Reynolds et al. 2020, p. 5). The current and most effective methods of formal assessment in STEM education are focused on criteria within the mono-disciplinary knowledge (MDK) and affective domains (MDA), as well as trans-disciplinary affective domains (TDA), as defined by Gao et al. (2020). Indeed, there exists barriers and challenges for the implementation of effective formal assessment, but we submit that many of these obstacles can potentially be overcome with the following strategies: 1) an increase in the use of innovative formative assessments in the classroom, 2) an increase in the use of alternative assessment strategies in formal assessment, 3) an increase in the use of emerging technologies in formal assessment, and 4) an improvement in the alignment of assessment with respect to learning goals and instruction. For example, according to Harris et al. (2020, p. 13), ‘the use of innovative formative assessments can significantly improve student learning experiences;’ these include the use of active learning environments, online simulated simulations, applying PBL and TBL learning, as well as increasing teacher professional development. The future of formal assessment is also exciting, as emerging technologies such as AI, VR, and SNA make for endless possibilities with respect to improving assessment methodologies and enhancing student learning experiences. Finally, both judgmental and statistical forecasting methods can potentially offer insights into (assessment) data through the use of specialty forecasting software packages (Sanders 2015).

Most frequent criteria applied in formal STEM assessment: MDK, MDA, and TDA

Findings from a study conducted by Gao et al. (2020) found that the most formative assessments in STEM education focused on assessments of monodisciplinary learning and transdisciplinary affective domains.

A dearth of formative assessments focused on interdisciplinary learning and practices. According to Gao et al. (2020) the ‘First Dimension’ disciplines include the main fields considered in STEM education such as science, technology, engineering, and mathematics, as well as their sub-fields such as physics in science and geometry in mathematics. ‘The first dimension concerns the nature of disciplines being assessed and includes three categories: mono-discipline, inter-discipline, and trans-discipline (Gao et al., 2020, p. 2).

According to Slavinec et al. (2019, p. 436), a monodisciplinary approach considers ‘teaching and learning from the standpoint of an individual discipline.’ It refers to the focus of learning in one subject area or subject-based teaching, such as only in mathematics or in engineering or in science. Essentially, it excludes learning more than one subject simultaneously. An interdisciplinary approach juxtaposes and integrates (parts of) two or more disciplines, focusing on establishing explicit connections between relevant disciplines (Miller, as cited in Gao et al. 2020).

His challenges of assessing interdisciplinary learning have been well documented (e.g., Herro et al. 2017). A trans-disciplinary approach refers to the unity of knowledge and skills beyond the disciplinary training (Nicolescu 2002). As Miller (1981, p.21) put it elegantly, ‘trans-disciplinary approaches are articulated conceptual frameworks which claim to transcend the narrow scope of disciplinary world views and metaphorically encompass the several parts of the material field which are handled separately by the individual specialized disciplines.’ Please see Figure 1 below from El Sayary (2021, p. 494).

Figure 1

According to Gao et al. (2020), the Second Dimension encompasses the learning objectives and includes four aspects: knowledge, skill, practice, and both the affective and cognitive domains.

The affective domain

‘The affective domain concerns constructs related to feelings and emotions, including communication, collaboration, self-direction, interest, attitudes, motivation, and values’ (Gao et al. 2020, p. 2). The psychomotor domain includes creativity, innovation, and problem-solving and deals with processes and skills that are physically performed, e.g. body movements and physical abilities. Many have argued that a significant outcome of an interdisciplinary STEM education approach is students’ positive enhancement in the affective domain (Gao et al. 2020).

The affective domain in our literature review includes measures such as students’ interest, engagement, attitude, and motivation for STEM contents and practices and career aspiration for STEM professions.

The cognitive domain

The cognitive domain includes critical thinking and independent learning. It is a basic learning objective of STEM education and is aimed at helping students develop content knowledge for one or more specific disciplines within STEM. For instance, a common approach in STEM education is to use engineering design to help students develop math and/or science knowledge (Becker & Park 2011). In this approach, one (or more) specific discipline(s) within STEM (e.g., math and science) is prioritized and the other discipline(s) (e.g., engineering) serves as a vehicle to deliver that discipline. Another common learning objective of STEM education within the cognitive domain is to help students develop skills that go beyond a single discipline.

According to Gao et al. (2020), results show that most assessments are focused on mono- disciplinary knowledge (MDK) and affective domains (MDA), as well as trans-disciplinary affective domains (TDA). However, it also clearly states that, ‘STEM learning processes and practices are complex and manifold, especially when one considers the many different features of disciplinary processes and practices’ (Gao et al. 2020, p. 2).

Please see Figures 2 and 3 below for reference.

Figure 2

Figure 3

Therefore, constructing assessment techniques as outlined in Figure 2 above such that they focus their criteria around mono-disciplinary knowledge (MDK) as well as mono- and trans-disciplinary affective (MDA and TDA) learning domains are the most common assessment techniques used within the field of STEM Education.

Recommendation #1

Increase the use of innovative formative assessments in the classroom. Innovative assessment techniques can include, but are not limited to, open book exams, generative AI assessment, case study-based assessment, practical assessment with real world problem solving, video interviewing, and online viva (allowing students to demonstrate their knowledge and skills in front of a panel of experts). Pedagogical research has shown that employing innovative assessment techniques can be effective at improving student learning, which encourages academic identity and self-efficacy in STEM students which in turn helps build competencies that go beyond technical skills. Specifically, research from Harris et al. (2023, p. 13) clearly demonstrates that the ‘use of innovative formative assessments can significantly improve student learning experiences.’ According to Wiggins (1998, p. 7), the aim of assessment should be ‘to educate and improve student performance, not merely to audit it.’ Finally, the twenty-first century model of competency skills recognizes the multifaceted nature of STEM learning and seeks to prepare students for the complex challenges of the future. Thus, the increased use of innovative formal assessment techniques in the classroom can ameliorate the student learning experience, prepare students for future real-world challenges, and potentially help to further refine the formal assessment process.

Recommendation #2

Increase the use of alternative assessment strategies in formal assessment. These include portfolios, presentations, and performance-based assessments, which offer a more comprehensive view of students’ abilities. ‘Integrating alternative assessments recognizes students’ diverse talents and strengths, moving beyond a one-size-fits-all approach to evaluation’ (Chen 2006; Maki 2023). Recent developments in STEM education have shifted towards more student-centered and experiential learning approaches (Keiler 2018; Manduca et al. 2017). For example, project-based learning (PBL) has become a dynamic strategy, offering students opportunities to apply theoretical knowledge in real-world contexts (Kwietniewski 2017). Also, inquiry-based learning (IBL) encourages curiosity-driven exploration, fostering a deeper understanding of STEM concepts through hands-on experimentation (Gaylor 2017). ‘These approaches enhance content retention and cultivate critical thinking and problem-solving skills’ (Mohd Al Hamad et al. 2024, p. 244). Therefore, alternate assessment strategies are complemented by these new approaches in learning, e.g. PBL and IBL, which promote the twenty-first century model of competency skills; this combination in turn directly enhances the learning process.

Recommendation #3

Increase the use of emerging technologies in formal assessment. ‘Emerging technology allows for the integration of student responses over time, space, and modalities to paint a richer picture of student abilities’ (Harris et al. 2023, pp. 15–16). Futhermore, emerging technologies enable data to be recorded and analyzed in finer-grained detail. This capability has the potential of enhancing understanding of the operations and behaviors students engage in when creating products of assessment. Coupled with technology-driven stimuli, such as simulations, ‘this log data can reveal what they (the students) are doing and when they are doing it, providing deeper evidence from which to infer proficiency’ (Ercikan & Pellegrino 2017, as cited in Harris et al. 2023, p. 16). ‘Emerging technological capabilities also hold promise for the analysis of less structured data, such as data associated with constructed response questions where students may be expressing their ideas in written and/or graphical form as an argument or explanation. The processing and analysis of rich, fine-grained multimodal data is likely to be greatly enhanced with emerging AI and associated machine-learning techniques’ (Harris et al. 2023, p. 16). Research and development efforts have already demonstrated reliable automated scoring of short written constructed responses for various topics and content in science and other subjects (Beggrow et al. 2014). ‘Incorporating (emerging technologies) gamification, simulations, and interactive platforms has further enriched the learning experience, making it more engaging and relevant to students’ lives’ (Mohd Al Hamad 2024, p. 4). Future work is likely to utilize machine learning to allow researchers to analyze more complex response data of the type described above (Zhai 2021). Thus, the extant research clearly demonstrates that emerging technologies have significant potential to help create more effective formal assessment techniques through increased understanding of the data behind the ‘how’ and ‘why’ of student behavior and proficiency.

Recommendation #4

Improve the alignment of assessment with respect to learning goals and instruction, by reducing the imbalance among the three components of the Alignment Triangle (see Figure 4 below); this improved alignment can potentially optimize the efficacy of formal assessment in STEM education. There is no doubt that formal assessment techniques have a pivotal role to play in effective teaching and learning in the STEM classroom. This role can be optimized when assessment is integrated into the flow of classroom instruction and used formatively. ‘An assessment should measure what is taught through instruction and what is taught should be drawn from well-conceived learning goals; however, it is anything but a simple process to achieve coherence across the three. All too often, there is an imbalance that results in these three components working at cross-purposes’ (Harris et al. 2023, p. 25). ‘The continued expansion of what students are supposed to know and be able to do provides an opportunity for technology to enhance the observation and interpretation components of the assessment (alignment) triangle’ (Harris et al. 2023, p. 16). Please see The Alignment Triangle in Figure 4 below.

Figure 4

Theories of learning serve as the central core for ensuring that the components are synchronized correctly. Thus, ‘theories of learning can be articulated with such specificity so as to support teachers’ instructional decision-making, and thereby present a viable solution for aligning learning goals, instruction, and classroom assessments with the intention of improving students’ opportunities to learn’ (Harris et al. 2023, p. 26). Thus, in order to ‘realize the use of (formal) assessments and assessment data to effectively advance STEM learning, a tight coupling of assessment with learning goals and instruction, will be required’ (Harris et al. 2023, p. 23). In other words, ‘(assessment) feedback needs to be timely and constructive. Successful assessment implementation involves a responsive teaching approach, fostering a dynamic and adaptive learning environment (Clark 2012; Furtak et al. 2019; Moreno 2018).

Recommendation #5

Increase research on how formal assessment actually influences student learning. ‘Although formative assessment is regarded as a promising way to improve teaching and learning, there is considerable need for research on precisely how it influences student learning and professional development’ (Rakoczy et al. 2019, p. 154). Dunn and Mulvenon (2009) stated that there is really a limited body of scientifically based empirical evidence to support the fact that formative assessment directly contributes to positive educational outcomes in STEM education. ‘A key concern needing investigation is to what extent do such tests gauge adaptive expertise (as opposed to routine expertise)? To what extent are such measures coherent and gauge the role of prior knowledge, including informal knowledge, in constructing meaningful knowledge’ (Baroody et al. 2024, p. 13). Like learning progression-based instruction, although formative assessment (in STEM education) is frequently recommended and used, relatively little rigorous or high-quality research supports its efficacy (Harris et al. 2023, p. 50). Logically, with an increased understanding of how formal assessment (through further research) can affect student learning, both learning outcomes and the efficacy of formal assessment can be improved.

The future of assessment

AI can potentially gauge if the student’s work is accurate, well-written, creative, and communicates its intended message effectively; also based on AI’s knowledge of research, AI can determine what makes a good presentation and what constitutes a project that is completed in a fully comprehensive manner.

The application of AI technology in STEM education is confronted with the challenge of integrating diverse AI techniques in the complex STEM educational system. Grounded upon a General System Theory (GST) framework, this research reviewed the empirical AI-STEM studies from 2011 to 2021 and proposed educational, technological, and theoretical implications to apply AI techniques in STEM education. Overall, the potential of AI technology for enhancing STEM education is fertile ground to be further explored together with studies aimed at investigating the integration of technology and educational system’ (Xu et al. 2022, Conclusion Section).

General system theory (GST) is a theoretical framework, arguing that the world is composed of different organic systems, which contain dynamically interacting elements and mutual relationships between them (Rapoport 1986). See Figure 5 below, the frequency of AI application categories in different learning contents up to the year 2022.

Figure 5

VR can help assess students’ skills with various abilities, by giving scores for various specific skills during a simulation of a various game, which tests different skill sets.

Increasingly, researchers are using Social Network Analyses (SNA) to characterize school culture and collaboration characteristics of teachers involved in professional development sessions. More recently, SNA has been used to understand STEM teaching and learning, and to suggest directions for future research (Alonso-Yanez, G. et al. 2016, Abstract).

The Implementation of Social (Statistical) Network Analysis (SNA) as it pertains to student performance could be applied to help with more effective assessment methods? The integration of SNA could help develop a unique assessment matrix for a specific STEM subject based on the corresponding data and conclusions of the SNA analysis for a given population or cross section of students with different abilities, for example.

Furthermore, there are new advances in forecasting, which can help predict behavior of groups of people, and this could theoretically be applied to analysis with regard to STEM education. Forecasting research activity is a complex task that involves different fields. These new applications could include: 1) data retrieving to retrieve scientific publications related to an emerging technology and generate a refined data set based on them, 2) text mining to structure and clean the input text, in order to obtain a cleaned data set to be analyzed, 3) bibliometrics to generate time series based on the analysis of the content of the data set. These series represent the research activity associated with different sub-technologies within the emerging technology and enable it to be depicted over time, 4) time series modeling to specify appropriate models for the time series using the structural time series modeling methodology, and 5) time series forecasting to obtain forecasts of the short-term development of the research activity related to the technology (Harvey, 1991).

Conclusion

This paper is keenly focused on formal assessment of integrative STEM activities at the tertiary level. In this paper we have outlined the need and importance of formal assessment, provided a view into the occurrence of formal assessment in monodisciplinary, interdisciplinary, and transdisciplinary classroom settings. In addition this paper has provided a glimpse into the literature in regards to the distinction between the assessment of cognitive, psychomotor and affective domains which should be a consideration for any effective educator. With this stated we understand the challenges that instructors face when performing formal assessments of integrative STEM activities and have acknowledged these challenges with reference to the latest literature.

In an effort to add to the budding literature on informal assessment we have provided literature-based recommendations on how educators can improve their formal assessment practices. This includes introducing innovative strategies for formal assessment, using alternative methods of formal assessment, use of emerging technologies in formal assessment, improving the alignment of formal assessment with explicit learning goals and instruction, and finally increasing research on informal assessment’s role in student learning. We conclude by discussing the future of formal assessment in integrative STEM, specifically targeting AI’s influence, VR’s emergence, how incorporating SNA can provide a more comprehensive approach to formal assessment, as well as the use of both judgmental and statistical forecasting methods.

Going forward, it is important to note that providing effective feedback through formal assessment should begin by first introducing effective pedagogical approaches. It is our recommendation that developing an effective framework for teachers of integrative STEM is of paramount importance; this brings us to the introduction of the concept of a 5Cs Framework.

STEM Educators have defined the 5Cs Framework which are critical to the learning and assessment process. They can be defined as follows: consistency, collaboration, cognition, conception, and creativity. These 5Cs criteria were found to be effective in the assessment process for postgraduate course introductory programming within the realm of STEM Education over three consecutive terms (Tom 2015). The 5Cs listed above thus refer to an assessment-based framework. However, there also exists a 5Cs Framework that refers to an optimized student-centered teaching pedagogy and is expounded upon below.

Critical Thinking => teaching students how to analyze problems logically and then devise solutions as well as strategies to work around the obstacles.

Communication => students learning resources and put through situations in a safe, interactive, and supervised environment where the student learns how to interact with their peers.

Collaboration => students learn how to work with peers and navigate through issues in a well-thought manner, participating in team projects and exposing them to an interactive learning environment.

Creativity => students learn to come up with new creative solutions to solve real-world problems.

Character => teaching students to develop good character traits rather than just focusing on grades, assignments, tests, or covering lessons (https://goglobalways.com/blog/steam-focused-education-5cs-benefits/).

Thus, the 5Cs framework may therefore be seen to provide a model for assessment and optimized student-centered teaching pedagogy. In any area where it is determined through assessment that there are weaknesses in the 5Cs, there can be improvement in pedagogical effectiveness. In other words, any learning method or teaching method that can focus on improvement of performance in any area of the 5Cs would be helpful to improve the learning process. For example, integrating the 5Cs framework into the assessment process as well as teaching pedagogy within the curriculum can potentially make teaching more effective.

More clearly defined learning outcomes can potentially help student performance improve as students are more aware of the expectations for learning. By clearly defining learning outcomes, teaching can be more effective, and we can potentially learn more from the assessment process. If we can increase the frequency and depth of involvement of the 5C’s in the assessment process as well as pedagogy within the curriculum, then students can be more effective learners.

Competing Interests

The author has no competing interests to declare.