Analysis of test items in school-based assessment and National Achievement Survey with reference to learning outcomes in science at the middle stage

The quality of learning in elementary education has always been a focal point in Indian Education Policy, subsequently mentioned in the 12th 5-year plan, aligning with the commitment of Sustainable Development Goal 4, particularly target 4.1. To ensure that the quality of learning has been met, regular learning assessment at the elementary level has been prioritized (NCERT, 2017). Continuous tests not only help the learners to identify their learning difficulties but also allow the teachers and administrators to recognize critical information to maintain quality education. These test habits of students and item preparation exercises of teachers are equally crucial indicators for exploring the efficacy of elementary education across India. For a systematic evaluation, in every 3 years, a massive assessment drive is carried out in whole India through the periodical National Assessment Survey (NAS) tests. However, most of the teachers at the elementary level in India are not much aware of the criteria or curricular expectations against which students need to be assessed and are assessed through NAS. These learning expectations are provided by objectives, more precisely by instructional objectives that in turn are translated into learning outcomes (LOs). Further, the assessment of students LOs is a critical component of an effective education system and in the context of science education, well-designed assessments that measures the students’ all-round development. The purpose of assessment is to determine whether the expectations (LOs) match the standards set by schools or national-level educational administration (Chen et al., 2021). Keeping in view that learning always operates in continuum, NCERT, India, has developed a document that includes LOs in all the curricular areas at the elementary stage (LOs at the Elementary Stage, NCERT, 2017). These LOs also find linkage with the curricular expectations and the pedagogical processes. These serve as the crucial indicators of what students are expected to know, understand, and apply by the end of the learning. The expected LOs have been developed classwise (NCERT Science text book for class VIII), to use those as evaluation criteria to guide the assessment. It is imperative that criteria used in the evaluation and assessments should be equivalent to the criteria used in the formation of a curriculum.

In recent years, the educational paradigm has shifted from merely rote memorization to the development of higher-order thinking skills focusing on competency-based education. This necessitates a robust system for designing and evaluating LOs, for which taxonomies are frequently used to maintain the standard of education (Nursaban et al., 2019). Taxonomies mediate a common language between learning–teaching–evaluation activities and keep alert all those involved in the transaction of curriculum, namely students, teachers, evaluators, educational administrators, and planners. The taxonomy has been developed by Bloom to categorize the LO according to the logic of cognitive, affective, and psychomotor domains (Bloom et al., 1956). However, because of the changes in cognitive psychology, meta-cognitive knowledge has been introduced in the original Bloom’s taxonomy. As a result, a more comprehensive and all-encompassing Revised Bloom has been developed.

The present study intends to make a comparative analysis of LOs in teacher-made questions of class VIII particularly in the subject science along with the specified LOs mentioned in the National Achievement Survey (NAS) Questions of class VIII science curriculum. The preliminary reports show that teachers prefer using items prepared by themselves more often than the standardized ones. As in contemporary science examination practices particularly in the elementary level, there exists an expectation that teachers who are recently trained with the latest pedagogical practices that aligned with achieving the LOs should align their assessment items with the specified LOs (Aristeidou et al., 2020). Further, the study aims to explore the intended LOs of class VIII science curriculum with reference to Revised Bloom’s Taxonomy, the alignment of the Questions prescribed in NAS and teacher-made tests with the intended LOs and intercompatibility between the teacher-made test items used for frequent and regular examinations at class VIII level science with standardized test items developed for NAS. NAS is a large-scale survey of student’s learning competencies, undertaken by the Ministry of Education, Government of India. The assessment framework is designed by NCERT to assess student’s capabilities in relation to learning objectives. Since 2017, the LO-based test items are executed to evaluate children’s progress and learning competencies as an indicator of the efficiency of the education system. Hence, it is imperative to find the interdependency of such elements of our curriculum.

By employing Revised Bloom’s Taxonomy as an analytical framework, this study provides empirical evidence to offer practical recommendations for understanding and analysing the question patterns and developing LOs and assessment items. Along with this, the study reveals that the NAS assessment increasingly supports higher-order thinking skills that can potentially develop the LOs. The findings have notable implications for further assessment reforms and for successfully inclining competency-based education in India.

The relationship between curriculum-defined LOs and teacher-made assessment practices has been extensively studied in elementary education, particularly in the field of science education. Revised Bloom’s Taxonomy has shown the importance of aligning assessments with clearly defined cognitive levels from remembering to higher order thinking skills such as creating and evaluating (Anderson and Krathwohl, 2001). Internationally, studies by Zorluoglu (2020) on LOs and Revised Bloom’s Taxonomy have demonstrated that for every cognitive process there exists an LO which was not homogenously distributed across the categories of cognitive process dimensions. Further, he analysed that the LOs are in conceptual knowledge dimensions and the questions included is mostly in factual knowledge dimensions, which creates a huge gap in the assessment and making it difficult to achieve the LOs. In the Indian context, the learning outcomes for science education at the elementary stage are predetermined by NCERT (2017), yet studies indicate persistent challenges in the implementations as there is lack in the teacher training and proper understanding of Revised Bloom’s taxonomy (Ahmadshah, 2019).

Research on assessment practices in elementary science education reveals a persistent gap between the intended LOs and the actual teacher-made classroom assessment. The distinction between testing and assessment can be made as assessment is the broader set of comprehensive processes of evaluating the student holistically, whereas testing is a more specific and structured event that measures the performance of the students against a pre-determined criterion (Brown, 2019). Teacher-made assessment conducted in elementary schools, consistently shows a predominance of lower-order cognitive questions which is mostly focused on factual recall type questions rather than applying and analysing types of test items which has emphasized on standardized tests (Jansen & Möller, 2022). This aligns with the international findings from PISA (Program for International Student Assessment) and TIMSS (Trends in International Mathematics and Science Study) assessment, which highlights the challenges of non-alignment of curriculum expectations with the classroom assessment practice (Ehren, 2022). The rapid demand for skill and competency-based workforce enhances the requirement of reinforcing LO-based assessment, which further emphasizes the importance of developing competency among students. This can be achieved by reflecting both LOs and learning process in the assessment process (Ramona & Bran, 2014).

The Revised Bloom’s taxonomy again provides valuable insights for analysing the science LOs. These LOs are derived from the curriculum expectations, and they demand teachers and stakeholders to direct the learning process in a desired manner and make them responsible and alert towards their role for ensuring quality education (Koireng, 2019). Although India’s engagement in science promotion activities is diversifying, this is far away from the developed nations. There still exists a lacuna in people’s passion for science and achieving the LOs in science (Mochahari, 2013). The NAS results provide solid evidences that there exists significant challenges in assessment in the field of science education. While in curriculum the LOs like “conducting simple scientific investigations” or “applying scientific concepts to daily life” has been emphasized still students’ performance remains weakest, particularly in application-based areas. Improved alignment of LOs with assessment could significantly enhance the science learning at the elementary level (Hailikari et al., 2022).

The literature finds several requirements such as better teacher and stakeholders’ understanding of science LOs, analysing and utilizing the standardized tests by NAS in the classroom assessment by the teacher, professional development in designing assessments aligning with the LOs and systematic support for improvement of competency-based assessment. The review underscores that while most of the work has been done to define the science LOs for elementary grades, there still exist considerable challenges that are mostly undiscovered. The study intends to address these gaps by systematically comparing intended, assessed, and achieved LOs in class VIII Science in Indian Elementary Science Education.

The objectives of the study are as follows:

• To examine the LOs in the science curriculum of NCERT with reference to the Revised Bloom’s Taxonomy.

• To analyse the test items of National Achievement Survey (NAS), 2017, 2021 with reference to LOs as prescribed by NCERT (2017).

• To analyse the Teacher-Made Questions with reference to the LOs as per NCERT (2017).

• To compare the NAS Questions and Teacher-Made Questions with reference to LOs.

• To compare the NAS performance of students in science at the state and national levels in class VIII.

The study employed a comprehensive mixed-method approach with a systematic document analysis through secondary sources, quantitative methods to examine the alignment between prescribes LOs, teacher-made assessments, and NAS for class VIII science. Secondary sources used for document analysis are mainly NCERT’s LOs (2017) and test items of NAS Framework, classifying all 58 prescribed outcomes using Revised Bloom’s Taxonomy to form a baseline for intended cognitive levels. For teacher-made assessment, a sample of 300 test items of class VIII science course, which were already executed for the year 2020–2021, was selected randomly from three different schools, respectively, i.e. Kendriya Vidyalayas of Bhubaneswar region, Demonstration & Multipurpose School of Bhubaneswar and Utkal University High School, Bhubaneswar. The selection of the schools was based on certain criteria they are, these schools show a regular implementation of the continuous and comprehensive evaluation pattern as mandated by the National Curriculum framework, and they ensure that regular teacher-made questions are frequently used. The inclusion of both CBSE and state board schools serves as an important comparative factor. While the first two schools follow the CBSE curriculum directly, the Odisha state school was deliberately selected as its textbooks have been normalized with the NCERT Publications. This normalization provides an appropriate base for comparing the LOs while maintaining the curriculum consistency.

The comprehensive data were collected from the documents of NAS 2017 and 2021 for class VIII from the NAS Reports published by the Ministry of Education, Government of India. Along with the state Executive Summary Report for the same years, prepared by the Directorate of TE and SCERT, Odisha. From these documents, a complete list of LOs used in NAS assessments at national and state levels, performance percentage for each LO, and the specific LO codes and outcomes were extracted.

The LO code and LOs for class VIII (science) taken for developing 25 numbers of NAS test items for 2017 and 2018 are given in Table 1.

In this section, first of all, the analysis of the LOs as identified from the NCERT curriculum for class VII and class VIII was carried out. The analysis has been carried out according to cognitive process dimension on RBT and is presented in Figures 1 and 2.

Figure 1

LO distribution in class VII, NCERT curriculum, 2017.

Figure 2

LO distribution of class VIII, NCERT curriculum, 2017.

A significant fact is observed in the LO codes of class VIII test items of NAS executed in 2017 and 2021. It shows an important cross-grade dimension in assessment design. A total 12 numbers of Los are selected for developing 25 numbers of survey test items for class VIII. Interestingly, out of 12 LOs, 6 LOs are from class VII. Hence, this necessitates the inclusion of class VII LOs while doing the analysis.

The systematic analysis of the LOs in science for classes VIII and VII, respectively, using Bloom’s taxonomy, reveals crucial insights about cognitive skill progression. As depicted in Figures 1 and 2, the curriculum shows intentional scaffolding across the grades. It has been shown that the class VII curriculum mostly focuses on foundational understanding (30%) and application-based LOs (25%), followed by creating 20% and analysing 15%, whereas class VIII curriculum emphasized on application-based LOs (27.78%) with creation and understanding competency has been put at the same footing, each with 22.22% of preferences. The percentage of analyzing LO (16.67%) is considerably lower than the preference percentage of understanding and application LOs. Notably, both grades show minimal emphasis on remembering (5–5.55%), which reflects the shift from rote memorization. The analysis further uncovered a critical decline in the evaluation category for both classes (5–5.55%), which suggests limited opportunities for the students to develop critical thinking skills.

The item analysis of 25 NAS test questions demonstrated the evolving alignment pattern with the vision of NCERT when executed for class VIII in science for the years 2017 and 2021. It is observed that there are 12 LO codes for 25 test items, but the total number of LOs used for the entire set is around 29. This is justified as each LO code SCI708, SCI710 and SCI711 contains two numbers of LOs or instructional verbs in the test items. The identified LOs are from understanding, applying, analysing, evaluating, and creating category, and no LO was found from the remembering category. The distribution pattern of LOs as observed in NAS test items for class VII (SCIENCE) in 2017 and 2018 surveys is presented in Figure 3. While the curriculum specifies 5.55% for remembering outcomes, NAS completely removed this category potentially signalling a positive shift towards meaningful assessment. The maximum number of test items is from understanding category (31% vs curriculum’s 22.22–30%) at the minimization of creating skills with (13.79 vs 22.22%). The least in the list was from the evaluating category (6.89%) while interestingly both analysing and application categories have equal importance with 24.13% preferences (Table 2).

Figure 3

LO distribution of NAS 2017 and 2021.

While comparing the LO distribution pattern in the curriculum of classes VII and VIII, it is well understood that keeping constant the remembering category at around 5%, understanding percentage is reduced from 30 to 27.785%, while slightly increasing the % preferences for categories belonging to higher cognitive domains such as applying, analysing, evaluating and creating. However, if we analyse the LO alignments in test items of NAS in 2017 and 2018, we find that the understanding category is comparable with the percentage prescriptions of LOs in class seven curriculum, but analysing the category LOs percentage is quite higher than that of both class VII and VIII curricula. As expected, the % of LOs identified in the test from applying the category and evaluating is comparable with the suggested LO percentage of those prescribed for both class VII and VIII curricula. Surprisingly, the % of LOs from the creating category is much below the suggested % of both class VII and VIII curricula; no test items are found from the remembering category (Figure 4).

Figure 4

Test items alignment with NCERT LOs.

The evaluation of 300-teacher-made test items shows that there is a substantial gap in the implementation. Only 38% of the total test items are aligned with the NCERT’s specified LOs, revealing that lower-order thinking skills (remembering – 52%, understanding – 33%) are overemphasized. LOs like critical experiment designing are found only in <3% of the items despite comprising 12% of the total curriculum expectations. Further, it has been revealed that state-board schools have the lowest alignment (29%) than the CBSE schools (44%), further suggesting that teachers are not adequately trained to align the test items with the expected LOs (Figure 5).

Figure 5

Test item alignment with cognitive levels.

Most concerning among all is the complete absence of evaluating skills and a very small amount of creating items (2%), indicating the undervaluing of these important competencies and over dependency on lower order thinking skills such as remembering and understanding (52 and 33%), respectively. Over dependency on textbook examples (73% items from textbook) further limited the opportunities for the use of real-world application-based items, which have been emphasized in the curriculum (Figure 6).

Figure 6

Comparison of items between NAS and the teacher-made test.

The comparison between the teacher-made questions and NAS questions reveals the assessment mismatch. While NAS devoted 24.13% to both applying and analysing, teacher-made tests have not emphasized these domains (12 and 3%, respectively). The main focus in the teacher-made questions was given to the remembering and understanding domain with 45 and 38%, respectively. It is more concerning that both the tests have given very less emphasize to evaluating domain (3% and 0). This further suggests that the teacher-made assessments are not preparing the learners for competency development as envisioned by NCERT and National Assessments (Figure 7).

Figure 7

Performance in 2017 and 2021: state vs national.

The overall analysis of NAS data for the years 2017 and 2021 in both state and national levels reveals nuanced patterns in competency development across the years. While the performance of the students at national levels appears stable between 2017 and 2021 (53–54.7%), this specific variation shows a little improvement in the competency levels. Performance at the state level shows a contrasting trajectory further offers valuable insights for the assessment cycle. The performance of the students for both years shows a little improvement throughout the years (48.30–51.50%) (Figure 8).

Figure 8

Improvement level: Odisha vs national.

At the national level, three LOs (SCI704, SCI705, and SCI811) showed modest gains of 3–5%, further suggesting targeted improvements in those particular competencies, but the decline in the other nine LOs is concerning. LO with code SCI711 (i.e. constructs models using materials from surroundings and explains their working) shows persistently poor performance (35% nationally). In the context of Odisha, six LOs show improvement as compared to the national level. The state’s 7.2% overall gain in class VII versus the national average of 5.1% suggests the effectiveness of its foundational learning interventions. The poor performances in both national and state (Odisha) level in consecutive NAS in 2017 and 2021 in LO code numbers SCI704, SCI10, and SCI711 may be attributed to the fact that all the three above-mentioned codes involve three interlinked LOs, namely understanding, application, and creativity, in such a way which is otherwise entangled with available physical situation or learning opportunity of the students.

The study explores the critical insight into the evolving scenario of assessment in science curriculum across India. With reference to revised Bloom’s taxonomy, the NAS provides a breakthrough to competency-based assessment, which suggests the importance of alignment of LOs in science assessment for middle school students (National Achievement Survey | Ministry of Education, GoI, 2021).

The findings of the study reveal significant disparities between India’s vision of competency-based assessment in science and its classroom implementations. While the NAS has predominantly shown the progress towards aligning the LOs (NCERT, 2021), the school-based assessment shows an inclination towards the lower-order cognitive skills. This implementation gap is quite evident with the overemphasis of lower-order thinking skills like remembering types test items (52%) while compared to the curriculum recommendation. The persistence of such assessment patterns despite the NCERT’s recommendation towards the higher order cognitive skills indicates issues such as inadequate teacher training to pedagogical conversions (Ponnambaleswari & Joseph, 2024).

According to Anderson and Krathwohl (2001) revised blooms taxonomy it has been emphasized that emphasizes, for an authentic science a balanced development across all the cognitive domains is required. As per the findings of the study, it has been shown that a creating-level question appears in only 2% of the teacher-made tests in school-based assessment versus the curriculum expectations, i.e. 22.22% as suggested by NAS, which reveals that the students are not being systematically prepared for complex problem-solving tasks (Liu & Israel, 2022). This explains the consistently poor performance across the integrated competencies like SCI711 (35% of the national average), which happens due to neglecting the higher-order skills in the classroom and school-based assessments (NAS, 2021). A significant decline in environment-related LO (SCI813) further highlights the vulnerabilities of school-based experiential learning. This suggests the need for more robust, multifaceted assessment approaches that can allow the development of competencies across all the diverse learning pathways (Mahajan & Sarjit Singh, 2017).

The relatively stronger performance trajectory of Odisha provides evidence of improvement of LOs can takes place with targeted interventions that includes adequate teacher training and equip learners with a classroom where they can be facilitated with the higher order competencies. The states gain (7.2%) versus national average (5.1%) suggests the effectiveness of its foundational learning programs. The constant urban–rural divides (15.3 points in Odisha vs 11.7 nationally) show the ongoing challenges of equitable access to science education (Ainscow, 2020).

In conclusion, the study underscores the critical need for the systematic alignment of competency-based science curriculum, school-based assessment and national evaluations to ensure achievement of meaningful LOs. The persistent dominance of lower order cognitive skills across all the test items of school-based assessment despite the NCERT’s guideline for emphasizing on higher order thinking skills further highlighting a significant implementation gap that undermines the learners potential to deal with the real-world problems. The disparities between NAS’s competency-based assessment and traditional teacher-made assessment design particularly in rural belts reveal an urgent need for teachers professional development across the curriculum. Therefore, reforms must be made to prioritize standardized assessment frameworks, equitable access and robust monitoring mechanism to bridge the gap between the prescribed curriculum, policy aspiration and grounded classroom practices, ensuring the shift towards the competency-based education.

Authors state no funding involved.

All the authors have contributed significantly to the conception and design of the study.

Authors state no conflict of interest.