Introduction
A call for the inclusion of multimodal meaning-making in assessment
The increasing digitisation of classrooms has brought to the fore questions regarding the changing textual patterns of school subjects and consequences for learning materials and curricular activities (e.g., Bezemer & Kress, 2008; Jewitt, 2011; Magnusson & Godhe, 2019; Mitsikopoulou, 2022; Ørevik, 2015). For example, digital software enables students easily and efficiently to combine verbal text with other semiotic modes such as images, moving images, and music. Jewitt (2014, p. 12) defines a semiotic mode as ‘a socially shaped and culturally given resource for making meaning’. A multimodal perspective of learning acknowledges that the affordances of modes and text authors’ choices of semiotic resources influence the ways in which curricular topics are represented (Jewitt 2003, p. 83; Skulstad, 2022). Notably, explicit recognition of multimodal aspects in processes of conceptualising meaning has the potential of including more learners in knowledge exchange (Archer & Breuer, 2015, p. 7). The need for students at all levels of education to develop multimodal literacy is thus evident. Multimodal literacy requires knowledge and understanding of semiotic systems and textual conventions as well as of the role of contextual factors in communication; furthermore, it involves an aesthetic dimension and a critical dimension (van Leeuwen, 2017).
Students are generally well immersed in multimodal communication, connected both to schoolwork and to their interests outside school (Bezemer & Kress, 2016; Erstad, 2013; Jewitt, 2008). Although forms of communication within school subjects have traditionally been constrained by established text formats codifying what counts as important knowledge and skills (Selander, 1994), the range of texts and communication forms used in teaching and learning contexts has broadened considerably with digitisation (Gilje, 2021). Research within language didactics has for many years addressed the challenge of synthesising out-of-school literacies with literacies developed through education, discussing the need for approaches or tools to connect students’ intersecting text worlds (Godhe, 2014; Lund, 2006). Proceeding from the principle that teaching, learning and assessment in school subjects need to be aligned and integrated (Bøhn, 2019; Lee, 2007), this also implies a need for assessment practices that recognise students’ multimodal meaning making (Jewitt, 2003; Johnson & Kress, 2003; Magnusson & Godhe, 2019; Mitsikopoulou, 2022). According to Bezemer and Kress (2016), recognition of students’ semiotic work involves acknowledging what often goes unnoticed in educational contexts and therefore requires ‘lenses and methods that move well beyond contemporary metrics’ (p. 3).
In order to inform further developments in the field, more research is needed as to how teachers work with and assess students’ multimodal text production (Silseth & Gilje, 2019). This article seeks to make a small contribution in this regard by reporting from a project involving two classroom interventions in the subject of English as an additional language (EAL) where peer and teacher assessment of students’ multimodal texts was combined, aiming for a rich appraisal of the students’ products, and promoting the development of literacies in broad terms.
Assessing multimodal meaning-making
A need for frameworks that include multimodal aspects of texts
Previous research has uncovered factors that may counteract the inclusion of students’ multimodal texts as evidence of competence in school subjects. One such factor is the traditionally marginal role assigned to multimodal texts in assessment situations. Although texts studied by learners during classroom work and test preparation are typically multimodal, tests and exam questions tend to elicit written texts (Jakobsen, 2019; Tan et al., 2020). Even where students’ multimodal texts are in fact assessed, there is a tendency for verbal language to be focused, while meanings expressed through other modes are largely overlooked (Aagaard & Silseth, 2017; Godhe, 2014; Silseth & Gilje, 2019; Towndrow et al., 2013).
Researchers have also noted uncertainty among teachers as to how to assess multimodal texts (Lund, 2009; Aagaard & Lund, 2013). Tan et al. (2020) found ‘evidence that teachers readily rely on assessment criteria that are linked to print media, when faced with the task of assessing multimodal texts created through digital media’ (p. 102). According to Aagaard and Lund (2013), some teachers report a sense of conflict between new ways of learning mediated by digital technologies and forms of assessment following pre-digital traditions. This suggests a need to integrate multimodal aspects of representation and communication more explicitly in assessment criteria used in schools.
Responding to the identified need of tools for the assessment of multimodal literacy, various approaches have been explored (e.g., Canale, 2019; Fjørtoft, 2020). The multiliteracies framework (Cazden et al., 1996) has been groundbreaking in this regard. Indeed, the multiliteracies paradigm has influenced education policies, for example, in Singapore and Australia, to incorporate aspects of multimodal literacy in EAL curricula (Mitsikopoulou, 2022). Furthermore, the Common European Framework for Reference of Languages (CEFR) (Council of Europe, 2001, 2020) has provided a foundation for further development of assessment frameworks addressing specific areas of multimodal literacy (e.g., Campoy-Cubillo and Querol-Julián, 2015; Sindoni et al., 2019). There is now a growing body of research and development combining theoretical perspectives, e.g., from writing studies and multimodal analysis, that can be drawn on to implement practices of assessment of multimodal texts (e.g., Hafner & Ho, 2020).
Multimodal social semiotics as a theoretical lens
Halliday’s theory of systemic functional linguistics (SFL) has been highly influential in the development of multimodal studies as an academic field. SFL posits that three dimensions of meaning are simultaneously enacted in any text, namely, representation of the world, interpersonal relations, and textual organisation, conceptualised as the ideational, interpersonal, and textual metafunctions (Halliday, 1994). These dimensions are reflected in several frameworks theorising the deployment and configuration of semiotic resources (e.g., Kress & van Leeuwen, 2021 [1996]; Lemke, 2002; Martinec & Salway, 2005; O’Halloran & Lim, 2014; Royce, 1998; Unsworth, 2001).
Multimodal social semiotics (MMSS) (Bezemer & Kress, 2016; Kress, 2010; Kress & van Leeuwen, 2021) provides a framework for analysis of multimodal texts with a view to interest, identities, and relations of power. Viewing the objective of communicative competence through this theoretical lens, students’ multimodal text production can be construed as sign-making through the transformation of available semiotic resources (Kress, 2010). The reader of a multimodal text is also a sign-maker, who forms hypotheses about the sign based on their interest (Kress & Jewitt, 2003, p. 13; Kress & Selander, 2012). By active engagement with an aspect of the world through socially and culturally shaped semiotic resources, learning takes place in the sense that the learner’s inner resources are remade (Kress, 2010). Students display evidence of learning through transformation and sign-making activities (Selander & Kress, 2017, p. 33). In the classroom, such transformative engagement may involve, for example, drawing on out-of-school literacies in text production. Kress and van Leeuwen’s (2001) term provenance, ‘where signs come from’ (p. 10), is of relevance when discussing the practice of ‘importing’ signs from one context and using it in another. An effect the sign-maker may achieve by such ‘import’ is to evoke ideas and values associated with the other context (Kress & van Leeuwen, 2001).
The mode of image has become increasingly dominant in the digital age (Kress, 2003) and plays a significant role in young people’s communication. An apt framework for the recognition of students’ visual meaning making is the Grammar of Visual Design (Kress & van Leeuwen, 2021). Its dimensions of representation, interaction, and composition builds on Halliday’s metafunctions, denoting, respectively, ideational, interpersonal, and textual meaning realised in visuals. Narrative representation of an unfolding event, for instance, may be expressed by vectors formed by lines (e.g., the direction of a depicted person’s gaze or hand movement); and conceptual representations, for example taxonomic relations, may be shown by hierarchical ordering of elements. Validity as an aspect of interaction concerns the degree to which something is represented as true, indicated by validity markers such as degrees of colour differentiation, contextualisation, or abstraction. Here truth is conceived as a semiotic construct contingent on the values and beliefs of the social groups participating in the communication (Kress & van Leeuwen, 2021, p. 150). Interaction in the form of offer or demand can be realised, respectively, as a person viewed from the side, positioned as an object of the reader’s scrutiny, or a depicted person initiating eye contact with the reader. Moreover, the use of camera angle may indicate individuals’ power or inferiority. As for the dimension of composition, the spatial organisation of elements in a text is significant, for example in terms of construing information value. An example is Given-New, in Western reading tradition organised from left to right, which indicates relations between the well-known, self-evident (Given) and the unknown, ‘problematic’ (New) (Kress & van Leeuwen, 2021, p. 187). Such elements from the grammar of visual design can be translated into accessible tools for students as they work on projects promoting multimodal literacy, as has been done also with other SFL-based frameworks (e.g., Lim, 2018).
Peer assessment of students’ multimodal texts
Peer assessment has been recommended as one of the ways to support students’ development of writing skills and awareness of assessment criteria (Lee & Coniam, 2013). A large-scale metastudy of control-group research documented that peer assessment contributes to improved academic achievement (Double et al., 2019). As these studies specifically addressed effects of peer assessment, the findings will also bear relevance to assessment of text production in other modalities than writing.
There is no denying, nonetheless, that creating a multimodal text involves a broader range of semiotic choices than writing within a conventional school genre. Moreover, the social worlds with which the producer of the text associates will influence these semiotic choices (Kress & Selander, 2012). It is thus useful for teachers and students to negotiate what aspects of multimodal meaning-making to consider in assessment situations (Godhe, 2014). Semiotic content that varies in meaning between generations and cultural groups may represent a problem when assessing multimodal meaning making, as exemplified in Aagaard and Silseth (2017). An underlying assumption regarding the present project, however, is that assessment from both the teacher’s and the students’ perspectives will provide a richer and accordingly more valid assessment of the students’ semiotic work. Unlike established written school genres where teachers traditionally possess the power to define what constitutes ‘a good text’, students may feel a higher degree of ownership to multimodal texts associated with genres encountered outside school and consequently state more authoritatively what they believe works to convey a message successfully. Negotiated assessment reduces the hierarchichal nature of the conventional teacher-student relationship and assigns the student a more active and responsible role in the assessment situation (Gipps, 2002, p. 77). This is conducive to increasing the learner’s agency in their semiotic work (Kress & Selander, 2012).
The context of the study: Norwegian upper secondary school
The small-scale study presented here, conducted as two classroom interventions in Norwegian upper secondary schools, addressed the following research question: How can peer assessment and teacher assessment complement each other in their recognition of upper secondary students’ semiotic work in the EAL subject? Complementary perspectives would be achieved by peers viewing the texts through the interpretive lens of youth culture, the teacher ensuring assessment of the texts in line with the aims of the subject, and the researcher anchoring the assessment on theory of multimodality. Both interventions were centred on a literacy event in the EAL classroom that involved students’ production of multimodal texts with the communicative purpose of persuasion, and the ensuing assessment of these texts. Literacy event is defined by Heath (1982, p. 93) as ‘any occasion in which a piece of writing is integral to the nature of participants’ interactions and their interpretive processes’. Extending the notion of writing to multimodal text creation, this literacy event would prompt the students to consider and make use of the joint affordances of verbal text and visuals to persuade their audience of a certain viewpoint, as well as discussing to what extent their peers succeeded in obtaining persuasive effects through their deployment of semiotic resources.
The subject of English is mandatory in Norway through years 1–11, and most students in Norwegian secondary education can thus be expected to master the language at a relatively high level. Communicative competence (Hymes, 1972) constitutes the overarching aim of language learning. Assessment situations conducted throughout the schoolyear feed into an overall grade which describes the student’s level of competence in the subject on completion of the course.1 Bøhn (2019) holds that there is no universal agreement as to what constitutes the construct of communicative competence: this may be operationalised in various ways. The core criterion of communicative competence is, nonetheless, the extent to which a person succeeds in making meaning in accordance with the context and purpose of communication. As noted by multiple scholars internationally (e.g., Archer, 2000; Skulstad, 2009), communication in its fullest sense cannot be achieved without developing the skills to interpret and express meaning through combinations of linguistic structures with other ways of representing the world. Communicative competence should therefore be conceptualised as comprising multimodal literacy (cf. Heberle, 2010; Skulstad, 2009).
Level-specific competence aims in the subject curriculum for English in Norwegian upper secondary education include ‘use appropriate digital resources and other aids in language learning, text creation and interaction’, and ‘discuss and reflect on form, content and language features and literary devices in cultural forms of expression in English from different media in the English-speaking world’. The subject curriculum construes the concept of text broadly, specifying that ‘[t]he texts can contain writing, pictures, audio, drawings, graphs, numbers and other forms of expression that are combined to enhance and present a message’ (Norwegian Ministry of Education and Research, 2019). Competence aims for both lower secondary and upper secondary English include the creation of multimodal texts. Even so, criteria for successful achievement of competence aims for text production in EAL at upper secondary level, stated by the Norwegian Directorate for Education and Training (Udir), focus on written and spoken English only, exemplified by the following criterion: ‘The student expresses themselves in a precise and nuanced language (…) and adapts the form of communication clearly and coherently to purpose, audience and context in various types of oral and written texts’ (www.udir.no, 2020, my translation, emphasis added). In other words, the EAL subject in Norway embraces a multimodal view of texts and learning activities but does not explicitly and consistently incorporate this view in documents outlining principles of assessment. This suggests a need for more explicit mention of multimodal aspects in the official discourse of assessment in EAL, supported by research addressing the assessment of students’ multimodal meaning making in the subject.
Methodological approach
A small-scale design-based research project
The present project can be described as design-based research, a category of classroom research where researchers and teachers in collaboration carry out an intervention aiming at solving an identified problem or instigating improvement in an area of practice (Anderson & Shattuck, 2012). Anchored on relevant theory, the project should be designed and conducted with an aim to inform practice in the average classroom (Brown, 1992). The practice that the present project sought to improve was the traditional assessment routine in the EAL subject, which typically focused exclusively on students’ expression through language. Complementing the general assessment criteria for the subject, the interventions consisted of developing criteria that considered multiple semiotic modes and implementing an assessment procedure for multimodal texts with separate rounds of peer and teacher assessment. Emphasising multimodal aspects of the texts, the criteria needed to consider how modes such as image and verbal language were organised and worked in relation to each other. For the communicative aspects, the assessment would consider to what extent the configuration of modes conveyed a clear and coherent message. With a view to particular aspects of the students’ transformation of signs, the criteria would also consider whether the students ‘imported’ signs from other contexts into their schoolwork and to what extent the text creators made original and creative use of semiotic resources to get their message across.
A characteristic of design-based research is iterative refinement and adjustment of the intervention (Anderson & Shattuck, 2012). Instead of implementing the assessment procedure as iterative cycles in the same group of students, I chose to conduct two similar interventions with two student groups aged 16–17 from different schools, to involve a broader range of participants. Their respective teachers of English collaborated with me in organising the literacy event; additionally, they joined me in forming the teacher/researcher assessment team. Participants for the project were purposefully selected (Creswell, 2014) by contacting teachers of English that I had previously encountered through my work in teacher education. Findings from the first intervention were reflected on and considered in the refinement of the assessment criteria prior to the second intervention.
Data collection
In agreement with the teachers, I paid an hour-long visit to the student group at the start of each literacy event, introducing the students to basic principles of multimodal analysis, presenting the aims of the project, and outlining the text assignment and pertaining assessment criteria. The students were asked to create a multimodal text aiming at convincing the reader/viewer of a viewpoint of their own choice, and the teacher organised and supervised the students’ text production during two consecutive double sessions of English. To optimise the students’ motivation the teacher permitted them to work individually or collaborate with one or two peers as they saw fit.
I rejoined the class at the time of the peer assessment. Knowing the students, the teacher put together groups of three or four peers who they envisioned could complement each other in the appraisal of the texts. Each group received one or two texts to assess. I gave the groups some time to discuss the text; then I stopped by each group once for each text they assessed and interviewed them. The interviews were conducted as focus group interviews with the aim of bringing out participants’ views on the topic at hand (Kvale & Brinkmann, 2015). I started by asking the peers how effective they found the text to be in terms of persuasion, then prompted them to elaborate on their views and/or interpret specific features of the text. The interviews lasted from 5 to 10 minutes, depending on the level of detail of the peers’ contributions. To ensure unimpeded discussion, the students were allowed to discuss in Norwegian. I transcribed the recordings of the peer group interviews verbatim.
The assessment by the teacher/researcher team was conducted separately, applying the following assessment criteria based on the considerations outlined in the previous section.
Relation between modes: visuals and verbal text agreeing (or purposefully conflicting) in ways that emphasised the message would be credited.
Coherence and clarity: clear, coherent communication in line with the communicative purpose of the genre would result in a high score.
Intertextuality: recognisable ‘import’ from one context to another would be credited.
Originality and creativity: engaging with the topic in an inventive way was assumed to reflect a high degree of transformation of signs and thereby learning.
Notably, these criteria were refined after the first intervention (see Ørevik, 2022 for details regarding the development of a more fine-grained set of assessment rubrics). The revised rubrics (see Appendix) were used by the teacher/researcher team to assess the texts in the second study. Further to a suggestion from the collaborating teacher, the students were given a shortened and simplified version of the rubrics to consult in their text production and peer assessment.
The teacher/researcher team convened in an assessment conference during each intervention. These conferences consisted of brief, pointed discussions, and were therefore not recorded. Taking an active part in the discussions myself, I took notes, marking the level of competence (minimum, medium or high) at which each text was placed by the team and noting the rationale supporting the assessment.
In sum, the data from the first intervention comprised 8 peer group interviews and notes from one assessment conference; the data from the second intervention comprised 5 peer group interviews and notes from one assessment conference.
Figure 1 visualises the procedure followed in both studies.
Figure 1
Intervention procedure.
Ethical considerations
The project was registered with the Norwegian Centre for Research Data (NSD/Sikt) and confirmed as complying with national requirements for data protection. I informed participants of their rights, including voluntary and anonymous participation and the right to withdraw from the study at any stage of the project. Students wishing to participate signed a consent form granting me permission to record group interviews2 and publish images of their texts connected to dissemination of the research.
Data analysis
As stated above, the project aimed to establish how assessment by the peers and the teacher/researcher team could complement each other. In line with the principles of thematic analysis (Braun & Clarke, 2006), I coded the transcriptions of the peer group discussions and the notes from the teacher/researcher’s assessment conferences in three subsequent processes. First, I noted elements of sign-making that each of the two assessor groups evaluated as succeeding or failing in fulfilling the communicative purpose of each text. Second, I divided these into categories marked ‘agreement’ or ‘complementary’ points relating to each text. Third, based on recurring points in the views expressed by the assessor groups, I identified emerging themes in the assessment of the texts connected to the assessment criteria listed above. The themes were then linked to the analytic dimensions of representation, interaction, and composition (Kress & van Leeuwen, 2021), viewing the findings through the theoretical lens underpinning the study. This made explicit how agreement and diverging views between the two assessor groups were distributed across dimensions of the students’ multimodal communication.
Results
The first student group produced seven multimodal posters, all digitally created. The second student group submitted five texts, comprising four posters, specifically one digital poster and three analogue-digital hybrids, and one video. Facsimiles of the students’ texts are shown in Figures 2 and 3.
Figure 2
Facsimiles of multimodal texts, classroom intervention 1.
Figure 3
Facsimiles of multimodal texts, classroom intervention 2.
Results of the two rounds of assessment, sorted into points of agreement and complementary views between the two assessor groups, are presented in Tables 1 and 2. A summary of the themes connected to relevant assessment criteria and MMSS dimensions is provided in Table 3.
Table 1
Agreement and complementary assessment, classroom intervention 1.
| TITLE OF POSTER | AGREEMENT | COMPLEMENTARY ASSESSMENT |
|---|---|---|
| A) Warning! Save the earth now! | Combinations of pointed written statements and visuals (images, colour, symbols) create a powerful effect. A cause-effect relation is achieved by depicting natural disasters followed by images of injured animals. | P: Polar bear is a good representative of precarious animals. T/R: Depicted animal making eye contact places a demand on the viewer. P: Mark Twain citation is a reminder that you can do nothing about the weather, but you can do something about the climate. T/R: Mark Twain citation is an example of creative transformation of semiotic resources. |
| B) We are conducting a mass extinction! | The black background and statistics underscore the seriousness of the message. Aesthetically pleasing images and layout makes the text attractive. | P: Depicted animals are typical representatives of precarious species. P: Images foreground the animals as valuable, accentuated by their beauty and the reminder that they may not last for long. T/R: Animal making eye contact places a demand. T/R: Missing references to statistics. |
| C) Why we should raise the legal age for tobacco and drinking to 21. | The message is at times incoherent. Claims in the text are unsupported. | P: Social situation in the car gives an attractive impression. P: Difficult to identify with depicted older individuals and images of arms/hands. Image of lungs: removed from everyday visual experience. |
| D) Children should learn how to use a PC in elementary school. | P: Too much written text is unappealing. P: Dark colour gives a sombre effect. T/R: The message is focused and coherent. | |
| E) Homeschool – better learning and more focus at home! | Clear and coherent message. | P: Images give immediate information about the content. P: The headline contains too much. P: Radioactivity symbol makes the danger real. T/R: Exact references are given. T/R: Depicted individuals do not make eye contact with the viewer: ‘offer’ instead of ‘demand’. |
| F) Hiking is man’s best medicine. | P: People having fun together is attractive. T/R: Most depicted people are viewed from behind, identification difficult. T/R: Black as background colour brings darkness, unfortunate in view of the topic. | |
| G) Travel the world | Aesthetically pleasing image and font. Written texts describe positive aspects of travelling, but without support or documentation. | P: The prospect of being close to an elephant in idyllic, exotic surroundings appeals to emotions. P: The written text contains difficult words. |
[i] ‘T/R’ = Teacher/researcher team; ‘P’ = Peer group.
Table 2
Agreement and complementary assessment, classroom intervention 2.
| TITLE OF POSTER | AGREEMENT | COMPLEMENTARY ASSESSMENT |
|---|---|---|
| H) Reduce the limit agefor drivers’ licence to 16 | Headline, images, and captions do not converge into a focused message. No support for written statements. | P: Should have written their own text instead of using Internet captions for the images. P: Image of passenger using phone fails to support the argument. |
| I) Fuck exam! | Overall focused message. Good connection between writing and visuals. | T/R: Style appealing to a young target group. P: Phone signals the student’s obligation to socialise with friends. T/R: Fist signals a threat. P: Interpretations of fist: a ‘knuckle bump’ of encouragement or a threat. T/R: Pink and yellow highlight the headline. P: Facial expression and posture of depicted student make the stress real. P: The message is easy to understand but needs more written text to build argument better. |
| J) 16 years old should be able to drive | The use of the ‘Drive through’ sign is creative. | T/R: Relations between visuals and writing function adequately, but communicative function of colours and symbols not always clear. T/R: Slight overload of communication, demanding for the reader. P: Aesthetic value makes the text convincing. P: Good organising conveys the message effectively. P: Photo of drinkers older than 16 works well with statement that 16-year-olds drink less. P: Photo of motorbike illustrates illogical point that cars have a higher age limit than motorbikes. |
| K) Wear a mask(video) | The video stages relevant and realistic situation, easy for youths to identify. Focus on the mobile phone underscores its importance for young people. | P: Convincing connection between wearing a mask and the opportunity to go to a party. P: Panes of written text break the film up in a pleasant way and create suspense. T/R: Panes of written text form transitions and have a cohesive function. P: At times difficult to hear due to low sound quality. |
| L) Reduce the age limit for driver’s licence to 16it can be very dangerous. | Clear message, tight relations between visuals and writing. | P: Use of traffic light colours is creative, attractive and underscores the topic. T/R: The affordances of font and size could have been used better. P: Causal relationship between accidents and reasons for accidents well expressed. T/R: Causal relationship unclear due to reversed sequence of images. P: Mixed views on the black background. P: Image of phone illustrates 16-year-olds’ immaturity. P: Grammatical errors disturb communication somewhat. |
[i] ‘T/R’ = Teacher/researcher team; ‘P’ = Peer group.
Table 3
Emerging themes from the assessment rounds.
| ASSESSMENT CRITERIA | PEERS | TEACHER/RESEARCHER TEAM | DIMENSION |
|---|---|---|---|
| Coherence and clarity | Ease of understanding | Interaction | |
| Coherence and clarity | Support of statements | Support of statements | Interaction |
| Coherence and clarity | Purposeful use of layout, font, and colour | Purposeful use of layout, font, and colour | Composition |
| Relation between modes | Language-visuals connection | Language-visuals connection | Composition |
| Originality and creativity | Aesthetic qualities | Composition | |
| Originality and creativity | Making the topic real | Making the topic real | Representation and interaction |
| Originality and creativity | Potential for identification | Enactment of offer/demand | Interaction |
As shown in Table 3, most of the themes were addressed by both peers and the teacher/researcher team. There were, however, important nuances in their perceptions, as evident in Tables 1 and 2.
The peers viewed it as important for the text to be easily understandable to get the message across. Text B, where images carried much of the functional load, received credit from the peers in that respect. Similarly, text J was characterised as easily comprehensible due to good organising of the elements. Too much written text (D) or too many difficult words (G), on the other hand, was deemed by the peers as hampering communication. Interestingly, in text L, the students were concerned about linguistic errors, whereas the teacher/researcher team found the text adequately communicative.
Both the peers and the teacher/researcher team noted that in several texts, quantifications were stated without documentation (e.g., texts B and F). Therefore, a point concerning accessible references as support of statements was worked into the revised assessment criteria.
Layout, font, and colour were commented on in several cases. The teacher and researcher found the layout of text A somewhat messy at first sight but concluded, in agreement with the peers, that it worked well in creating cause-effect relations. The peers found the use of images in text L effective to get the message across, pointing out the red lines forming an arrow from the photos depicting phone texting and drunk driving to the centred image of a serious car accident. The teacher/researcher team acknowledged this but remarked that the direction of the elements did not comply with the conventional Given-New information structure. Colours, particularly red and black, were found by the peers to have a powerful effect (e.g., texts A and B). The use of ‘traffic light’ colours red, orange, and green in text L was admired by the peers as inventive, as this poster concerned driving. In some cases, both the peers and the teachers detected a sombre, negative effect created by dark colours, but in different texts (D and F, respectively).
Both assessor groups considered good connection between language and visuals as important, as noted in their positive assessment of texts E and K. Text H received a less favourable appraisal, as the captions to the images were viewed as disconnected from the overall message of the text.
Aesthetic value was regarded by some of the peers as playing an important role in persuasion, more so than for the teacher and researcher, as evident in the assessment of Text J. Peers likewise appreciated the aesthetic qualities of text G, but here they also pointed out some drawbacks weakening the persuasive effect.
Another theme was the deployment of visuals to make a topic ‘real’. In text C, images contrasting healthy lungs with brown, tarred lungs were not considered effective by the students, since they did not associate images of isolated body parts with their own respiratory organs. In the two posters addressing the dangers of global warming, by contrast (texts A and B), the peers found that the visuals succeeded in representing the seriousness of the situation.
As for text I, the peers pointed to the facial expression and the posture of the exam candidate as expressing stress, while the teacher and researcher also commented on the images encircling the student as communicating effectively. An interesting point of contention here was the picture of a fisted arm, which the teacher/researcher team interpreted as a threat, as did one of the peers. Another peer, however, saw a friend offering an encouraging ‘fist bump’ greeting.
The point of importance perhaps most strongly emphasised in the peer assessment was the potential for identification with their own age group. For example, the peers found text C less than convincing, partly because ‘old’ men were depicted as representing the dangers of smoking and drinking. A similar view was expressed about an image in text J. By contrast, the peers remarked on images showing social scenes with people their own age as attractive and thereby persuasive. In several cases, the mobile phone appeared to signal mediated social activity. Text K, for example, showed the phone as the bearer of messages both from friends (invitation to a party) and from the hospital (results of a Covid test), and the depicted phone in text I was interpreted by the peers as indicating the student’s duty to socialise with friends. In text L the phone represented a distraction from driving. When asked if a 16-year-old was more likely than an 18-year-old to use her mobile phone when driving, one of the peers assessing text L replied that younger teenagers were generally more addicted to social media such as SnapChat and would feel compelled to look ‘if that boy, like, snaps you back’.
A theme also relating to personal appeal emerged in two of the other texts, where the teacher/researcher team saw a demand for empathy being placed by a depicted animal. In text A, the reader was met with the gaze of a koala apparently singed by a forest fire, while text B included a gorilla initiating eye contact with the reader. Based on these findings, the potential for interpersonal engagement was integrated in the revised assessment criteria (see Appendix).
Discussion
Didactic use of complementary assessment results
In the assessment of most of the students’ texts produced in the present project, the peers and the teacher/researcher team expressed coinciding views. This tendency was also evident in the more comprehensive study conducted by Sindoni et al. (2019, pp. 36–37), where correlation was found between teachers’ and students’ assessment of the same multimodal texts. In the case of the present study, both peers and the teacher/researcher team pointed out support of statements as a sign of credibility, which conveys the important message that multimodal texts need to hold the same standard of accountability as written texts. The two assessor groups also agreed that coherence between verbal and visual semiotic resources was crucial to get the message across, and they agreed on the persuasive power of finding apt visuals to represent core aspects of the message. Here, the peers’ appreciation of typical precarious animals depicted to represent the climate crisis, for example, resonates with Royce’s (1998) indentification of relations of meronymy between images and written text. Extending this further into the curricula of language subjects, identifications of rhetorical devices can form bridges between verbal expressions such as poems and visual expressions in literacy events, drawing, for example, on Royce’s (1998, 2007) work on intersemiotic complementarity.
Instances where views or nuances of meaning differ between groups of assessors, may form a valuable backdrop for reflection on literacy and communication. One example from the present study is text E, where the teacher/researcher team valued the coherent writing in the text, while the peers found the written text dense and unappealing. Such observations may inspire discussions of how to make semiotic choices with the aim to communicate effectively with the intended audience. Interestingly, text J was characterised as ‘overloaded’ and demanding by the teacher/researcher team, while the peers focused on the aesthetic qualities of the poster. This exemplifies how texts may be perceived fundamentally differently by teachers and students, which forms a useful perspective for teachers seeking out texts apt for classroom work on curricular topics. But this finding also shows a need to problematise the persuasive powers of aesthetics in class discussions, foregrounding the critical dimension of multimodal literacy (van Leeuwen, 2017).
Differences in the interpretation of signs, such as the depicted fist in text I, illustrate the reality that sign-making is ambiguous, and meaning is dependent on context and participants in communication (Halliday, 1994). This came to the fore where there was basic agreement between the two assessor groups but subtle nuances in interpretation, such as the role of the mobile phone in social activity. While the teacher/researcher team saw the depicted phone as a general indicator of its position in youth culture, the peers contributed additional layers of meaning. Overt negotiation of meaning in these cases resulted in extended insights, with the students, the teacher and the researcher all experiencing the role of the reader as sign-maker (Kress & Jewitt, 2003).
Implementing practices that draw on and enhance students’ multimodal literacy
According to Selander and Kress (2017, p. 34), ‘signs of learning consist of all the choices of central aspects and choices of expression that are made to show how one has understood something (in a new way)’ (my translation). Allowing students to present multimodal texts for assessment gives them a broader repertoire for meaning making and promotes their agency in the subject (Lim & Nguyen, 2022). As touched upon earlier, however, established codes of expression may still stand in the way of making full use of this repertoire. For example, Godhe (2014) observed students appearing uncertain as to whether their out-of-school literacies counted in the multimodal text they produced as a school assignment. This suggests a need to explicitly discuss with the students what counts as representation of knowledge (Romero & Walker, 2010) and cast a wide net to sample students’ knowledge and skills relevant to the subject. Findings from the present study indicate that out-of-school literacies feed into teenagers’ multimodal literacy, evidenced by the students’ readiness and potential to identify and assess transformation of signs in their peers’ multimodal communication.
Acknowledging the complex text worlds surrounding today’s students, teachers can introduce opportunities for varied text production to elicit evidence of students’ knowledge and literacies developed inside and outside school. Silseth and Gilje (2019) suggest that in addition to formative assessment and support during text production, multimodal texts can be included in learner portfolios along with other types of text. In a similar vein, Fjørtoft (2020, p. 9) recommends a longitudinal approach using multimodal digital classroom assessment to obtain rich data on students’ learning and development. As demonstrated through the present study, students’ recognition of their peers’ meaning making contributes additional valuable evidence of learning as transformation of semiotic resources.
Conclusion
The procedure of assessment implemented in the two classroom interventions described in this article brought out instances of both agreeing and complementary appraisal of students’ multimodal texts in the subject of EAL. Areas of agreement concerned compositional aspects, such as the importance of coherence between verbal text and visuals; they also concerned interactional aspects, such as the necessity of supporting claims and the persuasive effect of making the topic real, which connected both to representation and interaction. Differences or nuances in the assessors’ interpretation of texts mainly concerned the interactional dimension. Specifically, the peers valued deployment of semiotic resources that facilitated understanding, and they emphasised the potential of identification as crucial for persuasion. The teacher and researcher particularly valued the direct enactment of interaction in the form of demand in some texts. Highlighting the significance of context and social factors in communication, the two rounds of assessment contributed to negotiations of meaning drawing on literacies developed both inside and outside school. Such negotiations of meaning give valuable evidence of students’ semiotic work and are conducive to heightening their semiotic awareness.
Certain limitations of the study should be pointed out. Summarising and translating the participants’ authentic utterances always entail a risk of losing nuances of meaning. Furthermore, the assessment rounds marked the end of the design-based research conducted in each student group. Prolonged research periods involving several iterations might have established the assessment practice more firmly and allowed for additional refinement of the rubrics and the assessment procedure.
Acknowledging these limitations, the study showcases a way for teachers to implement a collaborative assessment procedure that involves students actively in the recognition of each other’s multimodal meaning making in EAL, thereby promoting the students’ agency in learning and assessment, and supporting their development of multimodal literacy as an aspect of communicative competence in the subject.
Additional file
The additional file for this article can be found as follows:
Additional File 1
Appendix: Revised assessment criteria for persuasive multimodal text. DOI: https://doi.org/10.16993/dfl.216.s1
Notes
Acknowledgements
I would like to express my sincere gratitude to the teachers and students who agreed to participate in these studies, and to Professor Aud Skulstad who gave me valuable feedback on an early draft of this article.
Competing Interests
The author has no competing interests to declare.