Introduction
Gianini (2023) argues that “the ‘raison d’être’ of education is to help us make informed choices of how we want to construct our lives and our societies”, however, Couldry and Hepp (2018) argue that our social reality is to a growing extent being constructed through data-driven and automated algorithms and decision-making processes, including Artificial Intelligences (AI) which is transforming societies and therefore, education and our different professional practices. Van Es and Schäfer (2017) state “students need to be educated to become critical data practitioners who are both capable of working with data and of critically questioning the big myths that frame the datafied society” (p. 12). So for advancing these skills, it is key to use assessment methods that can build a thorough understanding of the impact of data and AI in society, whereas authentic and real-world assessment can be a catalyser of critical data and AI literacies.
The advancement of data and artificial intelligence (AI) technology are profoundly impacting society, indicating a need to develop critical awareness of data and AI literacy in education across disciplines, as every single area of knowledge and professional practice is or will be affected by the coming transformations across education, business, the labour market and society. Thus, educators in HE are uniquely positioned to critically address the challenges and opportunities of data and AI, resisting the development of AI hostile learning environments, by moving towards AI responsible and enabling teaching ecosystems, by reframing how we assess, develop and co-create knowledge and understanding in our disciplines. Thus, the adoption of authentic and real-world assessment models that are enabled or include elements of critical data and AI literacy, allow us to develop a series of competencies for professional and citizenship practices, developing assessment models that foster critical and reflective thinking (Wolff et al., 2016; Long & Magerko, 2020; Cui et al., 2023; Pechenkina, 2024).
Our aim is to provide guidance for educators to support students in understanding the impact of data and AI in our lives, transforming assessment that addresses real-world challenges and builds capacity in critically understanding data and AI. We ground our approach in the principles of data justice, which is defined by Dencik et al. (2019) as denoting “an analysis of data that pays particular attention to structural inequality, highlighting the unevenness of implications and experiences of data across different groups and communities in society” (p.875). It can provide a useful framework for authentic and real-world assessment design, that promotes critical data and AI understanding in alignment with citizenship literacies amongst academics and learners (Brand and Sander, 2020; Sander, 2023), considering the data and AI ethics concerns challenges we face and the impact of data and AI on our societies and democracies in terms of biases, discrimination and skewed narratives (Moore et al., 2024; Coeckelbergh, 2020; Perla & Vinci 2023).
For this exploratory paper, we analyse the intersection of critical data and AI literacy, data justice, and authentic assessment, highlighting the potential of this approach in fostering a deeper understanding of data ethics, privacy, and equity by showcasing practical implications of using AI in developing authentic assessment grounded on data justice (Dencik et al., 2019).
We have used an integrative approach to our literature review (Torraco, 2005) as mitigation strategy in terms of algorithmic biases present in scholarly databases, as existing biases tend to be amplified in systematic reviews in terms of gender, and geographical representation and exclusion. Our research ethos aims at including a variety of voices in a diverse and inclusive manner to mitigate algorithmic knowledge inequalities (Almeida & Goulart, 2017; Kordzadeh & Ghasemaghaei, 2022; Atenas, Bussu and Nerantzi, 2023). This includes exploring literature with an inclusive lens towards ensuring that a large body of the literature selected for this paper is written by women, or that include women among the authors. We also meticulously reviewed literature from several databases and selected and integrated literature from the Global South and Global North and included insights, deliberately seeking out and including voices and perspectives from marginalised or underrepresented communities, to acknowledge, understand, and synthesise insights from varied cultural backgrounds.
Also, we reviewed a series of case studies and best practices in authentic and real-world assessment design that illustrate how educators can effectively incorporate critical data and AI literacies (Nerantzi et al., 2023; JISC, 2023) in their design, towards addressing ethical considerations, promoting inclusivity, and cultivating critical literacies amongst learners, to empower educators and learners to navigate the complex landscape of data and AI with confidence, fostering a generation of informed and responsible digital citizens.
In our research, we approach ‘critical’ data and AI literacy as the ability to critically analyse, interpret, and evaluate data including that which is produced by and through the use of AI, enabling learners to navigate the data-driven and AI world with discernment and agency (Atenas, Havemann & Timmermann, 2020; Atenas, Havemann & Timmerman, 2023; Govender, 2023; Dyrkolbotn, Pedersen & Slavkovik, 2018). AI literacy can be understood as a set of skills and competencies that enables individuals to critically assess AI technologies while interacting effectively with AI in their personal and professional lives (Ridsdale et al., 2015; D’Ignazio, 2017; Long & Magerko, 2020). AI literacy can be also seen as a component of data literacy, which has been defined by D’Ignazio and Bhargava (2016) as “the ability to read, work with, analyse, and argue with data as part of a broader process of inquiry into the world”.
By integrating critical data and AI literacies into research-based learning and authentic and real-world assessment educators can provide an opportunity for learners to develop a critical understanding of data, as they gain insights into the underlying algorithms and biases, and grapple with ethical dilemmas associated with data use, thereby becoming informed and responsible participants in a data-driven society (Atenas, Havemann & Timmerman, 2023), considering the risks and opportunities data and AI driven technologies to facilitate ‘human flourishing’ and human development (Nussbaum and Sen, 1993).
Through a combination of theoretical insights and practical examples, we aim to equip educators with the knowledge and tools necessary to embrace AI in authentic assessment while upholding the principles of data justice. Furthermore, we analysed a wide range of international initiatives and reflections to map examples and good practices in terms of developing and designing assessment models using AI, to select those that aim at addressing capacity building in critical data and AI literacy that have potential in civic engagement.
Defining Authentic and Real-world Assessment
Authentic assessment is not new, but is useful, practical and valuable, as in most disciplines, this method can help students to develop professional and citizenship skills at individual and collective level. This aspect could be promoted by for example developing products, writing reports, pitching ideas, creating prototypes or assessing – and redesigning processes towards improving practices to demonstrate understanding of their capacities in real world settings, including different layers of employability and participatory skills that are connected with different life scenarios and settings.
While the concept of authentic assessment is thought to have originated in K-12 education in the USA, the use of the term in HE contexts tends to refer to learner engagement in complex tasks that mirror the challenges they may encounter in their professional and personal lives (Nieminen, Bearman and Ajjawi, 2023). In fact, Frey (2013) and Mohamed and Lebar (2017) affirm that an assessment is considered to be authentic when its assignments, content, requirements, and grading techniques correspond to real world needs. It focuses on problem-solving, critical thinking, and the application of knowledge in authentic contexts (Gulikers et al., 2004). As Villarroel et al. (2018) argue, authentic assessment can have an impact on the quality and depth of learning achieved by the student (Wiggins, 1993; Dochy and McDowell, 1997) playing a crucial role in the development of higher-order cognitive skills (Ashford-Rowe, Herrington and Brown 2014; Mohamed & Lebar, 2017; Villarroel et al., 2018).
Gulikers, Bastiaens and Kirschner (2004) define authentic assessment as those that require students to use the same competencies, or combinations of knowledge, skills, and attitudes that they may need to apply their professional life, helping them to produce evidence of learning while it can be measurable under appropriate standards and be relevant for the students.
Authentic assessment processes can improve students’ learning autonomy, commitment and motivation (Raymond et al., 2013; Nicol, Thomson, and Breslin, 2014), but also self-regulation, metacognition and self-reflection skills (Pintrich, 2000; Vanaki and Memarian, 2009). McArthur (2022) notes that authentic assessment is often equated with the performance of so-called ‘real world’ tasks, noting that it is frequently assumed that ‘real world’ is the world of work” and that the value of ‘authenticity’ is economic. She proposes a model that approaches a richer understanding of society as a whole, which is transformative for social change, to prevent reinforce the status-quo.
In general terms, authentic assessment can help students to develop employability and professional skills as well as citizenship and critical thinking skills, addressing competency gaps between education, civic and professional life. This kind of assessment focuses on higher order thinking skills as students are required to apply knowledge creatively to solve problems. Authentic assessment goes beyond traditional testing methods, emphasising real-world, meaningful tasks that mirror the challenges learners may encounter in professional settings.
The idea of authentic assessment is tightly related to the concept of Real World Learning, which have been defined by Morley and Jamil (2021) as curricular activities that can connect learning, industry and society in HE reducing theory – practice and citizenship gaps, while empowering learners in advancing reflective practices and skills required to participate in society.This approach is in contrast with traditional methods of learning and teaching, that are typically focused on academic skills, competences and content, which may not always adequately equip students to become globally aware citizens and graduates ready for the society (Woodside, 2018; Morley and Jamil, 2021; O’Connor and McEwen, 2021).
The concept of civic engagement is wide and has several interpretations depending on the cultural context of HEIs, but in general terms, it can be understood as activities that universities establish as practical learning opportunities with communities.These activities collectively fall under the scholarship of engagement (Boyer, 1996; O’Connor, 2006; Owusu-Agyeman & Fourie-Malherbe, 2021), aiming to address pressing social, civic, and ethical issues by connecting university resources with real-world challenges (Sandmann, 2008), including partnership, reciprocity, mutual benefit, equity, empowerment, community voice, diversity, sustainability, challenge, and interdisciplinarity driven collaboration (O’Connor & McEwen, 2021), which includes that learners can work with data and AI accurately and efficiently to solve real life challenges (Hanney, 2021; Atenas at al., 2023). Figure 1 proposes an overview of these complex elements’ relationships.
Figure 1
Diagram defining real world learning in the context of assessment. Framework promoted by Archer, Morley & Souppez (2021) Real World Learning and Authentic Assessment https://link.springer.com/chapter/10.1007/978-3-030-46951-1_14 (CCBY).
By incorporating elements of critical data and AI literacies into authentic and real-world assessment design, we can harness the power of critically assess data and AI driven insights to provide contextual and relevant tasks to cultivate critical thinking skills. It gives educators a unique opportunity to leverage data and AI to develop innovative assessment practices, that include real life challenges while promoting critical data and AI literacy through a data justice lens, to ensure learners develop the necessary skills to engage responsibly in the digital age. Thus, involving real-world tasks with multiple solutions for the students, enables them to use critical thinking to find their own way to respond to a problem. This means a possible improvement of authenticity into assessment processes taking into account the learning design process with a balanced and objective approach in relation to critically understanding data and AI.
Designing authentic and real-world assessments requires careful consideration of the skills and competencies learners need to develop, thus the integration of critical data and AI literacy can add a layer of complexity and critical thinking, enhancing the relevance of these assessments. Thus it should involve realistic scenarios, aiming at solving a problem, simulating real life contexts, facilitating negotiating and decision-making or using the same kind of resources and technologies students would use in a professional setting.
One of the key challenges in designing authentic and real-world assessment, to develop a critical understanding of data and AI, is related to understanding how machines make automated decisions and how these affect us in every dimension of our lives. Thus, within the incorporation or use of data and AI in assessment, we need to ensure that we mitigate the fear of machines taking over human roles, while embedding a critical sense to evaluate automated decision-making processes (Richardson and Clesham, 2021; Zawacki-Richter et al., 2019). Kamalov et al. (2023) argue that integrating AI into educational settings can help personalise learning experiences, encouraging a more inclusive and effective learning environment, but this can only happen if personalisation is introduced through a critical approach. Indeed, environments can replicate broader systematic oppression in the form of racism or sexism, and to avoid this academics can act through the design of personalised authentic assessments.
Thus, our exploratory approach for supporting critical data and AI literacies in authentic and real-world assessment is embedding the principles of Data justice in assessment design. Data justice is a framework grounded in social justice principles that advocates for fairness, equity, and transparency in data collection, analysis, and use (Dencik, Hintz and Cable, 2017; Dencik & Sanchez-Monedero, 2022). Applying a data justice approach to authentic assessment enabled by data and AI entails a careful consideration of the ethical implications surrounding data collection, algorithmic biases, automated decisions, privacy concerns, and the impact on marginalised or vulnerable communities. Thus, the adoption of this approach aims at ensuring that authentic tasks and its assessment are inclusive, fairly modelled, empowering, and respectful of learners’ rights.
In this sense, the integration of critical data and AI literacies in authentic assessment practices offers exciting possibilities for promoting data justice and critical data and AI literacies in education, for both learners and educators alike. By adopting this approach, educators can leverage AI technologies to create meaningful learning experiences, cultivate ethical awareness, and empower learners to become active participants in shaping a just and equitable data-driven society. In this regard, AI might help open up teaching and learning possibilities that would otherwise be difficult to achieve, question traditional pedagogies, or support instructors in becoming more effective (Agostini & Picasso, 2023). The integration of AI in educational settings, in fact, requires data and AI literacy and this represents a challenge for educators, who are involved in deep changement processes, thus, both instructors must have clear ideas on the possibilities and risks of AI in education to critically integrate these technologies into their practice (Holmes & Tuomi, 2022).
In the sections below, we discuss various examples and case studies that highlight the practical implementation of AI-powered authentic assessments to showcase how educators can create assessment tasks that incorporate real-world data sets, simulate authentic problem-solving scenarios, and promote ethical considerations. Additionally, we will explore the challenges and considerations associated with the use of AI in assessment design, including concerns related to bias, privacy, and algorithmic transparency.
Developing Critical and AI Literacies through Authentic Assessment
Figure 2 proposes an overview about the relationship between data, AI and authentic assessment, the educators and students’ role in this new and dynamic relational flow, in order to sum up the complexity but also the essential intersection on which the protagonists of education are called upon to reflect and work.
Figure 2
Series of flags describing different roles in terms of AI and assessment design. Own work (CCBY).
By designing authentic assessment it is important to have realistic expectations that are according to the level of the students and consider the time students have available to complete the tasks, while encouraging students to be critical, free-thinking and creative, choosing techniques, platforms, tools and technologies. Authentic and real-world assessment can help students to develop a wide range of skills, both practical and intellectual, useful to solve problems in different contexts independently, while promoting and scaffolding lifelong learning skills, involving them in direct and authentic assessment experiences (Sadler, 1989; Agostini & Picasso, 2023).
Academics’ role, when designing authentic and real-world assessments that embed layers of critical data and AI literacy, is to motivate students to develop the skills needed to thrive in the datafied society, providing them with meaningful and well designed tasks, with a clear set of instructions, and scenarios and explaining why these are relevant to their future professional and everyday lives. Furthermore, instructors have to recreate similar settings as in work or citizenship environments, relevant for individual reflection and group work, by providing situational and contextual knowledge including the acquisition of relevant professional and citizenship attitudes and competencies.
The closer the assessments are to real practice and life, the greater the degree of authenticity, thus in its design it is key to include professional bodies, professionals from the sector and civic society, or community organisations, ensuring the task clearly reflects its alignment between learning outcomes, curriculum content, and future evidence-based knowledge. Thus, to integrate employability and citizenship in university settings, authentic and real-world assessment must be focused on the learning rather than in the grades, fostering professional and social reflective and critical skills. Furthermore, these assessment practices consider the challenges students with disabilities and other vulnerable groups may face, to mitigate potential disadvantages in settings that require them to adapt to new situations, thus, the design must include elements of reasonable accommodations.
Based on the models proposed by Lawrie (2023), Matheis and John (2024) and to design robust authentic and real-assessment, we have identified, for this study, from the scoping of diverse initiatives, including “101 creative ideas to use AI in education” by Nerantzi et al. (2023) and “Assessment ideas for an AI enabled world by JISC (2023), examples of activities that can help us advance data and AI literacies are listed below:
Build portfolios that reflect an understanding of data analysis;
Using open data to solve real-life problems that underlie elements of datafication or data ethics;
Writing guides to explain others an issue related with how data is collected to foster automated decision driven processes;
Prepare a training session about a particular topic that relates to how data and AI are changing professional practices in their disciplines;
Plan and design a project that has a component of data and AI, showcasing risks and benefits;
Discuss problems with customers;
Assess a clinical case using both professional decisions AI to compare and assess how humans vs machines respond to clinical information;
Review a legal case and defend or accuse someone, considering personal and AI biases in terms of discrimination, racism etc;
Coding and developing algorithms to automate processes and assess the impact these can have in a sector;
Review and assess different research methods and algorithms to see how these can affect vulnerable groups;
Writing reports and policy briefs effectively communicating data using data storytelling and data visualisations.
For example, different approaches to advance data and AI literacy can be scaffolded to different levels in different disciplines as can be seen in the Table 1 presented below.
Table 1
Examples of approaches, related to specific discipline, to advance data and AI literacy. Own work (CCBY).
| DISCIPLINE | ASSESSMENT MODEL | FIRST YEAR UNDERGRAD LEVEL 4 | SECOND YEAR UNDERGRAD LEVEL 5 | THIRD YEAR UNDERGRAD AND POSTGRAD LEVEL 6 AND 7 |
|---|---|---|---|---|
| Computer Science | Project-Based Learning | Apply basic programming skills to simple datasets. | Develop more complex algorithms for data analysis. | Innovate and optimise algorithms, demonstrating advanced programming skills. |
| Business | Case Study Analysis | Identify basic business data trends. | Analyse and interpret complex business datasets. | Critically evaluate business data, considering ethical and strategic implications. |
| Social Sciences | Research Paper | Discuss social implications of basic AI usage. | Explore the societal impact of AI in specific contexts. | Investigate and analyse complex social issues related to AI, applying critical perspectives. |
| Health Sciences | Group Presentation | Present basic health data insights. | Communicate findings on health-related data, using relevant tools. | Articulate advanced health data insights, considering ethical implications. |
| Engineering | Problem-Solving Exercise | Apply simple AI solutions to engineering problems. | Use intermediate AI techniques to address engineering challenges. | Employ advanced AI methodologies for innovative engineering solutions, evaluating their impact. |
| Environmental Studies | Real world Data Project | Collect and analyse basic environmental data. | Use advanced data analysis for complex environmental projects. | Engage in comprehensive environmental data projects, addressing critical ecological issues. |
| Humanities | Data Ethics Reflection | Reflect on basic ethical considerations in humanities research. | Analyse and reflect on ethical implications of data use in humanities contexts. | Demonstrate a nuanced understanding of data ethics in humanities, proposing ethical frameworks. |
In terms of assessing proficiency in data and AI literacy to advance the learner’s understanding of the impact of data and AI in the society, Table 2 proposed a critical overview about this topic.
Table 2
Prompts for assessing proficiency in data and AI literacy for learners. Own work (CCBY).
| CRITERIA | NOVICE | INTERMEDIATE | PROFICIENT |
|---|---|---|---|
| Understanding Data Concepts | Limited understanding of basic data concepts. | Solid understanding of fundamental data concepts. | Advanced understanding of data concepts, including data types, sources, and formats. |
| Data Analysis Skills | Struggles to analyse and interpret basic data sets. | Capable of conducting basic data analysis and drawing simple conclusions. | Excels in advanced data analysis, employs statistical methods, and derives meaningful insights. |
| AI Awareness and Understanding | Limited awareness of AI concepts and applications. | Clear understanding of AI principles and basic applications. | In-depth understanding of advanced AI concepts, including machine learning algorithms and their practical use. |
| Critical Thinking in AI Context | Struggles to critically evaluate AI outputs and implications. | Demonstrates the ability to critically assess AI models and outputs. | Excels in evaluating AI models, considering ethical implications, and proposing improvements. |
| Communication of Data and AI Insights | Struggles to communicate insights coherently. | Communicates data and AI insights effectively. | Presents complex data and AI findings clearly, using appropriate visualisations and language. |
| Data Ethics and Justice Considerations | Limited awareness of ethical considerations in data and AI. | Recognises basic ethical considerations and their importance. | Demonstrates a deep understanding of data ethics and justice, actively considers and addresses ethical concerns in data and AI practices. |
Incorporating Data Justice into Authentic and Real-world AI Enhanced Assessment Design
For Kamalov et al. (2023), AI systems embedded in assessment can have an important role in terms of process automatisation and quicker high quality and structured tailoring to enhance the students’ learning experience, helping educators to develop complex scenarios promoting problem solving exercises, in a consistent manne (Kochmar et al., 2022; Minn, 2022; Kamalov et al., 2023).
To incorporate Generative AI into authentic and real-world assessment, we need to consider that generative AI technologies can be used to create dynamic and interactive assessment experiences, so educators can design assessments that simulate challenging real-life scenarios that require learners to analyse, interpret, and evaluate data in complex contexts, allowing them to gain first-hand experience in dealing with data-driven challenges and ethical conundrums to develop critical data and AI literacy skills.
In connection to that, in fact, for the personalisation of the learning experience, teachers can use AI and also specifically Large Language Models. These models, for example, can analyse student’s products and create tailored feedback, suggesting materials aligned with their specific learning intended outcomes and formative needs. This kind of opportunity connected to AI use can scaffold university teachers in their practice, helping them saving time and effort in creating personalised resources and feedback, and also allow them to focus on other aspects of teaching, such as creating engaging and interactive lessons (Kasneci et al., 2023, pp. 2–3).
To ensure the ethical and inclusive integration of generative AI into authentic assessment, a data justice lens could be useful to ground learning in real life problems. This involves considering the potential biases, privacy concerns, and social implications associated with the data used and generated by AI models. Educators should prioritise fairness, transparency, and equity in the design and implementation of assessments. Additionally, attention should be given to addressing the concerns of marginalised communities and minimising any potential harm or discrimination (UNESCO -IRCAI, 2024).
Data justice’s role is primarily about redistributing power and resources to address social inequalities (e.g. Dencik, Hintz and Cable, 2019; Milan & Van der Velden, 2016), as it involve recognising the different experiences and perspectives of marginalised communities, as data, embedded in education and assessment, can help learning to understand and challenge power to promote social justice by uncovering the power dynamics that shape data-related practices (Couldry & Yu, 2018; Taylor, 2017; van Dijck & Poell, 2018).
Data justice highlights the importance of ensuring that data is used fairly, equitably, and in the interest of the public good. One of the key ideas of this approach is the idea that data is not neutral and that it can be used to reinforce existing power structures and inequalities, as data can be biased, incomplete, or manipulated to serve the interests of those who control it, such as corporations, governments, or institutions, leading to discrimination, exclusion, and other forms of social harm (Figure 3).
Figure 3
Series of rhombi describing the levels of power and arrows showing how it manifests. Source – Understanding the levels at which power operates in the collection and use of data, and how it manifests – Data Justice in Practice: A Guide for Impacted Communities (CCBY).
Data justice promotes the idea that the collection, use, and distribution of data should be fair and equitable, and should not perpetuate social inequalities. The concept has emerged in response to concerns about the increasing role of data in society, and the ways in which data-related practices can have unequal effects on different groups (Heeks & Swain, 2018; Dencik & Sanchez-Monedero, 2022), as data-related practices can perpetuate racial and gender inequalities, and calls for greater attention to the ways in which data is used to marginalise and exclude certain groups (Eubanks, 2018). For example, Boyd and Crawford (2012) and Mittelstadt et al. (2016) argue that data-related practices can have unintended consequences, and call for greater attention to the ethical implications of data use such as privacy, consent, and transparency.
Ideas for Robust Design and Implementation AI Enhanced Authentic and Real-world Assessment
Authentic and real-world assessment design using generative data and AI can facilitate the development of critical data literacy skills. Learners can engage with AI-generated data, algorithms, and models, enabling them to understand the underlying processes, question biases, and critically evaluate the ethical implications of AI-generated outputs. This fosters a deeper understanding of the complexities surrounding data use and equips learners with the ability to navigate and participate responsibly in the data-driven society.
Starting from the Swiecki et al. (2022) study, where they affirm that
“AI-based techniques have been developed to fully or partially automate parts of the traditional assessment practice. AI can generate assessment tasks, find appropriate peers to grade work, and automatically score student work. These techniques offload tasks from humans to AI and help to make assessment practices more feasible to maintain” (Swiecki et al., 2022, p.2).
It is possible to affirm that, when designing authentic assessments using generative data and AI, several considerations should be taken into account in terms of risks and biases (UNESCO-IRCAI, 2024). Clear learning objectives should be defined, aligning with the desired critical data literacy outcomes. The assessment tasks should be authentic, meaningful, and relevant to real world contexts. Careful attention should be given to the quality and diversity of the data used, ensuring it represents a range of perspectives and avoids perpetuating biases. Furthermore, providing opportunities for reflection, self-assessment, and feedback can enhance the learning experience and support learners’ growth in critical data literacy.
Taylor (2017) suggests a data justice approach based on three pillars: visibility, engagement with technologies and non discrimination “to determine not only what, but who is important and how they relate to the desired outcomes” (p.8) aiming to promote social justice in relation with technologies. Integrating these pillars in authentic and real-world assessment design allow us to support our learners in engaging with research ethics and human rights principles, advance their digital literacies and understanding the role of agency in the datafied society, to develop values of civic participation while becoming empowered citizens and pillars of data justice in relation with authentic and real world assessment design (Figure 4).
Figure 4
Pillars of data justice in relation with authentic and real world assessment design. Adapted from Taylor (2017) – Own work (CCBY).
Some ideas with examples on how to design effective authentic assessments to foster critical data literacy using generative AI under a data justice lens.
1. Case Study Analysis
Design a case study assessment where learners are provided with a dataset generated by a generative AI model. They must critically analyse the dataset, identify potential biases or ethical concerns, and propose strategies to address them. For example, learners could analyse a dataset generated by a language model to examine gender or racial biases in the generated text and suggest ways to mitigate those biases.
2. Algorithmic Accountability Project
Assign learners to investigate an AI algorithm that has real-world implications. They can critically analyse its dataset, model architecture, and decision-making processes. Learners should assess whether the algorithm is aligned with data justice principles, such as fairness and equity. For instance, learners could examine an AI system used for automated hiring decisions and evaluate its potential bias towards certain demographics, highlighting the need for fairness and transparency in algorithmic decision-making.
3. Data Privacy Impact Assessment
Ask learners to conduct a data privacy impact assessment on a generative AI application or platform. They should analyse the data collection and storage practices, assess the privacy risks, and propose measures to protect individuals’ privacy. For example, learners could assess a generative AI-based social media platform, evaluating the platform’s data collection policies and suggesting privacy-enhancing features, such as user-controlled data sharing and transparent data governance mechanisms.
4. Collaborative Data Storytelling
Engage learners in a group project where they collaboratively create a data-driven story using generative AI. Learners should critically examine the data sources, identify potential biases, and craft a narrative that highlights the ethical and social implications of the generated content. For instance, learners could use generative AI to create a virtual news article and critically analyse the potential impact of biassed or misleading information on public perception and societal discourse.
5. Ethical Data Simulation
Develop a simulation exercise where learners engage with generative AI-generated data and face ethical dilemmas related to data use. Learners must make decisions that balance the potential benefits of using the generated data with the ethical concerns involved. For example, learners could simulate a scenario where they have access to healthcare data generated by AI models and must determine how to responsibly use and share that data while protecting individuals’ privacy and ensuring equitable healthcare outcomes.
6. Bias Mitigation Strategies
Challenge learners to design and implement bias mitigation strategies for a generative AI system. They should critically analyse the training data, identify biases, and experiment with techniques to reduce or eliminate them. For instance, learners could work with a text generation model and explore methods such as dataset augmentation, algorithmic debiasing, or inclusive data collection to mitigate gender or racial biases in the generated text.
7. Civic Engagement through Data Advocacy
Encourage learners to engage in data advocacy by identifying a social issue and leveraging generative AI to raise awareness or inspire action. Learners can generate data visualisations, infographics, or persuasive narratives to communicate the issue effectively. They should critically reflect on the ethical implications of data representation and the potential impact of their advocacy work. For example, learners could use generative AI to create visualisations that highlight disparities in access to education or healthcare, advocating for equitable opportunities and policy changes.
For Group Work Assessment, some ideas are listed below:
1. Data Justice Impact Project
Form groups of learners to collaborate on a project that assesses the impact of generative AI on data justice in a specific domain. Each group can choose a different context, such as healthcare, finance, or education. They should critically analyse the potential risks, biases, and social implications of using generative AI in that domain. Groups can propose recommendations, policies, or interventions to address data justice concerns. For example, a group could explore the impact of generative AI in personalised medicine and suggest guidelines for ensuring equitable access and informed consent.
2. Designing Ethical AI Guidelines
Assign groups of learners to collaboratively develop ethical guidelines or a code of conduct for the responsible use of generative AI in their respective fields. They should critically consider the ethical challenges, potential biases, and human rights implications of using generative AI. Groups can explore case studies, consult relevant literature, and engage in discussions to inform their guidelines. For instance, a group of engineering students can design ethical guidelines for using generative AI in autonomous vehicles.
3. Data Justice Impact Assessment
Task groups of learners to conduct a data justice impact assessment on a generative AI application or platform. They should critically evaluate the platform’s data collection practices, algorithmic decision-making, and potential risks to privacy, fairness, and social justice. Groups can create reports or presentations that highlight the impact assessment findings and propose recommendations for improving data justice. For example, a group could assess a social media platform using generative AI and suggest measures to address algorithmic biases and protect users’ privacy.
4. Data Journalism
Propose activities to groups of students following the principles of data journalism in terms of “obtaining, reporting on, curating and publishing data in the public interest”, then any activity involving data in connection with journalistic reporting and editing (Stray, 2011; Coddington, 2015). As a team, students could be supported in the creation of reports connected to specific scientific or disciplinary areas using the data produced by the use of AI. Once they have researched the data, they will, as a group, check the reliability of the collected information, converge the relevant data and produce a journalistic report. This activity can be also combined with a moment of peer assessment: each group will assess, according to criteria co-constructed and shared with the teacher, the work of another group, providing and receiving feedback at the same time on their own work and on the general use of data, also in a journalistic key. This activity can support AI literacy skills in terms of discrimination and veracity of the data produced and also support processes of scientific dissemination by students.
In Table 3, we discuss practical examples of AI introduction in assessment processes, proposing different approaches in terms of design and implementations based on the report “101 creative ideas to use AI in education” by Nerantzi et al. (2023), which includes practical prompts and examples to assessment design and practices with the use of AI in education, and from a report called “Assessment ideas for an AI enabled world by JISC (2023), which proposes different approaches to AI introduction in educational context, with a specific focus on assessment practices.
Table 3
Examples of authentic assessment design and implementation through the use and with the support of specific AI systems. Own work (CCBY).
| TYPE | DATA JUSTICE PILLARS | EXAMPLE | CRITICAL DATA AND AI LITERACY |
|---|---|---|---|
| Scenario Based | Engagement with technology | Academics could investigate the notion of introducing (or having their students enter) the parameters of their assessments into technologies like ChatGPT and asking it to generate a real world brief by acting as a ‘client’ in the context of their discipline. ChatGPT, or another GenerativeAI system, was requested to play as a client for a marketing firm and develop a brief for a marketing expert to construct a campaign for an undisclosed product, including a budget, timeframe, and market reach. When students enter their evaluation settings, GenAI system develops a personalised task for them and generates a unique situation each time. It also helps increase the legitimacy and applicability of the evaluation (Nerantzi et al., 2023). | Competencies related to the selection of specific input data to introduce in the GenAI system but also the competence of critiquing and interpreting the data produced by the GenAI itself (Ng et al., 2021). |
| Students were introduced to a research proposal assessment that required them to propose a scenario-based research challenge and create a research study. Students were given an AI-generated study proposal, which they then reviewed in groups before sharing their findings with the rest of the group. A further benefit is the ease with which instructors may employ AI to create instructional tools. The goal was not just to deploy AI technologies to help students consolidate discipline-specific abilities, but also to illustrate the advantages and disadvantages of specific GenAI systems (Nerantzi et al., 2023). | Competences related to “work with, analyse, and argue with data as part of a broader process of inquiry into the world” (D’Ignazio, 2017) and “communicate and collaborate effectively with AI” (Long & Magerko, 2020, p. 2). | ||
| Case Studies | Engagement with technology | Using AI as a thought partner in the development of branching scenarios to provide learners with realistic critical thinking evaluations. Creating case studies can be time-consuming, and existing ones may lack essential components for a successful learning experience. Using AI, we can create not just a complete case study, but also pertinent discussion, feedback, and branching that lead the learner on a content-rich journey (Nerantzi et al., 2023). | Development of critical thinking assessment skills, promoted through the use of AI as a tool to create real world authentic scenarios, in a safe formative online environment (Cui et al., 2023). |
| Students choose a real world example of how AI has changed some element of practice, such as voting in a political election, financial decision making, parole judgements in law, or medical diagnosis. They then examine the ramifications and repercussions of the case, assessing the role it plays in occupations that may be related to their discipline but require specialised application. They can also identify some essential talents or attributes that they may need to enhance or develop in their current work or potential future career paths (JISC, 2023). | This specific activity could support the development of competences such as critical evaluation, AI literacy (e.g.ethics and data protection) and metacognition (Ashford-Rowe, Herrington, and Brown 2014; Mohamed & Lebar, 2017; Ng et al., 2021). | ||
| Human VS AI written assessment | Engagement with technology Visibility | Encourage students to analyse an article written by a person vs one created by a GenAI system. Ask students to review the GenAI output and provide instances of statements that appear plausible but are incorrect or do not make sense. Stimuli students to verify the sources and references. Ask them to identify any gaps in GenAI’s coverage and to share their views regarding the terminology introduced. The primary goal is to assist learners to strengthen their critical analytical abilities by making judgements (Nerantzi et al., 2023). | Development of data and AI literacy competencies related to the possibility of accessing, critically evaluating and using data sources (Prado & Marzal, 2013; Shields, 2005). |
| The goal is for students to question the authenticity, correctness, or applicability of ChatGPT (or other GenAI system) replies to their assessment titles, and then use Google to determine where the information came from. This then leads to a fact-checking exercise in which they verify or expand on what ChatGPT proposes. As a result, they just consider ChatGPT as a beginning point which leads to serious academic study. This might be done immediately, asynchronously, or as a preliminary stage before beginning, such as an annotated bibliography or text analysis exercise (Nerantzi et al., 2023). | “Encourage learners to investigate who created the dataset, how the data was collected, and what the limitations of the dataset are. This may involve choosing datasets that are relevant to learners’ lives” (Long & Magerko, 2020, p.6). | ||
| Project based | Non discrimination | Students develop a product that answers a real-life problem, e.g. practical solution for engineering or computers, a professional development template for a business student. Having completed a draft, they submit it to peers and/or stakeholders for comment. Students may use AI to create ideas, improve presentations, seek guidance on component selection, and more. They create a reflective narrative (or exhibition) to support their product, which details the design process, decisions taken, teamwork, and stakeholder participation. Realistic management of the scope of work and available resources is required (JISC, 2023). | This activity design could support the development of students’ competencies related to the sphere of metacognition, research, practical competence, assessment literacy, collaboration and they can become able to collect and critically analyse the data produced from AI (Sadler, 1989; Long & Magerko, 2020). |
| Students received an assignment, such as writing a policy evaluation of food security reform in a global context or curating an exhibition on a topic covered in their course. They pick their topic by the third week of the module. Early in the module, participants submit/present brief drafts (with instructions on what these should include) for comments from staff and students. AI may be used to produce ideas, which can then be questioned, changed, and merged as needed. They receive three pieces of input on the draft and they submit and present a final draft but do not receive comments on areas that they had the opportunity to submit and gain input on earlier in the semester (JISC, 2023). | This formative design can sustain the development of students’ competencies connected to research, planning, general/key professional and assessment skills, assessment skills, problem solving and AI literacy (Sadler, 1989; Ng et al., 2021). |
Starting from the categorisation of the assessment processes (Controlled Condition Exams; Take-Home Papers/Open Book; Quizzes & In-class Tests; Practical Exams; Dissertation; Coursework and other Assessments), JISC proposed different criteria to classify the characteristics of the AI assessment practices proposed, so 1. Authenticity; 2. Challenge; 3. Product; 4. Learning; 5. Staff demand; 6. Lifelong Learning.
Conclusions and Future Research Perspectives
Different benefits and possible limitations are described in this study: the importance of the human role is underlined in order to balance the design and the implementation of data and AI driven approaches and technology grounded on the values of social and data justice in teaching, learning and assessment practices, towards mitigate and reflect on biases (Ifelebuegu, 2023; UNESCO-IRCAI, 2024). The possibility to effectively use AI to design and scaffold assessment personalisation (Kamalov et al., 2023), ensuring to maintain high quality models for assessment design and implementation that foster real and authentic learning experiences, developing complex and critical skills fundamental for employability and citizenship purposes and the professional and personal students’ growth (Mohamed & Lebar, 2017).
The principles and values of data justice can help educators and learners to understand the intersecting axes of social inequality, such as race, gender, and class (Noble, 2018) and to operationalise intersectionality in data justice considering the ethical implications of data-related practices, including issues such as consent, privacy, and algorithmic bias (Boyd & Crawford, 2012) rethinking of data-related practices to ensure that they are aligned with ethical values and principles (Kitchin & Lauriault, 2014; Mittelstadt et al., 2016). To incorporate Data Justice into the HE Curriculum through authentic and real-world assessment, it is key to start by embedding their pillars: visibility, engagement with technology and non-discrimination alongside with other data ethics principles topics such as data privacy, and responsible data governance. We argue that the HE sector should embrace community-engaged, research and evidence-based assessment practices that advances social justice, addressing data driven inequities and analyse the impact of data and AI driven policies and technologies on marginalised or vulnerable communities.
For example, authentic and real-world assessment developed using AI design, must incorporate the strand of work in the data justice literature that is focused on the role of power in shaping data-related practices, addressing power imbalances within the data ecosystem, and ensuring that marginalised groups are able to participate in decisions about data-related practices calling for a more democratic and participatory approach to data governance (Milan and Van der Velden, 2016; Dencik, Hintz and Cable, 2017).
In terms of critical data and AI literacy, we argue that is important to encourage learners to be critical consumers of AI technologies by questioning their intelligence and trustworthiness (Long & Magerko, 2020, p. 9); moreover, as academics we must aim to empower the development of critical assessment skills, connected to the opportunity to improve competences related to visualise, clean and interpret data (Wolff et al., 2016). The design of authentic and real-world assessment experiences, if properly structured, could really sustain the ability to use data and AI in a conscious and informed way, highlighting the importance to strike a balance between encouraging interesting interactions and exposing students to various types of artificial intelligence (Havemann et al. 2023). This may include creating social, embodied learning experiences for more prevalent AI systems, also promoting transparency in all aspects of AI design (D’Ignazio, 2017; Long & Magerko, 2020).
In connection to that, AI applications may be used by educational researchers and practitioners to build appropriate assessment methods; besides, educators may help students to acquire engage with technology, to develop digital competencies and to critically yet efficiently interact and work with AI systems, in addition to obtaining knowledge and abilities in utilising them (Ng et al., 2023). Students have to be scaffolded from the development of these new specific skills in order to become critical data and AI practitioners who can work with data and AI while also critically challenging it; in light of this, the first challenge, from an institutional perspective, is connected effectively building capacities for academics. In fact, education professionals are the first players in the innovation process and therefore need to take a leading role for students, supporting the development of specific skills to cope with a data-driven society (Van Es and Schäfer, 2017).
Educators should reinforce their technological understanding in machine learning, natural language processing, and software development to educate students AI knowledge and abilities; more professional development programmes and institutional support is necessary to upskill and reskill teachers› AI related knowledge and competencies. In fact, as Ng et al. (2023) affirm, to create a meaningful curriculum, educators should employ appropriate instructional materials and pedagogies, as well as engaging features like games, curiosity, creativity, authenticity, cooperation, and competition (Ng et al., 2023).
The design of the learning experience as well as the assessment of learning, must in fact be structured by professionals capable to relate to the benefits and challenges proposed by the use of AI in education, profiles therefore with a deep reflective eye on the needs of students and able to maximise the learning experience through the new digital tools, while mastering the ethical use of data (Kamalov et al., 2023). Only starting from a careful and continuous professional development programmes, will it be possible to support the acquisition and advancement of data and AI literacy in students, which must be placed at the heart of learning and teaching and processes, through authentic and real life related experiences (Gulikers et al., 2004; Frey, 2013).
The next step of the research, in fact, will be characterised by the creation and validation of a specific theoretical framework focused on AI literacy and Academic Development: in the light of the evidence proposed, it seems urgent to define the competences for academics in the university contexts, to promote an efficient, ethical and responsible use AI and data and to scaffold the development of AI and data literacy for students (Long & Magerko, 2020). What are the specific steps? What is the perceived level of competences amongst academics? What are their formative needs? How is their professional profile changing? And what can we do as practitioners, researchers and academic developers to sustain a thoughtful transformation from traditional and digital, to a datafied and AI enriched but ethical educational setting?
These are the crucial questions that we have to ask in order to promote justice driven effective collaboration between institutions, educators and students in order to facilitate real change for the enhancement and innovation of university education.
Ethics and Consent
This study is of exploratory nature based on the used of published resources, so no Ethics Approval has been sought, yet we have used an ethical use of the resources reviewed.
Competing Interests
The authors have no competing interests to declare.
Author Contributions (CRediT)
Federica Picasso; Javiera Atenas; Leo Havemann and Anna Serbati: The four authors equally contributed to the ideas; formulation or evolution of overarching research goals and aims, as well as in the review of the literature, qualitative data analysis, and preparation, creation and/or presentation of the published work, both in the initial and the final draft. All authors have read and agreed to the submitted version of the manuscript.