Advancing Critical Data and AI Literacies Through Authentic and Real-World Assessment Design Using a Data Justice Approach
References
- Agostini, D., & Picasso, F. (2023). Large Language Models for Sustainable Assessment and Feedback in Higher Education: Towards a Pedagogical and Technological Framework. CEUR Workshop Proceedings (CEUR-WS.org).
https://ceur-ws.org/Vol-3605/ - Almeida, C. P. B. D., & Goulart, B. N. G. D. (2017). How to avoid bias in systematic reviews of observational studies. Revista CEFAC, 19, 551–155. 10.1590/1982-021620171941117
- Archer, M., Morley, D. A., & Souppez, J.-B. R. G. (2021). Real World Learning and Authentic Assessment. Applied Pedagogies for Higher Education, 323–341. 10.1007/978-3-030-46951-1_14
- Ashford-Rowe, K., Herrington, J., & Brown, C. (2014). Establishing the critical elements that determine authentic assessment. Assessment & Evaluation in Higher Education, 39(2), 205–222. 10.1080/02602938.2013.819566
- Atenas, J., Havemann, L., Rodés, V., & Podetti, M. (2023). Critical data literacy in praxis: An open education approach for academic development. Edutec, Revista Electrónica de Tecnología Educativa, 84, 49–67. 10.21556/edutec.2023.85.2851
- Atenas, J., Havemann, L., & Timmermann, C. (2020). Critical literacies for a datafied society: academic development and curriculum design in higher education. Research in Learning Technology, 28, 2020. 10.25304/rlt.v28.2468
- Atenas, J., Havemann, L., & Timmermann, C. (2023). Reframing data ethics in research methods education: a pathway to critical data literacy. International Journal of Educational Technology in Higher Education, 20(1), 1–27. 10.1186/s41239-023-00380-y
- Atenas, J., Nerantzi, C., & Bussu, A. (2023). A conceptual approach to transform and enhance academic mentorship: Through open educational practices. Open Praxis, 15(4), 271–287. 10.55982/openpraxis.15.4.595
- Boyer, E. L. (1996). The scholarship of engagement. Bulletin of the American Academy of Arts and Sciences, 49(7), 18–3. 10.2307/3824459
- Boyd, D., & Crawford, K. (2012). Critical questions for big data: Provocations for a cultural, technological, and scholarly phenomenon. Information, communication & society, 15(5), 662–679. 10.1080/1369118X.2012.678878
- Brand, J., & Sander, I. (2020). Critical data literacy tools for advancing data justice: A guidebook. Data Justice Lab.
https://datajusticelab.org/wp-content/uploads/2020/06/djl-data-literacy-guidebook.pdf - Coddington, M. (2015). Clarifying journalism’s quantitative turn: A typology for evaluating data journalism, computational journalism, and computer-assisted reporting. Digital journalism, 3(3), 331–348. 10.1080/21670811.2014.976400
- Coeckelbergh, M. (2020). AI ethics. The Mit Press. ISBN: 9780262538190. 10.7551/mitpress/12549.001.0001
- Couldry, N., & Hepp, A. (2018). The mediated construction of reality. John Wiley & Sons.
- Couldry, N., & Yu, J. (2018). Deconstructing datafication’s brave new world. New media & society, 20(12), 4473–4491. 10.1177/1461444818775968
- Cui, Y., Chen, F., Lutsyk, A., Leighton, J. P., & Cutumisu, M. (2023). Data literacy assessments: a systematic literature review. Assessment in Education: Principles, Policy & Practice, 30(1), 76–96. 10.1080/0969594X.2023.2182737
- Dencik, L., Hintz, A., & Cable, J. (2017). Towards data justice. Data Politics, 167. 10.4324/9781315167305-9
- Dencik, L., Hintz, A., & Cable, J. (2019). Towards data justice? The ambiguity of anti-surveillance resistance in political activism. Big Data & Society, December, 1–12. 10.1177/2053951716679678
- Dencik, L., & Sanchez-Monedero, J. (2022). Justicia de datos. Revista Latinoamericana de Economía y Sociedad Digital. 10.53857/kynu7699
- D’Ignazio, C. (2017). Creative data literacy: Bridging the gap between the data-haves and data-have nots. Information Design Journal, 23(1), 6–18. 10.1075/idj.23.1.03dig
- D’Ignazio, C., & Bhargava, R. (2016). DataBasic: Design principles, tools and activities for data literacy learners. The Journal of Community Informatics, 12(3), 83–107.
https://hdl.handle.net/1721.1/123450 . 10.15353/joci.v12i3.3280 - Dochy, F., & McDowell, L. (1997). Assessment as a Tool for Learning. Studies in Educational Evaluation, 23 (4), 279–298. 10.1016/S0191-491X(97)86211-6
- Dyrkolbotn, S., Pedersen, T., & Slavkovik, M. (2018). On the distinction between implicit and explicit ethical agency. In Proceedings of the 2018 AAAI/ACM Conference on AI, Ethics, and Society (pp. 74–80)
https://dl.acm.org/doi/pdf/10.1145/3278721.3278769 . 10.1145/3278721.3278769 - Eubanks, V. (2018). Automating inequality: How high-tech tools profile, police, and punish the poor. St. Martin’s Press.
- Frey, B. B. (2013). Modern classroom assessment. SAGE Publications. 10.4135/9781506374536
- Gianini, F. (2023). Generative AI and the future of education.
https://unesdoc.unesco.org/ark:/48223/pf0000385877 - Govender, K. (2023). Age of Agency: Rise with AI. CRC Press. 10.4324/9781032684895
- Gulikers, J. T. M., Bastiaens, T. J., & Kirschner, P. A. (2004). A five-dimensional framework for authentic assessment. Educational Technology Research and Development, 52(3), 67–86. 10.1007/bf02504676
- Hanney, R. (2021).
Making Projects Real in a Higher Education Context . In D. A. Morley, & M. G. Jamil (eds.), Applied Pedagogies for Higher Education. Cham: Palgrave Macmillan. 10.1007/978-3-030-46951-1_8 - Havemann, L., Corti, P., Atenas, J., Nerantzi, C., & Martínez-Arboleda, A. (2023). Making the case: Opening education through collaboration. Rivista di Digital Politics, 3(2), 305–326. 10.53227/108468
- Heeks, R., & Swain, S. (2018). An Applied Data Justice Framework: Analysing Datafication and Marginalised Communities in Cities of the Global South. Development Informatics Working Paper, 74. 10.2139/ssrn.3425885
- Holmes, W., & Tuomi, I. (2022). State of the art and practice in AI in education. European Journal of Education, 57(4), 542–570. 10.1111/ejed.12533
- Ifelebuegu, A. (2023). Rethinking online assessment strategies: Authenticity versus AI chatbot intervention. Journal of Applied Learning and Teaching, 6(2). 10.37074/jalt.2023.6.2.2
- JISC. (2023). Assessment ideas for an AI enabled world. (2023, September 6). Repository.jisc.ac.uk.
https://repository.jisc.ac.uk/9234/ - Kamalov, F., Santandreu Calonge, D., Gurrib, I. (2023). New Era of Artificial Intelligence in Education: Towards a Sustainable Multifaceted Revolution. Sustainability, 15, 12451. 10.3390/su151612451
- Kasneci, E., Seßler, K., Küchemann, S., Bannert, M., Dementieva, D., Fischer, F., Gasser, U., Groh, G., Günnemann, S., Hüllermeier, E., Krusche, S., Kutyniok, G., Michaeli, T., Nerdel, C., Pfeffer, J., Poquet, O., Sailer, M., Schmidt, A., Seidel, T., Stadler, M., & Kasneci, G. (2023). ChatGPT for good? On opportunities and challenges of large language models for education. Learning and individual differences, 103, 102274. 10.35542/osf.io/5er8f
- Kitchin, R., & Lauriault, T. (2014).
Towards critical data studies: Charting and unpacking data assemblages and their work . In J. Eckert, A. Shears, & J. Thatcher, (eds.), Geoweb and Big Data. University of Nebraska Press. Available at SSRN:https://ssrn.com/abstract=2474112 - Kochmar, E., Vu, D. D., Belfer, R., Gupta, V., Serban, I. V., & Pineau, J. (2022). Automated Data-Driven Generation of Personalized Pedagogical Interventions in Intelligent Tutoring Systems. International Journal of Artificial Intelligence in Education. 10.1007/s40593-021-00267-x
- Kordzadeh, N., & Ghasemaghaei, M. (2022). Algorithmic bias: Review, synthesis, and future research directions. European Journal of Information Systems, 31(3), 388–809. 10.1080/0960085X.2021.1927212
- Lawrie, G. (2023). Establishing a delicate balance in the relationship between artificial intelligence and authentic assessment in student learning. Chemistry Education Research and Practice, 24(2), 392–393. 10.1039/D3RP90003G
- Long, D., & Magerko, B. (2020). What is AI literacy? Competencies and design considerations. In Proceedings of the 2020 CHI conference on human factors in computing systems (pp. 1–16). 10.1145/3313831.3376727
- Matheis, P., & John, J. J. (2024).
Reframing Assessments: Designing Authentic Assessments in the Age of Generative AI . In Academic Integrity in the Age of Artificial Intelligence (pp. 139–161). IGI Global. 10.4018/979-8-3693-0240-8.ch008 - McArthur, J. (2022). Rethinking authentic assessment: work, well-being, and society. Higher Education, 85(1), 85–101. 10.1007/s10734-022-00822-y
- Milan, S., & Velden, L. V. D. (2016). The alternative epistemologies of data activism. Digital culture & society, 2(2), 57–74. 10.14361/dcs-2016-0205
- Minn, S. (2022). AI-assisted knowledge assessment techniques for adaptive learning environments. Computers and Education: Artificial Intelligence, 3, 100050. 10.1016/j.caeai.2022.100050
- Mittelstadt, B. D., Allo, P., Taddeo, M., Wachter, S., & Floridi, L. (2016). The ethics of algorithms: Mapping the debate. Big Data & Society, 3(2). 10.1177/2053951716679679
- Mohamed, R., & Lebar, O. (2017). Authentic assessment in assessing higher order thinking skills. International Journal of Academic Research in Business and Social Sciences, 7(2), 466–476. 10.6007/IJARBSS/v7-i2/2021
- Moore, S., Hedayati-Mehdiabadi, A., Law, V., & Kang, S. P. (2024). The Change We Work: Professional Agency and Ethics for Emerging AI Technologies. TechTrends, 1–10. 10.1007/s11528-023-00895-1
- Morley, D. A., & Jamil, M. G. (2021). Introduction: Real World Learning—Recalibrating the Higher Education Response Towards Application to Lifelong Learning and Diverse Career Paths. Applied Pedagogies for Higher Education, 1–17. 10.1007/978-3-030-46951-1_1
- Nerantzi, C., Abegglen, S., Karatsiori, M., & Martínez-Arboleda, A. (2023). 101 Creative ideas to use AI in education.
https://zenodo.org/records/8072950 - Nicol, D., Thomson, A., & Breslin, C. (2014). Rethinking feedback practices in higher education: a peer review perspective. Assessment & Evaluation in higher education, 39(1), 102–122. 10.1080/02602938.2013.795518
- Nieminen, J. H., Bearman, M., & Ajjawi, R. (2023). Designing the digital in authentic assessment: is it fit for purpose? Assessment & Evaluation in Higher Education, 48(4), 529–543. 10.1080/02602938.2022.2089627
- Ng, D. T. K., Lee, M., Tan, R. J. Y., Hu, X., Downie, J. S., & Chu, S. K. W. (2023). A review of AI teaching and learning from 2000 to 2020. Education and Information Technologies, 28(7), 8445–8501. 10.1007/s10639-022-11491-w
- Ng, D. T. K., Leung, J. K. L., Chu, K. W. S., & Qiao, M. S. (2021). AI literacy: Definition, teaching, evaluation and ethical issues. Proceedings of the Association for Information Science and Technology, 58(1), 504–509. 10.1002/pra2.487
- Noble, S. U. (2018). Algorithms of Oppression: How Search Engines Reinforce Racism. New York University Press. 10.2307/j.ctt1pwt9w5
- Nussbaum, M., & Sen, A. (Eds.). (1993). The quality of life. Clarendon press. 10.1093/0198287976.001.0001
- O’Connor, J. S. (2006). Civic engagement in higher education. Change: The Magazine of Higher Learning, 38(5), 52–58. 10.3200/CHNG.38.5.52-58
- O’Connor, K., & McEwen, L. (2021). Real World Learning Through Civic Engagement: Principles, Pedagogies and Practices. Springer EBooks, 63–89. 10.1007/978-3-030-46951-1_4
- Owusu-Agyeman, Y., & Fourie-Malherbe, M. (2021). Students as partners in the promotion of civic engagement in higher education. Studies in Higher Education, 46(6), 1241–1255. 10.1080/03075079.2019.1666263
- Pechenkina, E. (2024). Artificial intelligence for good? Challenges and possibilities of AI in higher education from a data justice perspective. Open Book Publishers, 239–266. 10.11647/obp.0363.09
- Perla, L., & Vinci, V. (2023).
Enhancing Authentic Assessment in Higher Education: leveraging Digital Transformation and Artificial Intelligence . In AIxEDU 2023–High-performance Artificial Intelligence Systems in Education (Vol. 3605, pp. 1–7). CEUR-WS.https://ceur-ws.org/Vol-3605/5.pdf - Pintrich, P. R. (2000). Multiple goals, multiple pathways: The role of goal orientation in learning and achievement. Journal of Educational Psychology, 92(3), 544. 10.1037/0022-0663.92.3.544
- Prado, J., & Marzal, M. A. (2013). Incorporating Data Literacy into Information Literacy Programs: Core Competencies and Contents. Libri, 63(2), 123–134. 10.1515/libri-2013-0010
- Raymond, J. E., Homer, C. S., Smith, R., & Gray, J. E. (2013). Learning through authentic assessment: An evaluation of a new development in the undergraduate midwifery curriculum. Nurse education in practice, 13(5), 471–476. 10.1016/j.nepr.2012.10.006
- Richardson, M., & Clesham, R. (2021). Rise of the machines? The evolving role of AI technologies in high-stakes assessment. London Review of Education, 19(1). 10.14324/lre.19.1.09
- Ridsdale, C., Rothwell, J., Smit, M., Ali-Hassan, H., Bliemel, M., Irvine, D., Kelley, D., Matwin, S., & Wuetherick, B. (2015). Strategies and best practices for data literacy education. Knowledge synthesis report. 10.13140/RG.2.1.1922.5044
- Sadler, D. R. (1989). Formative assessment and the design of instructional systems. Instructional science, 18(2), 119–144. 10.1007/BF00117714
- Sander, I. (2023). Critical datafication literacy–a framework for educating about datafication. Information and Learning Sciences. 10.1108/ILS-06-2023-0064
- Sandmann, L. R. (2008). Conceptualization of the scholarship of engagement in higher education: A strategic review 1996–2006. Journal of Higher Education and Outreach and Engagement, 12(1), 91–104.
- Shields, M. (2005). Information literacy, statistical literacy, data literacy. IASSIST quarterly, 28(2–3), 6–6. 10.29173/iq790
- Stray, J. (2011). A Computational Journalism Reading List. Retrieved from
http://jonathanstray.com/a-computational-journalism-reading-list - Swiecki, Z., Khosravi, H., Chen, G., Martinez-Maldonado, R., Lodge, J. M., Milligan, S., Selwyn, N., & Gašević, D. (2022). Assessment in the age of artificial intelligence. Computers and Education: Artificial Intelligence, 3, 100075. 10.1016/j.caeai.2022.100075
- Taylor, L. (2017). What is data justice? The case for connecting digital rights and freedoms globally. Big Data & Society, December, 1–14. 10.1177/2053951717736335
- Torraco, R. J. (2005). Writing Integrative Literature Reviews: Guidelines and Examples. Human Resource Development Review, 4(3), 356–667. 10.1177/1534484305278283
- UNESCO-IRCAI (2024). “Challenging systematic prejudices: an Investigation into Gender Bias in Large Language Models”.
- Van Dijck, J., & Poell, T. (2018). Social media platforms and education. The SAGE handbook of social media, 579–591. 10.4135/9781473984066.n33
- Van Es, K., & Schäfer, M. T. (2017). The datafied society. Studying culture through data. Amsterdam University Press. 10.5117/9789462981362
- Vanaki, Z., & Memarian, R. (2009). Professional ethics: beyond the clinical competency. Journal of professional nursing, 25(5), 285–291. 10.1016/j.profnurs.2009.01.009
- Villarroel, V., Bloxham, S., Bruna, D., Bruna, C., & Herrera-Seda, C. (2018). Authentic assessment: creating a blueprint for course design. Assessment & Evaluation in Higher Education, 43(5), 840–854. 10.1080/02602938.2017.1412396
- Wiggins, G. (1993). Assessing Student Performance. Jossey-Bass. 10.5860/choice.31-5581
- Wolff, A., Gooch, D., Montaner, J. J. C., Rashid, U., & Kortuem, G. (2016). Creating an under-standing of data literacy for a data-driven society. The Journal of Community Informatics, 12(3), 9–26. 10.15353/joci.v12i3.3275
- Woodside, J. M. (2018). Real-world rigour: An integrative learning approach for industry and higher education. Industry and Higher Education, 32(5), 285–289. 10.1177/0950422218784535
- Zawacki-Richter, O., Marín, V. I., Bond, M., & Gouverneur, F. (2019). Systematic review of research on artificial intelligence applications in higher education–where are the educators? International Journal of Educational Technology in Higher Education, 16(1), 1–27. 10.1186/s41239-019-0171-0
DOI: https://doi.org/10.55982/openpraxis.16.3.667 | Journal eISSN: 2304-070X
Language: English
Page range: 291 - 310
Submitted on: Feb 2, 2024
Accepted on: Apr 19, 2024
Published on: Aug 29, 2024
Published by: International Council for Open and Distance Education (ICDE)
In partnership with: Paradigm Publishing Services
Keywords:
© 2024 Federica Picasso, Javiera Atenas, Leo Havemann, Anna Serbati, published by International Council for Open and Distance Education (ICDE)
This work is licensed under the Creative Commons Attribution 4.0 License.