Skip to main content
Have a personal or library account? Click to login
Bias in AI (Supported) Decision Making: Old Problems, New Technologies Cover

Bias in AI (Supported) Decision Making: Old Problems, New Technologies

Open Access
|Apr 2025

References

  1. “AI’s best use case may actually be in our judicial system”, IASToppers (2023). https://www.iastoppers.com/articles/ai-s-best-use-case-may-actually-be-in-our-judicial-system [Accessed 02.03.2025]
  2. AI HLEG, Ethics Guidelines for Trustworthy AI (Europa, 2020). https://ec.europa.eu/futurium/en/ai-alliance-consultation.1.html [Accessed 02.03.2025]
  3. AI HLEG, White Paper on Artificial Intelligence: a European approach to excellence and trust (Europa, 2020) https://commission.europa.eu/publications/white-paper-artificial-intelligence-european-approach-excellence-and-trust_en [Accessed 02.03.2025]
  4. Ajunwa Ifeoma, “An auditing imperative for automated hiring systems.” Harvard Journal of Law & Technology 34 (2020).
  5. Albaroudi Elham, et al., “A Comprehensive Review of AI Techniques for Addressing Algorithmic Bias in Job Hiring” AI 5, no. 1 (2024): 383404. DOI: 10.3390/ai5010019
  6. Ali Sajid, et al., “Explainable Artificial Intelligence (XAI): What we know and what is left to attain Trustworthy Artificial Intelligence”, Information Fusion 99 (2023). DOI: 10.1016/j.inffus.2023.101805
  7. Allsop James, “Technology And The Future of The Courts”, University of Queensland Law Journal 38, no. 1 (2019): 114. https://journal.law.uq.edu.au/index.php/uqlj/article/view/1539/1293 [Accessed 02.03.2025]
  8. Amini Alexander, et al., “The impact of social segregation on human mobility in developing and industrialized regions” EPJ Data Science 3, no. 6 (2014). DOI: 10.1140/epjds31
  9. Andrus McKane, et al., “What We Can’t Measure, We Can’t Understand: Challenges to Demographic Data Procurement in the Pursuit of Fairness”, FAccT ‘21: Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency (ACM, 2021): 249260. DOI: 10.1145/3442188.3445888
  10. Angwin Julia, et al., “Machine Bias”, ProPublica (2016). https://www.propublica.org/article/machine-bias-risk-assessments-in-criminal-sentencing [Accessed 02.03.2025]
  11. Baldini Davide, Kate Francis, “AI Regulatory Sandboxes between the AI Act and the GDPR: The role of Data Protection as a Corporate Social Responsibility”, CEUR Proceedings (2024). https://cris.maastrichtuniversity.nl/en/publications/ai-regulatory-sandboxes-between-the-ai-act-and-the-gdpr-the-role- [Accessed 02.03.2025]
  12. Bannon Alicia, Douglas Keith, “Remote Court: Principles for Virtual Proceedings During the COVID-19 Pandemic and Beyond” Northwestern University Law Review 115, no. 6 (2021): 1875920.
  13. Barabas Chelsear, “Beyond Bias. Ethical AI in criminal law” in: The Oxford Handbook of Ethics of AI, eds. Markus D. Dubber et al. (2020). DOI: 10.1093/oxfordhb/9780190067397.013.47
  14. Barysė Dovile, Roee Sarel, “Algorithms in the court: does it matter which part of the judicial decision-making is automated?”, Artificial Intelligence and Law 32, no. 1 (2023): 117146. DOI: 10.1007/s10506-022-09343-6
  15. Bingham Tom, Rule of Law (Penguin Books, 2011).
  16. Brożek Bartosz, et al., “The black box problem revisited. Real and imaginary challenges for automated legal decision making”, Artificial Intelligence and Law 32 (2024): 427440. DOI: 10.1007/s10506-023-09356-9
  17. Brownsword Roger, Law 3.0 (Routledge, 2020). DOI: 10.4324/9781003053835
  18. Bruijne Kars de, Erwin van Veen. “Mind the Frame: Bias at the Data-Analysis Stage.” Pride and Prejudice: Addressing Bias in the Analysis of Political Violence (Clingendael Institute, 2017). http://www.jstor.org/stable/resrep17336.5 [Accessed 02.03.2025]
  19. Buscemi v. Italy, ECtHR Judgement 16.09.1999.
  20. Carlson Alyssa M, “The Need for Transparency in the Age of Predictive Sentencing Algorithms”, Iowa Law Review 103, no. 1 (2017): 303329.
  21. Casaleiro Paula, et al., Judicial Perceptions and Use of Technology. Portuguese Report (University of Coimbra, 2023). https://estudogeral.uc.pt/bitstream/10316/108789/1/Judicial%20perceptions%20and%20use%20of%20technology.pdf [Accessed 02.03.2025]
  22. CCJE Opinion No. 26 (2023): Moving forward: the use of assistive technology in the judiciary, (Council of Europe, 2023). https://rm.coe.int/ccje-opinion-no-26-2023-final/1680adade7 [Accessed 02.03.2025]
  23. Changqing Shi, et. al., “The Smart Court – A New Pathway to Justice in China?”, International Journal for Court Administration 12, no. 1 (2021). DOI: 10.36745/ijca.367
  24. Chatziathanasiou Konstantin, “Beware the Lure of Narratives: “Hungry Judges” Should Not Motivate the Use of “Artificial Intelligence” in Law”, German Law Journal 23, no. 4 (2022): 452464. DOI: 10.1017/glj.2022.32
  25. Chen Benjamin M., Zhiu Li, “How Will Technology Change The Face Of Chinese Justice?” Columbia Journal Of Asian Law 34, no. 1 (2021): 158.
  26. Chen Zhisheng, “Ethics and discrimination in artificial intelligence-enabled recruitment practices”, Humanities and Social Sciences Communication 10 (2023). DOI: 10.1057/s41599-023-02079-x
  27. Chomanski Bartolomiej, Lode Lauwaert, “Digital privacy and the law: the challenge of regulatory capture”, AI & Society (2024). DOI: 10.1007/s00146-024-02041-8
  28. Chouldechova Alexandra, “A fair prediction with a disparate impact: a study on bias in recidivism prediction instruments”, Big Data 5, no. 2 (2016). DOI: 10.1089/big.2016.0047
  29. Christian Brian, Alignment Problem. Machine Learning and Human Values (W.W. Norton, 2020).
  30. Cofone Ignatio, “Algorithmic Discrimination Is an Information Problem”, Hastings Law Journal 70, no. 6 (2019): 13891444.
  31. Contini Francesco, “Artificial Intelligence and the Transformation of Humans, Law and Technology Interactions in Judicial Proceedings”, Law, Technology and Humans 2, no. 1 (2020): 418. DOI: 10.5204/lthj.v2i1.1478
  32. Costabel Attilio M, “The Future of Online Justice According to Susskind: From COVID-19 Emergencies to Global Platforms”, Journal of Multidisciplinary Research 15, no. 1 (2023): 7986.
  33. COVID-19 Continuity of Court Operations During a Public Health Emergency Workgroup. Post-pandemic recommendations. https://www.americanbar.org/content/dam/aba/administrative/judicial/2021-az-post-pandemic-rec.pdf [Accessed 02.03.2025]
  34. Crofts Penny, Honni Rijswijk, Technology: New trajectories in law (Routledge, 2021): 86. DOI: 10.4324/9780429288685-5
  35. Demková Simona, “The Decisional Value of Information in Semi-Automated Decision-Making”, Review of European Administrative Law 2, no. 14 (2021): 2950 https://realaw.blog/2021/11/19/the-decisional-value-of-information-in-semi-automated-decision-making-by-simona-demkova/ [Accessed 02.03.2025]. DOI: 10.7590/187479821X16254887670874
  36. Dustin Jeffrey, Amazon scraps secret AI recruiting tool that showed bias against women. Reuters (2018). https://www.reuters.com/article/us-amazon-com-jobs-automation-insight/amazon-scraps-secret-ai-recruiting-tool-that-showed-bias-against-women-idUSKCN1MK08G [Accessed 02.03.2025]
  37. Engel Christoph, et al., “Code is law: how COMPAS affects the way the judiciary handles the risk of recidivism” Artificial Intelligence and Law (2024). DOI: 10.1007/s10506-024-09389-8
  38. Fenwick Mark, et al., “Regulation Tomorrow: What Happens When Technology Is Faster than the Law?”, American University Business Law Review 6, no. 3 (2017). https://digitalcommons.wcl.american.edu/cgi/viewcontent.cgi?article=1028&context=aublr [Accessed 02.03.2025]. DOI: 10.2139/ssrn.2834531
  39. Galán Carlos, “Certification as a control mechanism of Artificial Intelligence in Europe” (Futurium) https://ec.europa.eu/futurium/en/system/files/ged/c._galan_phd_-_ai_paper.pdf [Accessed 02.03.2025]
  40. Gautrin and Others v. France, ECtHR Judgement 25.10.1998.
  41. Gualdi Francesco, Antonio Cordella, “Artificial Intelligence and Decision-Making: the question of Accountability” Proceedings of the 54th Hawaii International Conference on System Sciences, (2021): 22972306. DOI: 10.24251/HICSS.2021.281
  42. Hanafi Imam, et al., “Legal study on ethical issues in the use of artificial intelligence for legal decisions: critical literature review”. Inspirasi Futura: Media kajian Keislaman (2024). DOI: 10.31941/pj.v23i2.4659
  43. Harris David, et al., Law of the European Convention on Human Rights (Oxford University Press, 2023). DOI: 10.1093/he/9780198862000.001.0001
  44. Hinkle Joshua C, David Weisburd, “The irony of broken windows policing: A micro-place study of the relationship between disorder, focused police crackdowns and fear of crime”, Journal of Criminal Justice 36, no. 6 (2008): 50312. DOI: 10.1016/j.jcrimjus.2008.09.010
  45. Hooker Sararh, “Moving beyond “algorithmic bias is a data problem”, Patterns 2, no. 4 (2021) https://www.sciencedirect.com/science/article/pii/S2666389921000611 [Accessed 02.03.2025]. DOI: 10.1016/j.patter.2021.100241
  46. “Independence and Impartiality of the Judiciary An overview of relevant jurisprudence of the European Court of Human Rights”, The Aire Centre (2021) https://www.rolplatform.org/wp-content/uploads/2021/09/independence-and-impartiality-of-judiciary-eng.pdf [Accessed 02.03.2025]
  47. James Gareth, et al., “Unsupervised Learning” in: An Introduction to Statistical Learning. Springer Texts in Statistics, ed. Gareth James (Springer, 2023): 503556. DOI: 10.1007/978-3-031-38747-0_12
  48. Janatian Samyar, et al., “From Text to Structure: Using Large Language Models to Support the Development of Legal Expert Systems”, in JURIX 2023: The Thirty-sixth Annual Conference, eds. Giovanni Sielno et al. (IOS, 2023): 379. https://ebooks.iospress.nl/volumearticle/65584 [Accessed 02.03.2025]. DOI: 10.3233/FAIA230962
  49. Karrar v. Belgium, ECtHR Judgement 31.08.2021.
  50. Kashif Javed, Li Jianxin, (2024) “Artificial Intelligence in Judicial Adjudication: Semantic Biasness Classification and Identification in Legal Judgement (SBCILJ)”, Heliyon 10, no. 9 (2024). DOI: 10.1016/j.heliyon.2024.e30184
  51. Kerikmäe Tanel, et al., “Frontiers in AI Judiciary: A Contribution to Legal Futurology”, Acta Baltica Historiae et Philosophiae Scientiarum 11, no. 2 (2023): 5575. DOI: 10.11590/abhps.2023.2.03
  52. Köchling Alina, Marius C. Wehner, “Discriminated by an algorithm: a systematic review of discrimination and fairness by algorithmic decision-making in the context of HR recruitment and HR development”, Business Research 13 (2020): 795848. [Accessed 02.03.2025]. DOI: 10.1007/s40685-020-00134-w
  53. Kordzadeh Nima, Maryam Ghasemaghaei, “Algorithmic bias: review, synthesis, and future research directions”, European Journal of Information Systems 31, no. 3 (2021): 388409. DOI: 10.1080/0960085X.2021.1927212
  54. Krištofík Andrej, “Európska a česká perspektíva francúzskeho zákazu profilovania sudcov” Trestněprávní revue – odborný recenzovaný časopis pro trestní parno, no. 4 (2022): 239245.
  55. Krištofík Andrej, Pavel Loutocký, “ODR and Online courts: What is their future after the AI Act” Jusletter IT. Die Zeitschrift für IT und Recht 27, no. 1 (2024): 197204. DOI: 10.38023/9375d02d-afd2-47fd-9e71-ba679710f15a
  56. Kusner Matt J, et al., “Counterfactual Fairness” in Advances in Neural Information Processing Systems, eds. I. Guyon et al. (NIPS, 2017): 406676. http://papers.nips.cc/paper/6995-counterfactual-fairness.pdf [Accessed 02.03.2025]
  57. Langford Mikael, Mikael R. Madsen, “France Criminalises Research on Judges” Verfuassunblog (2019). https://verfassungsblog.de/france-criminalises-research-on-judges/ [Accessed 02.03.2025]
  58. Larson James, “How We Analyzed the COMPAS Recidivism Algorithm”, ProPublica, (2016). https://www.propublica.org/article/how-we-analyzed-the-compas-recidivism-algorithm [Accessed 02.03.2025]
  59. Lepri Bruno, “The tyranny of data? The bright and dark sides of data-driven decision-making for social good”, in Transparent Data Mining Big and Small Data, eds. Tania Cerquitelli et al. (Springer, 2017): 324. DOI: 10.1007/978-3-319-54024-5_1
  60. Letsas George, “The ECHR as a Living Instrument: Its Meaning and its Legitimacy” in: Constituting Europe, eds. Andreas Follesdal (Cambridge University Press, 2013): 106141. DOI: 10.1017/CBO9781139169295.005
  61. Micallef v. Malta [GC], ECtHR Judgment 15.10.2009. DOI: 10.1016/S1350-4789(09)70022-4
  62. Mokander Jakob, et al., “Ethics-Based Auditing of Automated Decision-Making Systems: Nature, Scope, and Limitations”, Science and Engineering Ethics 27, no. 44 (2021). DOI: 10.1007/s11948-021-00319-4
  63. Moore Taylor R, “Trade Secrets and Algorithms as Barriers to Social Justice”, Center for Democracy and Technology (2017). https://cdt.org/wp-content/uploads/2017/08/2017-07-31-Trade-Secret-Algorithms-as-Barriers-to-Social-Justice.pdf [Accessed 02.03.2025]
  64. Morewedge Carey K, et al., “Human bias in algorithm design” Nature Human Behaviour 7 (2023): 182224. DOI: 10.1038/s41562-023-01724-4
  65. Niller Eric, “Can AI Be a Fair Judge in Court? Estonia Thinks So”, WIRED (2019). https://www.wired.com/story/can-ai-be-fair-judge-court-estonia-thinks-so/ [Accessed 02.03.2025]
  66. Pasquini v. San Marino, ECtHR Judgement 08.03.2021.
  67. Pedersen Tore, Christian Johansen, “Behavioural artificial intelligence: an agenda for systematic empirical studies of artificial inference”, AI & Society 35 (2020): 519532. DOI: 10.1007/s00146-019-00928-5
  68. Pescador Valero v. Spain, ECtHR Judgement 17.06.2003.
  69. Polčák Radim, Internet a proměny práva, (Auditorium, 2009).
  70. Reichman Amnon, Giovanni Sartor, “Algorithms and Regulation” in Constitutional Challenges in the Algorithmic Society, eds. Hans-Wolfgang Micklitz et al. (Cambridge University Press, 2021): 131181. DOI: 10.1017/9781108914857.009
  71. Reiling Dory, “Courts and Artificial Intelligence”, International Journal for Court Administration 11, no. 2 (2020). DOI: 10.36745/ijca.343
  72. Reiling Dory, Technology for Justice. How Information Technology can support Judicial Reform (Amsterdam University Press, 2009). DOI: 10.5117/9789087280710
  73. Roth Andrea L, “Trial by Machine”, Georgetown Law Journal 104, no. 5 (2016): 12451306.
  74. Rovatsos Michael, et al., “Landscape Summary: Bias in Algorithmic Decision-Making”, Centre for Data Ethics & Innovation (2019) https://assets.publishing.service.gov.uk/government/uploads/system/uploads/attachment_data/file/819055/Landscape_Summary_-_Bias_in_Algorithmic_Decision-Making.pdf [Accessed 02.03.2025].
  75. Rudin Cynthia, “Stop explaining black box machine learning models for high stakes decisions and use interpretable models instead”, Nature Machine Intelligence 1 (2019): 206215. DOI: 10.1038/s42256-019-0048-x
  76. Rudin Cynthia, et al., “The Age of Secrecy and Unfairness in Recidivism Prediction”, Harvard Data Science Review 2, no. 1 (2020). DOI: 10.1162/99608f92.6ed64b30
  77. Rustavi 2 Broadcasting Company Ltd and Others v. Georgia, ECtHR Judgement 9.12.2019.
  78. Sances Michael, Hye Y. You, “Who Pays for Government? Descriptive Representation and Exploitative Revenue Sources”, Journal of Politics 79, no. 3 (2017): 10904. DOI: 10.1086/691354
  79. Schwartz Reva, et al., “Towards a Standard for Identifying and Managing Bias in Artificial Intelligence” (NIST, 2022). https://nvlpubs.nist.gov/nistpubs/SpecialPublications/NIST.SP.1270.pdf [Accessed 02.03.2025]. DOI: 10.6028/NIST.SP.1270
  80. Shi Jiahui, “Artificial Intelligence, Algorithms and Sentencing in Chinese Criminal Justice: Problems and Solutions” Criminal Law Forum 33 (2022):121148. DOI: 10.1007/s10609-022-09437-5
  81. Sobowale Julia, “How Artificial Intelligence is changing the legal profession”, ABA Journal (2016) https://www.abajournal.com/magazine/article/how_artificial_intelligence_is_transforming_the_legal_profession [Accessed 02.03.2025]
  82. Soukupová Jana, “AI-based Legal Technology: A Critical Assessment of the Current Use of Artificial Intelligence in Legal Practice”, Masaryk University Journal of Law and Technology 15, no. 2 (2021): 279300. DOI: 10.5817/MUJLT2021-2-6
  83. “State v. Loomis”, Harvard Law Review 130, no. 1530 (2016). https://harvardlawreview.org/print/vol-130/state-v-loomis/ [Accessed 02.03.2025]
  84. Straton Papagianneas, “Automating Intervention in Chinese Justice: Smart Courts and Supervision Reform” Asian Journal of Law and Society 10, no. 3 (2023): 46389. DOI: 10.1017/als.2023.5
  85. Straton Papagianneas, Nino Junius, “Fairness and justice through automation in China’s smart courts”, Computer Law & Security Review 51, no. 1 (2023). https://www.sciencedirect.com/science/article/pii/S0267364923001073 [Accessed 02.03.2025]. DOI: 10.1016/j.clsr.2023.105897
  86. Sunstein Cass, “Algorithms, Correcting Biases”, Social Research: An International Quarterly 86, no. 2 (2019): 499511. DOI: 10.1353/sor.2019.0024
  87. Susskind Richard, Online courts and future of justice (Oxford University Press, 2012), 601.
  88. Tata Cyrus, “Introduction: Why Study Judicial Decision Support Systems?” International Review Of Law Computers & Technology 14, no. 3 (2000): 27982. DOI: 10.1080/713673365
  89. Tian Niu W, Michael Yuan, “Intelligent Justice’: AI Implementations in China’s Legal Systems” in Artificial Intelligence and its discontents, ed. Ariane Haneemayer (Springer, 2022): 197222. DOI: 10.1007/978-3-030-88615-8_10
  90. Vardanyan and Nanushyan v. Armenia, ECtHR Judgement 25.10.2016.
  91. Wang Liyuan, et al., “A Comprehensive Survey of Continual Learning: Theory, Method and Application” IEEE Transactions on Pattern Analysis and Machine Intelligence 46 (2024). DOI: 10.1109/TPAMI.2024.3367329
  92. Weinberger Ota, Law, Institutions and Legal Politics (Springer, 1991). DOI: 10.1007/978-94-011-3458-3
  93. Xiang Alice, “Mirror, Mirror, on the Wall, Who’s the Fairest of Them All?”, Daedalus 153, no. 1 (2024): 250267. DOI: 10.1162/daed_a_02058
  94. Xie Hanbo, “The promising future of cognitive science and artificial intelligence”, Nature Reviews Psychology 202, no. 2 (2023). DOI: 10.1038/s44159-023-00170-3
  95. Yeung Karen, “Why Worry about decision-making by machine?”, in Algorithmic regulation, ed. Karen Yeung (Wiley, 2017): 3233.
  96. Zeleznikow John, “Negotiation, Online Dispute Resolution, and Artificial Intelligence”, in Handbook of Group Decision and Negotiation, Negotiation, Online Dispute Resolution, and Artificial Intelligence, eds. Marc Kilgour, Colin Eden (2021):11251147. DOI: 10.1007/978-3-030-49629-6_38
DOI: https://doi.org/10.36745/ijca.598 | Journal eISSN: 2156-7964
Language: English
Page range: 3 - 3
Published on: Apr 28, 2025
Published by: International Association for Court Administration
In partnership with: Paradigm Publishing Services

© 2025 Andrej Krištofík, published by International Association for Court Administration
This work is licensed under the Creative Commons Attribution 4.0 License.