1. Introduction
Artificial intelligence has been promising to transform the legal field and administration of justice for more than thirty years.1 The ambition to have robotic judges capable of deciding autonomously2 and predicting (or suggesting) what decision will be made considering past judgments has been given enormous prominence in academia and business.3 The promise of quick judgments based on jurisprudence and statistical calculations offered a new approach to improve timeliness, efficiency and legal certainty. The risks of discrimination and undue influence on judicial decisions, evident from the initial implementations of judicial decision support systems, did not deter developers committed to advancing predictive justice and legal analytics systems, primarily based on machine learning.4
Until at least 2021, enthusiasm for substituting human decision-makers with machines or providing judges with more or less binding suggestions overshadowed critical perspectives. Numerous projects have proliferated across Europe; however, the recent assessment from the Consultative Council of European Judges shows just modest results.5 Focusing on Europe, the subject of this discussion, difficulties can be mainly explained by two significant barriers: the costs for data cleaning and annotation and the availability of jurisprudence databases. As a consequence, AI and legal-tech systems are more advanced in France, where legislation requires the publication of judgments in electronic formats.6 This development is also pronounced in common law jurisdictions, where the significance of case law is paramount.
The arrival of GenAI in November 2022 completely changed the scenario. While predictive justice was still stuck between the design and pilot stages, lawyers, magistrates and citizens suddenly discovered systems (chatbots or digital assistants like ChatGPT or Bard) that answer legal questions and can analyze and synthesize contracts, judgments and petitions. According to many, this technological revolution will have an impact on organizations and professions similar, if not more significant, than the transition from typewriters to microprocessors and personal computers.7 This new scenario makes it pressing to examine the preliminary impacts of integrating GenAI within legal professions and the administration of justice. The topic is complex and can be analyzed on different levels. The first level deals with the changes this type of technology is already causing to the activity systems of judges, lawyers and citizens facing legal issues. The focus is on how GenAI performs activities previously done by humans, what the risks are, how to reduce them, and, more generally, the implications of the new activity systems for a fair trial. The second level includes the implications of these dynamics on the organization of the judiciary and the legal profession, as well as their governance.8 Relevant questions concern the effects and challenges for self-governing bodies. As will be seen, GenAI is pervasive and calls for steering and control at multiple levels; hence, it can pose issues of independence. The third and no less critical level is the economic one. Indeed, this technological revolution is inextricably associated with the platforms’ economy and their systems of licensing and service rents, also analyzed as digital feudalism9 or techno-feudalism.10 In the legal sector, this technological change will likely reduce staff costs while increasing licensing expenditures. Hence, resources will shift from local recipients (secretaries, legal advisors, and paralegals) to legal techs and, ultimately, the owners of large language models (LLMs). This article considers the first two levels that capture GenAI’s current impacts on justice administration. It provides a basic understanding of technologies and how they are changing law practice, and it explores to what extent they can be governed. This understanding is necessary for analyzing the broader economic implications for the justice sector that are expected in a later stage and, hence, are not worth discussing in this paper.
With these limitations, section 2 introduces the essential technological features and their fast evolution, while section 3 focuses on examples of GenAI use in the legal and judicial spheres. This approach aims to contextualize applications and identify early impacts on judges’ and lawyers’ activity systems. Section 4 considers the first attempts at self-regulation or regulation through soft law and guidelines. The analysis identifies both areas of substantial agreement between guidelines issued by different institutions and two fundamentally divergent approaches. One approach emphasizes the role of certification (or authorization) of the system and the transparency in its use. The other approach focuses on lawyers’ and judges’ ethical and professional responsibilities, which are reaffirmed as essential for using GenAI properly. Assessing the two approaches, the concluding section posits that, in light of the weaknesses of certification mechanisms, GenAI can only positively influence the values of a fair trial if the user can accurately verify the contents offered by the systems. This verification must ensure their factual accuracy, the absence of hallucinations (incorrect or misleading results generated by AI, such as references to facts, as sentences, that do not exist) and the avoidance of other cognitive distortions.
2. Large Language Models and GenAI
In late 2022, OpenAI brought ChatGPT 3.5 online; in spring 2023, Microsoft and Google launched applications with similar functionalities, and other big players took positions in the field. Subsequently, the discourse on LLMs and GenAI has become pervasive, a new hype in the techno-legal field.11
LLMs are AI algorithms that, through probabilistic calculations, learn to predict the next word in a sentence. Chatbots with GenAI capabilities generally use LLMs to generate responses to ‘prompts’, i.e. natural language instructions given to the system to obtain an output.12 Since LLMs are pre-trained on extensive datasets, GenAI can generate responses to a wide array of requests or commands. To encapsulate this versatility, the European Union AI Act classifies these systems as general-purpose artificial intelligence (AI) or foundation models.13 In their basic version, GenAI chatbots such as ChatGPT, Gemini or Copilot are available online for free and ready to respond to any request made by judges, lawyers or citizens. They can create a petition, assess a case or suggest the outcome of a proceeding, as some judges have already experienced.14 Their output can seem plausible, but their quality can only be established by those with precise knowledge of the subject. Sometimes, answers pass off situations that never existed as accurate, generating hallucinated projections of reality as judgments that do not exist.15 Furthermore, chatbots like ChatGPT pose severe privacy and data protection concerns since all data exchanged through these applications – at least in their basic license – are captured by the system and become new training data.16 Moreover, as these systems are trained on non-contextualized or irrelevant data, they can give wrong answers, as when English legal professionals find answers based on US law.17
The expectations of GenAI transformative effects on the legal sector are widespread.18 However, the mentioned issues limit GenAI use (in their basic license) to previously anonymized documents or ancillary tasks that do not deal with sensitive data. Therefore, it would be desirable to have GenAI applications capable of referring to specific data sets (e.g. English rather than American law and case law) to reduce (or null) the risk of hallucination to protect the confidentiality of the data uploaded and to work with contextualized data. Technological evolutions and new contractual arrangements have addressed these issues and, to some extent, unlocked the usage of GenAI in the legal sector.
On the technological side, since 2023, the companies running LLM and GenAI developed tools, such as APIs (Application Programming interfaces) and GPTs (Generative Pretrained Transformers)19 enabling the customization of GenAI systems for the requirements of specific domains, functions and jobs.20 According to systems providers, this step has created the technological preconditions for developing systems that respond to the needs of specific domains, avoid the risk of non-pertinent answers and reduce the risk of hallucinated answers.21 Not less essential, using APIs and GPTs is considered simple, to the point of not requiring programming knowledge (no code or low code), at least in some cases.22 Technological development has been accompanied on the contractual side by new licensing models that ensure confidentiality and exclude the data uploaded from training, removing other obstacles to using domain systems in the legal and judicial spheres.
Technological and contractual changes have empowered the development of ‘domain’ or specialized systems designed to carry out a sector’s typical tasks, such as drafting a contract or summarizing a petition. Such domain systems are trained on specific and pertinent data sets containing the relevant knowledge to perform the required tasks. Training can be carried out by associating examples in which a correct answer is associated with a request. The procedure can result in ‘semantic injections’ of domain-specific knowledge into the domain model. The operation contributes to decoupling the underlying LLM model from the domain-specific knowledge relevant to effectively executing the prompts.23 Prompt engineering also plays a role in improving and making the task description performed by a GenAI model more precise and accurate. It can involve framing a question, establishing a style, providing relevant context or assigning a role to the chatbot to make prompts easier to interpret and process by the GenAI model. Hence, a better definition of the input makes it possible to obtain more coherent answers that are responsive to the needs of the task at hand. At the end of the process, the correctness and relevance of the answers increase.24
From a system development perspective, the build-up of domain systems is relatively simple, incomparably faster, and less costly than previously developed AI systems.25 Training is primarily done and incorporated into the general-purpose LLM. Hence, the development of domain systems is mainly a matter of defining the scope of use, ensuring the availability of reference data, and identifying the tasks to be performed and the questions the system will be called upon to answer. Finally, the contractual rules mentioned above promise confidentiality and non-use of the uploaded data outside the domain application, thus making these systems better aligned with the requirements of the legal and judicial sectors. GenAI is unlocked, at least from the technological, contractual and data protection perspectives.
In addition to domain systems, a second point of GenAI deployment into the legal domain — as in any other professional sector — is its integration into the office applications provided by companies like Google or Microsoft. Digital assistants, embedded into word processors or spreadsheets, are already part of the primary digital working environment of lawyers, judges and pro se litigants.26 The two innovation streams make GenAI available to all legal professionals and promise to transform legal and judicial activities radically.
3. GenAI for the legal professions
The features of LLMs and GenAI, and the new technological and contractual developments, explain the rise and growth of domain-specific applications and the attention of all those working in the legal sector. A study by Goldman Sachs predicts that almost half of all legal advice could be done by AI systems;27 surveys addressed to lawyers and law firms in recent months confirm the growing relevance of these technologies, at least in the Anglo-Saxon world.28 Applications mainly target law firms, but judiciaries are also developing systems to ease access to justice or the work of magistrates and court clerks, as discussed below. Another expanding area not addressed in this discussion involves the development of GenAI systems designed to facilitate online mediation and arbitration.29 In summary, every facet of judicial and legal operations is currently influenced, or will soon be impacted, by GenAI. This section delves into these applications, starting with a focus on digital assistants for law firms, then moving on to systems designed to enhance access to justice, and finally, exploring the initial forays into developing GenAI systems for judges and clerks.
3.1. Digital assistants for lawyers
An exploratory review carried out without any claim to exhaustiveness at the beginning of March 2024 to identify the functionality offered shows how domain systems for law firms are rapidly emerging in civil and common law countries.30 According to the developers, these systems promise efficient data and document analysis, information extraction and text generation customized for the legal business. The functionalities include summarizing legal documents by capturing their salient points, reviewing contracts by identifying critical clauses that need special attention and suggesting improvements to legal documents. They claim to be able to generate content from scratch, for example, drafting contracts based on pre-existing templates31 or carrying out searches with natural language prompts and then drafting legal opinions.32 Some systems are ‘generalist’ and thus designed to work on any law or practice area, while others specialize in specific fields.
Concretely, these systems can be used for many different purposes and needs. For instance, the system can be asked to identify jurisprudence relevant to a case. Then the lawyer can prompt it to analyze the case in the light of that same jurisprudence. After checking the output, the user can request suggestions to support the case and then draft a claim based on the arguments already identified. Ideally, the lawyer should be able to focus on the most critical and essential activities, leaving the preliminary tasks such as research, analysis and drafting to the system.
Regarding technological features, domain systems are typically designed following a retrieval-augmented generation (RAG) model; an approach that optimizes the output of LLMs by referencing an authoritative knowledge base outside their training data sources. RAG extends the capabilities of an LLM to specific domains (such as the legal) or an organization’s internal knowledge base without retraining the model.33 It is emerging as a cost-effective approach to improving the relevance and accuracy of LLM outputs in given professional contexts.34 Hence, domain systems work on the dataset made available by the provider or the same users. In this way, they can benefit from broad legal databases, such as collections of judgments and court decisions supplemented by doctrine and statistical analysis made available by the provider. They can also work just on the case file or the entire legal firm’s knowledge base (for instance, contracts, statutes, memoranda, deeds, and e-mails), extending the search and text generation to the group of documents relevant to executing the task. Therefore, the extensiveness of the database made available to customers is one of the suppliers’ competitive assets; on the same token, the extensiveness of the law firm knowledge base becomes a competitive advantage for law firms. Large law firms and leading legal database providers have relevant competitive advantages.
The availability of jurisprudential databases becomes crucial for the growth of the emerging legal tech business sector. In this context, the accessibility and reusability of judicial decisions are not just a matter of access to justice since they are closely linked to GenAI’s potential for an open and competitive market. Hence, policy decisions are necessary to determine the extent and contractual arrangements for making judgments available or not and clarify the distinction between free and paid content and the level of anonymization required.
According to software developers, domain-specific GenAI systems exhibit a lower tendency toward generating hallucinations.35 This reduced propensity may be attributed to the systems operating solely on selected datasets or their ability to refer back to the original documents used in formulating responses.36 While this development addresses a significant hurdle in adopting such systems within the legal domain, it is essential to conduct rigorous testing and empirical research to validate the claims made by the providers and ensure the systems’ reliability.
A first assessment of this kind, just conducted at Stanford University, shows that the claims made by leading companies such as Casetext, Thomson Reuters, and LexisNexis regarding their systems’ ability to eliminate or avoid hallucinations are not entirely accurate.37 The study found that responses from these systems contained hallucinations at rates ranging from 17% to 33%.
The issue is at least partially addressed but definitely not solved, and human checks are still required.
Given these emerging technological features, the proliferation of such applications is anticipated to persist, and GenAI is expected to become increasingly integrated into commercial and public legal databases and case management systems.38
3.2. GenAI and access to justice
GenAI is not just a matter of improving data and document management for law firms, as discussed above. It is also associated with the idea that the delivery of legal services and judicial activity will be redesigned, making justice more accessible and less costly. Indeed, the ability to extract content from specific data sets and document collections also eases access to justice. Improvements can occur through a lawyer’s fee reduction or systems that can enable and support the assessment of legal issues without lawyers’ involvement.39 Finally, Ministries and other agencies can design systems to deliver legal information to citizens and users of judicial services.
Better access to legal information is precisely the goal pursued by the Portuguese Ministry of Justice with its Practical Guide to Access to Justice (GPJ).40 The guide is a virtual assistant that provides quick and simplified questions and answers to help citizens access legal information. This chatbot aims to simplify access to relevant information, speed up interactions with the administration of justice and enable users to act in predetermined situations. The system, at the beta stage, deals with issues related to divorce and the registration of companies online. The assistant is based on the GPT 4.0 language model, available on the Microsoft Azure OpenAI platform, and does not collect personal data. All answers are based exclusively on content provided by the Ministry’s Digital Justice platform. The answers are thought to be simple and concise and are often associated with a link that allows users to go further or start the procedure. The Ministry plans to extend it to alternative dispute resolution and online criminal record certificates.41
From an initial test carried out in mid-March 2024, the system gave consistent answers to simple questions, such as asking for advice to obtain a divorce between Portuguese citizens. When the legal situation became more complex, such as obtaining a divorce between a Portuguese citizen and a non-EU citizen, the GPJ first admitted that it did not know how to answer and then gave misleading answers. After a few weeks, the same test provided more convincing answers, directing the user toward an apparently sound explanation.42 Therefore, the system learns, with an apparent discontinuity from the classic FAQ, which can only couple predetermined questions and answers. However, while the classic FAQ are fixed and established by a subject with the necessary knowledge, the GPJ can answer any question and – as seen – can either provide wrong answers or change answers over time. The risk of changing answers due to the algorithm’s learning capacity is a phenomenon that needs to be controlled. The Portuguese example, in any case, is just a first experiment and fully demonstrates the potential that GenAI can have to make legal information more accessible for citizens and justice seekers.
3.3. GenAI for judges and court clerks
The features and functionalities designed for lawyers and law firms make these systems attractive for the work performed by judges and court clerks. For example, the Spanish Ministry of Justice has built a system for summarizing and classifying procedural acts to simplify the work of judges and clerks.43 The Public Jurisprudential Data Bank (BDP) of the Italian Ministry of Justice will provide abstracts of judgments created by an unsupervised artificial intelligence system, supposedly GenAI.44 However, the system has yet to go into operation, and the few measures with which an abstract is associated have been realized by university law graduates thanks to a project conducted by the Court of Appeal of Florence.45
It is difficult to estimate how much these systems, whether freely accessible or paid for, are used by judges, judicial assistants, court clerks and lawyers behind closed doors. The evidence is anecdotal and constantly growing. Regarding use by judges, the first known case in February 2023 refers to a Colombian judge who asked a GenAI system to help him decide a case concerning an autistic child and his medical insurance. The dialogue (questions and answers) between the judge and the chatbot was reported in the ruling and – of course – sparked the debate.46 The following month, an Indian judge asked ChatGPT for advice on granting bail to a murder suspect. 47 In the same days, a Pakistani judge made ‘experimental’ use of ChatGPT-4 to decide a sexual assault case. The judge asked for a legal definition of “consent” and included the answer in the ruling.48 In September 2023, speaking at a conference, an English Court of Appeal judge admitted that he had used ChatGPT to summarize an area of law in which he was an expert. He received an answer that he found acceptable and was then included in the judgment.49
These examples testify that GenAI is already utilized in judicial chambers, demonstrating its multipurpose potential. As far as lawyers are concerned, there are several well-known petitions with references to non-existing court decisions, hence hallucinations, starting with the Meta vs. Avianca case of May 2023 in the United States.50 Other cases have been reported in Canada (February 2024) and previously in the UK (December 2023) by an unrepresented litigant.51 This anecdotal evidence only points out what can be considered the tip of an iceberg. Having GenAI embedded in word processors (see section 2) will make GenAI drafted deeds as ordinary as using the word processor itself. As observed, ‘the genie is already out of the bottle’, and we must understand how to manage the consequences.52 To pursue this goal, the following section analyzes guidelines issued to regulate the use of this new technology in the justice sector.
4. Guidelines for GenAI in the legal field
The proliferation of GenAI systems in the legal and judicial world and the hallucinations introduced, perhaps unwittingly, into court proceedings triggered the proliferation of guidelines for the responsible use of these tools.53
At the beginning of June 2023, the European Bars Association (Fédération des Barreaux d’Europe, FBE) published the guideline ‘European lawyers in the era of ChatGPT.’54 On 12 December 2023, the Courts and Tribunals Judiciary, which performs the self-governing functions of the judiciary in England and Wales, issued ‘Artificial Intelligence (AI) Guidance for Judicial Office Holders’.55 Later on, in February 2024, the Council of Europe’s Commission for the Efficiency of Justice (CEPEJ) launched the ‘Use of Generative Artificial Intelligence (AI) by judicial professionals in a work-related context.’56 Also, the legal tech industry is developing guidelines, as in the case of Predictice. This French company proposed the ‘Charter for the responsible use of generative Artificial Intelligence’ (Charte pour une utilisation responsable de l’intelligence artificielle générative), co-signed with 10 law firms.57
The guidelines discussed here do not claim to be exhaustive, which is difficult to pursue in a fast-changing domain. Since they originate from bodies with varying institutional missions and mandates, some are directed exclusively at lawyers, while others address lawyers and the judiciary. The subsequent sections will first illustrate the points of convergence before focusing on their differences.
4.1. Understanding GenAI and LLMs
The consensus among various guidelines underscores the imperative of comprehending the attributes, constraints, and operational dynamics of GenAI systems, understanding both the functional technological features and the contractual stipulations prescribed by service providers. This understanding is a prerequisite to ensure the conformity of use with deontological and legal standards.
The guidelines illustrate some features and limitations of the systems. The English one clarifies that LLMs ‘generate new text using an algorithm based on the prompts they receive and the data they have been trained upon’ and that the ‘quality of any answers you receive will depend on how you engage with the relevant AI tool, including the nature of the prompts you enter. Even with the best prompts, the information may be inaccurate, incomplete, misleading, or biased.’58
The guidelines then emphasize that GenAI is only a complementary tool of personal expertise and cannot replace legal professionals’ judgment and case management experience.
4.2. Confidentiality
The guidelines unanimously emphasize the need to respect the confidentiality of the data processed, thus guaranteeing professional secrecy, data protection and privacy. Underlining the point, Predictice clarifies how providers must ensure that data uploaded ‘are not used to train algorithms.’59 This must be understood in terms of competition between law firms. Indeed, using a given law firm’s data for training will soon improve the functioning of the applications, with subsequent advantages for other law firms. In passing, this clause is expressly stipulated by paid suppliers of commercial GenAI systems for law firms.
The utmost caution about confidentiality is to be found in the CEPEJ document, which suggests to ‘Enter only non-sensitive data and information which is already available in the public domain’60 and conversely, not to use the system ‘in [the] case [that] you would be required to enter and thus disclose personal, confidential, copyright-protected or otherwise sensitive data.’61 This last warning makes the currently available GenAI systems unusable for activities relating to the analysis and management of concrete cases. It remains to be understood whether this position is taken in the light of a lack of confidence in the technological and contractual commitments of service providers (and their services that already ensure confidentiality and non-use of data for algorithm training purposes) or of a substantial lack of knowledge of the technical and commercial developments that already guarantee the necessary confidentiality. Similarly, the English guidelines suggest not to ‘enter any information into a public AI chatbot that is not already in the public domain. Do not enter information that is private or confidential. Any information that you input into a public AI chatbot should be seen as being published to all the world.’62 Furthermore, on the one hand, ‘paid subscriptions have been identified as generally more secure than non-paid’ but, on the other, ‘beware that there are a number of 3rd party companies that license AI platforms from others and are not as reliable in how they may use your information. These are best avoided.’63 It needs to be clarified whether this is a closure to the legal techs providing GenAI applications emerging in the second half of 2023. The combination of partly conflicting indications can be encapsulated in this statement: ‘In future, AI tools designed for use in the courts and tribunals may become available, but, until that happens, you should treat all AI tools as being capable of making public anything entered into them.’64 Diverging from the CEPEJ framework, the English guidelines recognize contemporary contractual and technological advancements. Nonetheless, they ultimately endorse a precautionary principle that limits the use of GenAI.
These guidelines, and technological regulations in general, struggle to keep up with the speed of technological development.65 Consequently, it is necessary to reflect on whether a specific regulation (which qualifies the functions, operations and features of the system) is necessary, or whether the extraordinary development speed of these technologies makes it an essentially useless effort. In this case, anchoring GenAI use on ethical and deontological criteria becomes an attractive option.
4.3. Output control
All guidelines clarify that GenAI ‘should complement your legal competencies rather than replace them’ and that it should be utilized ‘to streamline processes in your law firm and improve the efficiency of legal services.’66 Consequently, GenAI should not replace the lawyer’s professional judgment, critical ability and competence. The Predictice document emphasizes that lawyers must ‘critically analyse the results and recommendations generated by the AI, taking into account the specifics of each case and contextual elements that may not be taken into account by the algorithm.’67 Moreover, before submission to a judge, it is imperative to verify the reliability and accuracy of the information provided by GenAI.68
Similarly, the English guidelines specify, ‘The accuracy of any information you have been provided by an AI tool must be checked before it is used or relied upon.’69 The CEPEJ prescribes to ‘check the correctness of the answers, and if regulatory references are provided, verify their existence.’70 Furthermore, it invites the users to ‘remain in control of [their]choice and the decision-making process and take a critical look at the made proposal.’71 Finally, the system should not be used ‘in case you cannot assess the result for factual correctness and bias.’72 Similarly, the English guidelines recap that Public AI chatbots are inadequate for finding information that cannot be verified and ‘do not provide answers from authoritative databases.’73
Lawyers, judges and users are responsible for overseeing the GenAI output. As such, the level of expertise of the overseer directly influences the simplicity and efficiency of the quality assessment of the output: the more experienced the overseer is, the more straightforward and economical the assessment will be; conversely, less experienced overseers may encounter greater complexity. Whether dealing with complex contractual or procedural documentation or a more straightforward task such as summarizing a deed, the evaluation effectiveness relies on the evaluator’s familiarity with the contents analyzed by the system. Consequently, the efficacy and efficiency of these evaluations correlate directly with the specific skills of those conducting them. In certain instances, such as those involving a self-represented litigant, the oversight of the output may be unattainable or cursory. Paradoxically, the greater the reliance on the tool by users, the less capable they may be of controlling the outcomes.
4.4. Disclosure or responsibility?
While the guidelines essentially agree on the need to understand the tools’ features, ensure the confidentiality of data and privileged information and verify the quality of the output, their first fundamental difference concerns the disclosure of GenAI usage. Depending on the parties involved, this may concern the relationships between lawyer and client, lawyer and judge and between the judge and parties to the trial.
The FBE claims that the lawyer has an obligation of transparency and must ‘assume responsibility for the outcomes and consequences of utilising GenAI while being mindful of your professional obligations and potential liability’. More specifically, it suggests to ‘maintain transparent communication with your clients regarding the use of GenAI in your legal practice,’ specifying purpose, benefits, limitations and safeguards, and ensuring clients understand their role in their case.74 The lawyers’ professional obligations include the responsibility for the outcomes and consequences of GenAI. Predictice agrees that ‘lawyers should be transparent with their clients about the use of GenAI and its impact on the pricing of their legal services.’75
CEPEJ clarifies: ‘Be transparent and always indicate if an analysis or content was generated by generative AI.’76 This last warning seems to apply to any relationship between judges, lawyers and clients.
The English guidelines take a different approach: ‘All legal representatives are responsible for the material they put before the court/tribunal and have a professional obligation to ensure it is accurate and appropriate. Provided AI is used responsibly, there is no reason why a legal representative ought to refer to its use, but this is dependent upon context.’77 This understanding expressly applies to the relationship between lawyers and judges, while it needs to be clarified whether it also applies between lawyers and clients. Therefore, in the English case, the transparency obligation stated by CEPEJ, FBE and Predictice is replaced by responsibility. Accordingly, GenAI must be used appropriately (for the functions suggested), and its outputs must be controlled to submit accurate and appropriate procedural documents, thereby reaffirming the longstanding duty of any lawyer.78 The lawyer will then be responsible towards third parties. It can be noted, however, that the difference between the English and the other approaches does not remark a separation between the worlds of civil and common law. Indeed, American judges have repeatedly issued standing orders mandating lawyers to disclose their use of GenAI.79
4.5. Authorization or self-regulation?
The CEPEJ guideline specifies that the judges or lawyers must ‘make sure that the tool’s use is authorized and appropriate for the desired purpose.’80 While appropriateness can be assessed considering the inputs provided by the same guidelines, in Europe, except for the English case being discussed, no rule authorizes using GenAI, and the authorization required by CEPEJ makes GenAI unusable. Later, the same document takes a softer stand and observes that the tool should not be used if prohibited by the law or the organization. Discussing the notable difference between the two statements is beyond this paper’s scope; what matters is exploring if it is reasonable or necessary to authorize the use of these tools. Indeed, the English guidelines look at GenAI from a different perspective: ‘Judicial office holders are personally responsible for material which is produced in their name. Judges are not generally obliged to describe the research or preparatory work which may have been done in order to produce a judgment. Provided these guidelines are appropriately followed, there is no reason why GenAI could not be a potentially useful secondary tool.’81
The substantial disparity between the two approaches highlights a distinct attitude towards judicial professionals. The English guideline demonstrates a reliance on the professionalism of the English judiciary, while CEPEJ emphasizes a bureaucratic perception of the judicial profession, wherein judges are constrained to utilize only tools endorsed by an overarching authority. Having first considered the functions provided by GenAI systems designed for the legal sector and, subsequently, the guidelines for the proper use of these technologies, the next concluding step involves reassembling the overall picture, starting with an examination of the extent to which the different regulatory approaches align with recent technological developments.
5. Conclusion: GenAI between users’ control and certification
GenAI is rapidly spreading in the administration of justice through free chatbots made available by various service providers or domain applications designed for analyzing, drafting and summarizing legal documents. Systems are in place to facilitate access to justice and the analysis and synthesis of judicial files by judges and clerks. Finally, chatbots are now embedded in word-processing systems, browsers and many other applications. GenAI, with its multipurpose functionalities, is already ubiquitous. With such a seemingly unstoppable penetration, introducing guidelines was an inevitable first attempt to contain and direct the use of GenAI in the legal sector. This answer passed through the identification of the functions that can be performed by the applications and of conditions for use to be respected for its proper use.
Also, there is a gap between the GenAI functions considered acceptable by the guidelines, limited to summarizing deeds and judgments and drafting administrative documents, and those provided by the domain systems adopted by leading law firms, which include legal research and the drafting and analyzing of contracts and legal documents, not just administrative ones. This gap underscores the need for further analysis to understand how this broader set of functionalities affects legal and judicial activities.
Looking at the conditions to be respected, all the guidelines agree about the need to understand the functioning of GenAI, identify its limits and develop an awareness of the risks associated with its use. There is also agreement on the importance of maintaining the confidentiality of the data collected by documents and proceedings. Some doubts emerge about the actual ability of domain systems and paid GenAI licenses to guarantee confidentiality even when assured by contractual provisions.82
The guidelines, however, differ concerning the obligation to inform judges, clients, or procedural parties about the use of the tool, as well as the necessity to obtain pre-authorization for its use or to certify its features. CEPEJ, Predictice, and FBE, albeit with some differences, require authorization, certification, and the duty to information. The legitimation of GenAI use arises from the integration of the authority that sanctions the use of technology and transparency, which must consistently be paired with the conscientious application of technology and the safeguarding of confidentiality. The approach is at least partly shared by judges in common law countries who, through standing orders, have required parties and lawyers to declare the use of GenAI.
The English approach is different: the obligation to provide information is supplanted by lawyers’ or judges’ responsibility and ethical duties. If a document submitted to the court or issued by the judge contains incorrect information, the issue lies not with the GenAI system but with the user, whether a lawyer or a judge, who failed to conduct the necessary verifications. The effectiveness of human control over the texts suggested by the machine becomes the pillar of making the system usable in the legal business. Identifying tasks in which the system can be used (e.g., summarizing documents and drafting administrative texts) and controlling the output excludes the need for certifications and authorizations. This approach is not new since it has already been adopted with applications such as voice recognition, commonly used in judicial offices to draft procedural documents.83 It is not imperative to elucidate the system’s inner workings; rather, it is essential to verify the coherence of the output with the given inputs. This approach is feasible, yet the user, rather than the technology provider or any overarching authorizing authority, retains full accountability for the use of the technology’s output.
If the Anglo-Saxon approach places all the safeguards in the user’s ability to control the systems’ output, the advantage is its simplicity and cost-effectiveness. Controls, authorizations, and certifications are inherently complex, entail significant costs, and possess only temporary effectiveness in autonomously evolving systems.84 Finally, it is still being determined whether the authorization and certifications discussed here are added to those already provided for general-purpose systems provided by the EU AI Act, or must also be carried out on domain systems. The road to GenAI regulation still needs to be paved, and users remain accountable for its usage. A different approach would be benchmarking GenAI systems’ performances, as recently done by Magesh and colleagues in their assessment of the hallucination, responsiveness, and accuracy of leading GenAI systems for legal professionals.85
The last point to be discussed is the disclosure of the use of GenAI. Without making ethical or deontological points, what matters here is the significance of such a statement in working environments in which GenAI becomes ubiquitous and growingly embedded into the working environment of legal professionals, starting with word processors and office applications introduced in Section 2. When everything is drafted using, to some extent, GenAI, the meaningfulness of the disclosure will be lost.
As usual, technology deployment is advancing much faster than regulation. This understanding, however, should not be interpreted as a green light for the indiscriminate use of GenAI. Instead, it highlights the difficulties in regulating or containing GenAI deployment in the legal sector and, more broadly, in society. As Antoinette Rouvroy noted, ‘adopt AI, think later’ has become commonplace, with technologies introduced without assessing implications at the different levels.86 The effects can be undesirable even in the case of functions identified by the guidelines as appropriate for the legal industry. Each user, remaining accountable for technological errors, can become the victim of machine mishaps and a perpetrator against third parties.
As a final step, the impact assessment of GenAI on the values that define the administration of justice must be carried out.87 This exercise is complex due to the novelty of the technology, the lack of specific studies and the plurality of functions potentially performed.
Domain systems offer to synthesize and draft documents and conduct legal searches and analyses, which can improve the operational efficiency of lawyers and judges. However, the effect will be positive only if associated with careful control of the system’s outputs. If, for any reason, this does not happen, it would end up injecting uncertainty and confusion with adverse effects on the principles of due process, as when lawyers have cited non-existent jurisprudence.88 There is a trade-off between output control and efficiency: the more competent the system’s user is (for example, the legal matter analyzed or the act summarized is well known), the quicker and more effective the control will be. Conversely, a user who ignores the legal matter of the document summarized by GenAI must spend relevant time evaluating the output, thus reducing the efficiency gain promised by the system. Otherwise, the user will have to assume the risk of not carrying out any checks, which is not negligible in work pressure conditions. Lastly, GenAI promises more than efficiency gains. The Portuguese case shows its impact on access to justice. However, in this case, regular checks are also needed to prevent the machine from changing answers to the same question through learning processes. Then, GenAI can have positive implications for due process if there is consistent, expert control and supervision of the output quality. Conversely, when the output of GenAI is not adequately controlled due to haste, incompetence, or other reasons, it can become a conduit for incomplete, non-factual and biased analysis and information.
However, controls cannot be taken for granted. In many of the discussed use cases, there exists a risk of principal-agent problems,89 information asymmetry between technology users and clients and weak accountability mechanisms,90 all of which can proliferate moral hazard.91 These factors can undermine trust in the professions and the administration of justice. The risk is significant, and the remedy, though modest, is straightforward: clarifying ethical and deontological duties, empowering humans and requiring critical thinking against the quick fixes provided by machines.
Notes
[1] Guido Governatori et al., “Thirty years of Artificial Intelligence and Law: the first decade”, Artificial Intelligence and Law 30, no. 4 (2022), https://doi.org/10.1007/s10506-022-09329-4.
[2] Eric Niiler, “Can AI Be a Fair Judge in Court? Estonia Thinks So”, Wired (25 March 2019). https://www.wired.com/story/can-ai-be-fair-judge-court-estonia-thinks-so/. [Accessed 28 January 2020]. In passing, two years later the Ministry of Justice decided to halt the project, see Maria Elisa Tuulik, “Estonia does not develop AI Judge”, news release, 16 February, 2022, https://www.just.ee/en/news/estonia-does-not-develop-ai-judge.
[3] Federico Galli and Giovanni Sartor, “AI approaches to predictive justice: a critical assessment”, Humanities and Rights Global Network Journal 5, no. 1 (2023), https://www.humanitiesandrights.com/journal/index.php/har.
[4] For a critique appraisal of this term, see Dory Reiling, “Courts and Artificial Intelligence”, International Journal for Court Administration 11, no. 2 (2020), https://doi.org/10.36745/ijca.343.
[5] CCJE, Compilation of responses to the questionnaire for the preparation of the CCJE Opinion No. 26 (2023) “Moving forward: use of modern technologies in the judiciary”, Consultative Council of European Judges, (Strasbourg: Council of Europe, 23 March 2023), https://rm.coe.int/compilation-of-responses-for-opinion-no-26–2023-/1680ab4f2a.
[6] See the dashboard at the Resource Center Cyberjustice and AI, Council of Europe at https://public.tableau.com/app/profile/cepej/viz/ResourceCentreCyberjusticeandAI/AITOOLSINITIATIVESREPORT?publish=yes [Accessed 7 June 2024].
[7] Claudio U. Ciborra, “The Platform Organization: Recombining Strategies, Structures, and Surprises”, Organization Science 7, no. 2 (1996), https://www.jstor.org/stable/2634975; Jan Hatzius et al., The Potentially Large Effects of Artificial Intelligence on Economic Growth, Goldman Sachs (2023).
[8] Natalie Pierce and Stephanie Goutos, “Why Lawyers Must Responsibly Embrace Generative AI”, Berkeley Business Law Journal 21, no. 2 (2024), https://doi.org/10.2139/ssrn.4477704. p. 51.
[9] Mariana Mazzucato, “Preventing Digital Feudalism”, Project Syndacate (Oct 2, 2019 2019). https://www.project-syndicate.org/.
[10] Yanis Varoufakis, Technofeudalism. What Killed Capitalism (The Bodley Head, 2023).
[11] Gartner, What’s New in the 2023 Gartner Hype Cycle for Emerging Technologies Gartner Group (2023), https://www.gartner.com/en/articles/what-s-new-in-the-2023-gartner-hype-cycle-for-emerging-technologies.
[12] Courts of New Zealand, Guidelines for use of generative artificial intelligence in courts and tribunals (Judges, Judicial Officers, Tribunal Members and Judicial Support staff), Courts of New Zealand (7 December 2023), p. 1, https://www.courtsofnz.govt.nz/assets/6-Going-to-Court/practice-directions/practice-guidelines/all-benches/20231207-GenAI-Guidelines-Judicial.pdf.
[13] Tambiama Madiega, General-purpose artificial intelligence, European Parliamentary Research Service (Bruxelles: European Parliament, 2023).
[14] Juan David Gutiérrez, “Critical appraisal of large language models in judicial decision-making”, in Handbook on Public Policy and AI, ed. Regine Paul, Emma Carmel, and Jennifer Cobbe (Cheltenham: Edward Elgar Publishing, 2024).
[15] Michael D. Murray, “Artificial Intelligence and the Practice of Law Part 2: Working With Your New AI Staff Attorney”, (June 14, 2023 2023), https://doi.org/10.2139/ssrn.4478748; Benjamin Weiser, “Here’s What Happens When Your Lawyer Uses ChatGPT”, The New York Times (27 May 2023 2023). https://www.nytimes.com/2023/05/27/nyregion/avianca-airline-lawsuit-chatgpt.html. [Accessed 8 June 2024].
[16] CEPEJ, Use of Generative Artificial Intelligence (AI) by judicial professionals in a work-related context, European Commission for the Efficiency of Justice, Council of Europe, Working group on Cyberjustice and Artificial Intelligence (Strasbourg, 12 February 2024), https://rm.coe.int/cepej-gt-cyberjust-2023–5final-en-note-on-generative-ai/1680ae8e01.
[17] Courts and Tribunals Judiciary, Artificial Intelligence (AI) Guidance for Judicial Office Holders, Courts and Tribunals Judiciary (12 December 2023), https://www.judiciary.uk/wp-content/uploads/2023/12/AI-Judicial-Guidance.pdf.
[18] Natalie A. Pierce and Stephanie L. Goutos, “Why lawyers must responsibly embrace generative AI”, Berkeley Business Law Journal 21, no. 2 (2024 forthcoming), Available at https://papers.ssrn.com/sol3/papers.cfm?abstract_id=4477704#; Thomson Reuters Institute, ChatGPT and Generative AI within Law Firms. Law firms see potential, eye practical use cases and more knowledge around risks, Thomson Reuters (2023); LexisNexis, Lawyers cross into the new era of generative AI, LexisNexis (London, 2024), https://www.lexisnexis.co.uk/insights/lawyers-cross-into-the-new-era-of-generative-ai/index.html#group-section-AI-adoption-CIm7Y03Umf.
[19] See <https://openai.com/index/introducing-gpts/> [Accessed 19 June 2024].
[20] Sabrina Küspert, Nicolas Moës, and Connor Dunlop, “The value chain of general-purpose AI. A closer look at the implications of API and open-source accessible GPAI for the EU AI Act”, Ada Lovelace Instiute Blog, 10 February, 2023, https://www.adalovelaceinstitute.org/blog/value-chain-general-purpose-ai/. [Accessed 19 June 2024].
[21] “LexisNexis Launches Second-Generation Legal AI Assistant on Lexis+ AI”, 2024, https://www.lexisnexis.com/community/pressroom/b/news/posts/lexisnexis-launches-second-generation-legal-ai-assistant-on-lexis-ai; “GPT-4 alone is not a reliable legal solution—but it does enable one: CoCounsel harnesses GPT-4’s power to deliver results that legal professionals can rely on”, Casetext, 2023, accessed June 7, 2024. [Accessed 7 June 2024].
[22] José Martins, Frederico Branco, and Henrique Mamede, “Combining low-code development with ChatGPT to novel no-code approaches: A focus-group study”, Intelligent Systems with Applications 20 (2023), https://doi.org/doi.org/10.1016/j.iswa.2023.200289.
[23] Pompeu Casanovas, Mustafa Hashmi, and Marta Poblet, “Generative AI and the Rule of Law” (paper presented at the Artificial Intelligence Governance Ethics and Law (AIGEL), Barcelona, December 19 2022).
[24] Jules White et al., “A Prompt Pattern Catalog to Enhance Prompt Engineering with ChatGP”, arXiv:2302.11382 [cs.SE] (2023).
[25] Martins, Branco, and Mamede, “Combining low-code development with ChatGPT to novel no-code approaches: A focus-group study.”
[26] Microsoft News Center, “Introducing Microsoft 365 Copilot: your copilot for work”, news.microsoft.com, Microsoft, 16 March, 2023, https://news.microsoft.com/2023/03/16/introducing-microsoft-365-copilot-your-copilot-for-work/; “Bring your best ideas to life with Gemini for Google Workspace”, Google, 2024, accessed April 29, 2024, https://workspace.google.com/intl/en_uk/solutions/ai/.
[28] Thomson Reuters Institute, ChatGPT and Generative AI within Law Firms. Law firms see potential, eye practical use cases and more knowledge around risks; LexisNexis, “LexisNexis Launches Second-Generation Legal AI Assistant on Lexis+ AI.”
[29] Hannes Westermann, Jaromir Savelka, and Karim Benyekhlef, “LLMediator: GPT-4 Assisted Online Dispute Resolution” (paper presented at the Proceedings of the ICAIL 2023 Workshop on Artificial Intelligence for Access to Justice co-located with 19th International Conference on AI and Law, 2023).
[30] The survey analyzed the functions provided by 10 GenAI applications developed in Europe, but also in US, Canada, and Japan.
[31] Deloitte Legal, Legal Tech Landscape, Deloitte Touche Tohmatsu limited (2023), https://www2.deloitte.com/content/dam/Deloitte/nl/Documents/legal/deloitte-nl-legaltech-landscape.pdf.
[32] For example, see Lexroom.ai, [accessed 7 June 2024].
[33] For an introduction of the features of RAG see “What is Retrieval-Augmented Generation (RAG)?”, NVIDIA, 2024, accessed 7 June, 2024, https://www.nvidia.com/en-us/glossary/retrieval-augmented-generation/. [Accessed 7 June 2024].
[35] Varun Magesh et al., “Hallucination-Free? Assessing the Reliability of Leading AI Legal Research Tools”, arXiv:2405.20362 [cs.CL] (2024), https://doi.org/10.48550/arXiv.2405.20362.
[36] See as example Jay Rao and Babu Kariyaden Parambath, “Best practices to build generative AI applications on AWS”, AWS Machine Learning Blog, 14 March, 2024, https://aws.amazon.com/it/blogs/machine-learning/best-practices-to-build-generative-ai-applications-on-aws/. [Accessed 7 June 2024].
[37] Magesh et al., “Hallucination-Free? Assessing the Reliability of Leading AI Legal Research Tools.” As examples of claims made by legal tech see LexisNexis, “LexisNexis Launches Second-Generation Legal AI Assistant on Lexis+ AI.” Casetext, “GPT-4 alone is not a reliable legal solution—but it does enable one: CoCounsel harnesses GPT-4’s power to deliver results that legal professionals can rely on.”
[38] See as example https://www.lexisnexis.co.uk/, [accessed 7 June 2024].
[39] Colleen V Chien et al., “How Generative AI Can Help Address the Access to Justice Gap Through the Courts”, Loyola of Los Angeles Law Review (2024 (forthcoming)), https://ssrn.com/abstract=4683309.
[40] https://justica.gov.pt/Servicos/Guia-pratico-da-Justica/pk_vid/cced02719ddccece1711276536ea9032. [Accessed 24 March 2024]. At the access date, the system was in beta version.
[41] Carlos Gandarez, “Practical Guide to Access to justice. (GPJ – Guia Prático de Acesso à Justiça)” (CEPEJ Webinar Generative AI in the field of Justice, online, CEPEJ, 20 February 2024).
[43] This system is part of a more extensive portfolio of AI systems used by the Spanish Ministry, among other things, to anonymize and classify procedural documents automatically. See Javier Hernández, Generative AI. Insights from the Spanish MoJ, Directorate-General for the Digital Transformation of the Administration of Justice, Ministry of the Presidency, Justice and Parliamentary Relations of Spain (Madrid, March 2024).
[44] Claudio Castelli, “Banca dati nazionale della giurisprudenza, come funziona e gli impatti sugli uffici giudiziari”, Agenda Digitale (8 feb 2024). https://www.agendadigitale.eu/documenti/giustizia-digitale/banca-dati-nazionale-della-giurisprudenza-come-funziona-e-gli-impatti-sugli-uffici-giudiziari/.
[45] Alessandro Nencini, Relazione sull’amministrazione della giustizia nel distretto, anno 2023, Corte d’Appello di Firenze (Corte d’Appello di Firenze, 27 Gennaio 2024 2024), https://ca-firenze.giustizia.it/cmsresources/cms/documents/inaugurazione_2024_signed.pdf.
[47] Ben Cost, “Judge asks ChatGPT to decide bail in murder trial”, New York Post (New York), 29 March 2023, https://nypost.com/2023/03/29/judge-asks-chatgpt-for-decision-in-murder-trial/.
[48] Web Desk, “Pakistani court announces first ever decision with ChatGPT help in ‘sex case’”, Daily Pakistan (11 April 2023). https://en.dailypakistan.com.pk/11-Apr-2023/pakistani-court-announces-first-ever-decision-with-chatgpt-help-in-rape-case. [Accessed 10 June 2024].
[49] Hibaq Farah, “Court of Appeals Judge Praises ‘Jolly Useful’ ChatGPT After Asking It For Legal Summary”, The Guardian, 15 September 2023, https://www.theguardian.com/technology/2023/sep/15/court-of-appeal-judge-praises-jolly-useful-chatgpt-after-asking-it-for-legal-summary. [Accessed 10 June 2024].
[51] Maura R Grossman, Paul W Grimm, and Daniel G Brown, “Is disclosure and certification of the use of generative AI really necessary?”, Judicature 107, no. 2 (2023).
[52] Grossman, Grimm, and Brown, “Is disclosure and certification of the use of generative AI really necessary?”, 69; Maura R Grossman et al., “The GPTjudge: justice in a generative AI world”, Duke Law & Technology Review 23, no. 1 (2023), https://scholarship.law.duke.edu/cgi/viewcontent.cgi?article=1378&context=dltr.
[53] The first reaction to the discovery of generative AI hallucinations was a series of standing orders issued by American judges instructing parties to declare the use of generative AI systems. Soon, Canada followed that path, with courts issuing new regulations stipulating that lawyers or parties must inform the court about using artificial intelligence systems (such as ChatGPT or any other AI platform) for legal research or drafting procedural documents submitted to the court. Grossman, Grimm, and Brown, “Is disclosure and certification of the use of generative AI really necessary?.”
[54] The European Bars Federation, European lawyers in the era of ChatGPT, The European Bars Federation (June 2023 2023), https://www.fbe.org/wp-content/uploads/2023/06/European-lawyers-in-the-era-of-ChatGPT-FBE-Guidelines-on-how-lawyers-should-take-advantage-of-the-opportunities-offered-by-large-language-models-and-gene-kopia.pdf.
[55] Courts and Tribunals Judiciary, Artificial Intelligence (AI) Guidance for Judicial Office Holders.
[56] CEPEJ, Use of Generative Artificial Intelligence (AI) by judicial professionals in a work-related context.
[57] Predictice, Charte pour une utilisation responsable de l’intelligence artificielle générative, Predictrice (Paris, 2023). Several other Generative AI guidelines are not considered here. Outside Europe, the guidelines of the Court of New Zealand (CNZ), the self-governing body of the New Zealand judiciary Courts of New Zealand. Since duties and tasks differ, the guidelines are calibrated for judicial officers, lawyers, and clerks, Courts of New Zealand, Guidelines for use of generative artificial intelligence in courts and tribunals (Judges, Judicial Officers, Tribunal Members and Judicial Support staff). The English and New Zealand guidelines share a similar approach. On 16 November 2023, the State Bar of California approved guidelines for using these technologies coherently with the lawyers’ code of ethics. California State Bar, Practical guidance for the use of generative artificial intelligence in the practice of the law, California State Bar (California State Bar, 16 November 2023), https://www.calbar.ca.gov/Portals/0/documents/ethics/Generative-AI-Practical-Guidance.pdf.
[58] Courts and Tribunals Judiciary, Artificial Intelligence (AI) Guidance for Judicial Office Holders. Section 1.
[59] Predictice, Charte pour une utilisation responsable de l’intelligence artificielle générative. Section 1.
[60] CEPEJ, Use of Generative Artificial Intelligence (AI) by judicial professionals in a work-related context. Section E.5.
[61] CEPEJ, Use of Generative Artificial Intelligence (AI) by judicial professionals in a work-related context. Section E.4.
[62] Courts and Tribunals Judiciary, Artificial Intelligence (AI) Guidance for Judicial Office Holders. Section 2.
[63] Courts and Tribunals Judiciary, Artificial Intelligence (AI) Guidance for Judicial Office Holders. Section 5.
[64] Courts and Tribunals Judiciary, Artificial Intelligence (AI) Guidance for Judicial Office Holders. Section 2.
[65] Marks Fenwick, Wulf A. Kaal, and Erik P. M. Vermeulen, “Regulation Tomorrow: What Happens When Technology Is Faster than the Law”, American University Business Law Review 6, no. 3 (2017), https://heinonline.org/HOL/Print?collection=journals&handle=hein.journals/aubulrw6&id=596.
[67] Predictice, Charte pour une utilisation responsable de l’intelligence artificielle générative. Section 2.
[68] Predictice, Charte pour une utilisation responsable de l’intelligence artificielle générative. Section 3.
[69] Courts and Tribunals Judiciary, Artificial Intelligence (AI) Guidance for Judicial Office Holders. Section 3.
[70] CEPEJ, Use of Generative Artificial Intelligence (AI) by judicial professionals in a work-related context. Section D.6.
[71] CEPEJ, Use of Generative Artificial Intelligence (AI) by judicial professionals in a work-related context. Section D.9.
[72] CEPEJ, Use of Generative Artificial Intelligence (AI) by judicial professionals in a work-related context. Section E.3.
[73] Courts and Tribunals Judiciary, Artificial Intelligence (AI) Guidance for Judicial Office Holders. Section 1.
[76] CEPEJ, Use of Generative Artificial Intelligence (AI) by judicial professionals in a work-related context. Section D.7.
[77] Courts and Tribunals Judiciary, Artificial Intelligence (AI) Guidance for Judicial Office Holders. Section 7.
[78] In passing, this point is anticipated also by the guidelines provided by the New Zealand judiciary.
[79] California State Bar, Practical guidance for the use of generative artificial intelligence in the practice of the law.; Grossman, Grimm, and Brown, “Is disclosure and certification of the use of generative AI really necessary?”.
[80] CEPEJ, Use of Generative Artificial Intelligence (AI) by judicial professionals in a work-related context. Section D.1.
[81] Courts and Tribunals Judiciary, Artificial Intelligence (AI) Guidance for Judicial Office Holders. Section CTJ 6.
[82] Shenglai Zeng et al., “The Good and The Bad: Exploring Privacy Issues in Retrieval-Augmented Generation (RAG)”, arXiv:2402.16893 [cs.CR] (23 February 2024). https://arxiv.org/abs/2402.16893.
[83] Francesco Contini, “Artificial Intelligence and the Transformation of Humans, Law and Technology Interactions in Judicial Proceedings”, Law, Technology and Humans 2, no. 1 (2020), https://doi.org/10.5204/lthj.v2i1.1478.
[84] Ben Shneiderman, “Bridging the Gap Between Ethics and Practice”, ACM Transactions on Interactive Intelligent Systems 10, no. 4 (2020), https://doi.org/10.1145/3419764.
[85] Magesh et al., “Hallucination-Free? Assessing the Reliability of Leading AI Legal Research Tools.”
[86] Antoinette Rouvroy, “Adopt AI, think later”. The Coué method to the rescue of artificial intelligence (2020), https://www.academia.edu/42067794/_Adopt_AI_think_later_The_Cou%C3%A9_method_to_the_rescue_of_artificial_intelligence.
[87] Antonio Cordella and Francesco Contini, Digital technologies for better justice. A toolkit for action (Washington DC: IADB, 2020). https://publications.iadb.org/en/digital-technologies-for-better-justice-a-toolkit-for-action).
[88] Murray, “Artificial Intelligence and the Practice of Law Part 2: Working With Your New AI Staff Attorney.”
[89] For instance, a judge may be pressured by court leaders to decide more cases, or a lawyer may be pushed by a client to finalize a case assessment too quickly. In these scenarios, the interests of the principals (court leaders, clients) are not aligned with those of the agents (judges, lawyers), leading to a principal-agent problem.
[90] As a simple example, a client does not have effective mechanisms to check if the lawyer has made the required controls on GenAI outputs.
[91] Noam Kolt, “Governing AI Agents”, Available at SSRN (2024), https://doi.org/10.2139/ssrn.4772956.
Acknowledgements
This paper summarizes the findings of a series of workshops and conferences on artificial intelligence promoted in Milan by the Gruppo IA & Diritto of the Italian Osservatori per la Giustizia Civile during 2023 and 2024, as well as of the conference ‘IA and Democratic Values’ held by the Italian Bar Association at the G7 of the BAR held in Rome on 16 April 2024. I am grateful to all those who – in these events – offered me the opportunity to explore the subject, particularly Elena Riva Crugnola and Silvia Toffoletto. I am also grateful to the colleagues — too many to mention here — who helped me with their insightful discussions, valuable suggestions, and regular updates, and to the two blind reviewers for their valuable comments. Nonetheless, the author is solely responsible for the content of this work.
Competing Interests
The author has no competing interests to declare.