By: Rachel J. Eisenhaure, Esq.
There is no doubt ChatGPT has the potential to change the legal profession. The form that change will take is as yet unknown.
In considering the role that ChatGPT might play in litigation, there are two different roles to consider. The first role, and the one most heavily discussed by others, is the role that it could play in drafting prose. Some theorize that ChatGPT, being cheap and efficient, will replace the junior associate class in law firms. ChatGPT has interesting potential as a drafting tool for litigation, though at this stage it is too early to present a threat to associate jobs. For drafting complex documents, ChatGPT has a fatal flaw: what the tech industry terms “hallucination.” It invents facts. Clearly one would not want a lawyer inventing facts. So too one would not want a lawyer relying on a tool that invents facts. This flaw is a dealbreaker…for now, at least. Perhaps the hallucination problem can be solved with oversight once users are sufficiently trained to spot flaws, the same way a generation of lawyers has learned to use text searchable legal databases without taking quotes out of context. To our children, spotting a hallucination may seem as obvious as avoiding clicking on virus attachments in email or spotting an urban legend on social media. For those of us in the present, anyone relying on ChatGPT to generate writing must – at a minimum – carefully vet the product to make sure that the ChatGPT-produced text is free of hallucinations.
But there is a second potential use for ChatGPT: as a tool for generating ideas. ChatGPT is like the fabled monkey with the keyboard, quickly producing different ideas which may or may not have value. Eventually it will produce the works of Shakespeare. But only a human reader can recognize whether the generated text has value and sort those ideas worth saving for consideration from the ones destined for the recycle bin.
Cost-effective idea generation can significantly enhance appellate practice. Because of the evolving nature of the common law, appellate courts are often presented with problems that have not previously been addressed. It takes creativity to bring proposed solutions to the Court. The best proposed solutions take into account the arc of existing common law as well as the future impact of the decision. A court can make the most informed decision about the possible directions for the growth of the common law when it is presented with options, and those options are supported with full briefing on the historical justification for the proposed rule as well as analysis of possible future impact. ChatGPT can help at the first step: generating possible solutions for advocates to consider and modify. The use of ChatGPT to generate creative solutions poses no tricky ethical questions because its work does not substitute for human analysis, but supplements it. Human analysis is still the prime mover.
This use of ChatGPT has already occurred in the judicial system. A judge in Columbia used ChatGPT to propose an option for extending the law, where no common law solution already existed. Judge Juan Manuel Padilla Garcia posed legal questions about a novel fact pattern before him to the ChatGPT to see how it proposed to extend existing law.[1] He did not use ChatGPT as a simple writing tool, but as an idea generator. Judge Garcia then evaluated the responses of ChatGPT as he would any other secondary analysis with limited persuasive value. He did not defer to ChatGPT’s synthesis of the law. He did not delegate his judicial authority to ChatGPT. Instead, he treated ChatGPT’s proposed solution no differently than any other solution proposed by the parties. ChatGPT was simply a way to generate a third option to consider. It was still within the role of the judge to consider whether that solution was an appropriate extension of the common law and the best available proposal for the extension of the common law.
Another potential use of ChatGTP in the appellate context is A/B testing of brief-writing approaches. A/B Testing is a way to test two options to consider which option performs better. A/B Testing is used by the tech industry to show multiple versions of the same webpage to different users to gauge reaction to the alternate versions. The data from the testing guides design decisions. A/B testing is a useful tool to evaluate the impact of a change in user experience. For a major appeal where cost is no option, it is not unheard of for a firm to assign two groups to brief the case independently, so that the resulting briefs can be compared and the best parts of each brief incorporated into the final product. While this approach has great value for idea generation, producing two briefs is prohibitively costly for most applications. Most appeals involve lawyers writing only a single set of briefs, without the benefit of comparing fully realized options or presenting those options to sample audiences.
If cost were no option, there is an incredible value in seeing two different approaches. Just as an interior designer might mock up two different versions of a room or an architect might mock up two different views of a house, ChatGPT could allow a law firm to mock up multiple approaches to writing the brief, modulating the style of argument and the order of ideas without incurring the cost of writing a brief twice or more. Envision, for example, the creative power of enlisting sample readers to compare and provide feedback on the styles and organization of full briefs the way that potential oral argument responses are vetted through a moot court.
Certain ethical considerations must be kept in mind: Lawyers have a duty to maintain client confidentiality, a duty codified in Massachusetts as Rule 1.6 of the Rules of Professional Conduct. Artificial intelligence is designed to learn. Before a system using artificial intelligence is given confidential client facts, lawyers must be sure they – and their clients – understand whether the system is retaining these facts for future use. Rule 1.6 required “informed consent” by the client for any disclosure of confidential information. It would be completely unacceptable for a system to generate text that relied upon confidential facts it had learned on another matter. To lawyers, the importance of client confidentiality is understood. Humans, whether lawyers or otherwise, can understand that they should not use confidential facts in another matter. It is not clear whether artificial intelligence can draw that line. Until we can be certain that confidentiality will be protected, no confidential facts should be input for these systems.
Lawyers also have a duty of candor to the tribunal, which in Massachusetts is Rule 3.3 of the Rules of Professional Conduct. As chatbots become more common, it is possible that either court rules or ethical standards will adopt limitations on the use of chatbot-generated text in court filings. Under the current rules, a lawyer must ensure that the information placed in front of the court is true and accurate. If ChatGPT has been used to generate text, the lawyer must confirm that the overall meaning of the generated text is accurate and argues persuasively for the result that the lawyer intends, confirm the accuracy of all subsidiary facts referenced in the generated text, and confirm the accuracy of all legal citations referenced in the generated text. Additionally, courts may require that all lawyers filing any brief include a certification at the end of the brief that the filing either does not contain generated text or that any generated text has been checked for accuracy, similar to the current certification requirements in appellate briefs for word count compliance.
Judges, too, must be cautious of their ethical obligations in utilizing this tool. In Massachusetts, Canon 2.3 of the Code of Judicial Conduct requires that judges avoid bias. Because chatbots are trained on large amount of data and online texts, chatbots have the potential to regurgitate the biases implicit in whatever corpus of material they were trained upon.[2] At a minimum, use of a chatbot should be disclosed in the opinion. Where a judge does not defer to the algorithm, but uses his judgment to independently evaluate options that a chatbot has generated, he is not displacing his judgment with that of the machine.
Although there are limitations to the use of ChatGPT in law, the potential for reducing the cost of producing sample documents allows for unprecedented experimentation that, if used appropriately, will enhance the quality of work throughout the profession and particularly at the appellate level. Lawyers must be mindful of the existing ethical rules in any use of this software: ChatGPT-generated text is not ready to be relied upon in any court filing because of the unsolved hallucination problem and the challenge of sufficient oversight for a technology we are still learning to understand. Nonetheless, the tool can be put to use in other ways. Use of ChatGPT to create style samples of briefs allows for A/B testing of the impact of legal writing and allows for the creation of comparison samples of different advocacy styles that is rarely feasible due to the cost of drafting. While the technology is not yet at a point where we can outsource the writing of a final draft (and it may never be), we are at the point where generated text can be used to create samples for A/B testing of different argument approaches as part of planning the argument
In light of the unique concerns presented by the use of ChatGPT specifically, and AI generally, in the legal profession, we as a community should consider sooner rather than later what rules should apply. In the evolution of ethical rules to govern the use of new technologies, there is no fate but what we make for ourselves.
[1] https://www.vice.com/en/article/k7bdmv/judge-used-chatgpt-to-make-court-decision (last accessed 3/29/2023). Judge Juan Manuel Padilla Garcia’s findings are available here: https://www.diariojudicial.com/public/documentos/000/106/904/000106904.pdf (last accessed 3/29/2023).
[2] Compare, for example, the findings of racial bias in the COMPAS recidivism algorithm. See, e.g., https://www.propublica.org/article/how-we-analyzed-the-compas-recidivism-algorithm (last accessed 3/29/2023).
*****
This alert is for informational purposes only and may be considered advertising. It does not constitute the rendering of legal, tax or professional advice or services. You should seek specific detailed legal advice prior to taking any definitive actions.