Responsible AI use for courts: Minimizing and managing hallucinations and ensuring veracity

Key insights:

• • • AI usage in courts needs verifiable reliability— Unlike other fields, errors and hallucinations caused by AI in a court setting can create due-process issues.

    • Skepticism is professional responsibility— Judges’ interrogation of AI sources and accountability concerns are vital guardrails to minimizing these problems.

    • Governance over perfection— Courts and legal professionals should focus on systematic management of AI hallucinations through clear protocols, human oversight, and mandatory verification to ensure veracity.

AI hallucinations have become one of the most urgent and most misunderstood issues in professional work today; and as generative AI (GenAI) moves from and interesting experiment to common usage in many workplace infrastructures, these issues can cause significant problems, especially for courts and the professionals and individuals that use them.

Jump to ↓

Responsible AI use for courts: Minimizing and managing hallucinations and ensuring veracity

Today, AI can be used in everything from assisted research to guided drafting of documents, court briefs, and even court orders. With the development of tools supported by GenAI and agentic AI, the very infrastructure of professional work has shifted to include these offerings.

Yet, in most business settings, a wrong answer is an inconvenience. It requires minor corrections and has minimal impact. In the justice system, a wrong answer can be a due-process problem that strongly underscores the need for courts and legal professionals to ensure that their AI use is verifiably reliable when it counts.

At the same time, the direction of travel is clear: AI adoption isn’t a fad we can simply wait out, and it isn’t inherently at odds with high-stakes decision-making. Used well, these tools can reduce administrative burden, speed up access to relevant information, and help court professionals navigate large volumes of material more efficiently. The real question is not whether courts will encounter AI in their workflows, but how they will define responsible use, especially in moments in which accuracy isn’t a feature, it’s the foundation.

“Whether you are a judge [or] an attorney, credibility is everything, particularly when you come before the court.”

— Justice Tanya R. Kennedy Associate Justice of the Appellate Division, First Judicial Department of New York

To examine these issues more deeply, the Thomson Reuters Institute has published a new report, Responsible AI use for courts: Minimizing and managing hallucinations and ensuring veracity. The report frames hallucinations not as a sensationalistic gotcha, but as a practical risk that must be managed with policy, process, and professional judgment. The report also features valuable insight on this subject from judges and court stakeholders who today are evaluating AI in the real operating environment of legal proceedings, courtroom expectations, and the daily administration of justice.

This perspective is essential. Technical teams can explain how models generate language and why they sometimes produce confident-sounding errors. However, judges and court staff can explain something equally important — what accuracy actually means in practice. In courts, accuracy isn’t just about getting the gist right; rather, it’s about precise citations, faithful characterization of the record, correct procedural posture, and language that withstands scrutiny. As the report points out, relied-upon hallucinated information isn’t merely bad output, it can lead to a potential distortion of justice.

Managing AI as professional responsibility

Crucially, the report reflects that judicial skepticism about AI is not simple technophobia — it’s professional responsibility. Judges are trained to interrogate sources, weigh credibility, and understand the downstream consequences of errors. Judges may ask, What is the provenance of this information? Can I reproduce it independently? And who is accountable if it’s wrong? These questions aren’t barriers to innovation; indeed, they are the guardrails that this innovation requires.

What emerges is a pragmatic middle ground that embraces the upside of AI use in courts while treating hallucinations as a predictable occurrence that can be managed systematically. Rather than concluding AI hallucinates, therefore AI can’t be used, the more workable conclusion is AI can hallucinate, therefore AI outputs must be designed, handled, and verified accordingly, likely with other advanced tech tools. As the report points out, courts don’t need a perfect AI; rather, they need repeatable protocols that keep human decision-makers in control and keep the record clean.

As the report ultimately demonstrates, managing hallucinations in courts isn’t about chasing perfection, it’s about protecting veracity. It’s about using the right advanced tech tools to build workflows in which the technology consistently supports the truth-finding process instead of quietly eroding it. And it’s about recognizing that in the legal system, responsibility doesn’t disappear when a new tool arrives — it becomes even more important to ensure the new tool doesn’t erode that either.

You can download

a full copy of the Thomson Reuters Institute’s “Responsible AI use for courts: Minimizing and managing hallucinations and ensuring veracity” by filling out the form below: