A defense lawyer in New Mexico gave an AI tool the trial transcript from a murder appeal and asked for a summary. The summary described witnesses who did not exist and police testimony that was never given. The lawyer submitted the summary to the court without verifying it, and the New Mexico Supreme Court held him in contempt, fined him, and referred him for disciplinary review.

This case opened the second session of the Thomson Reuters Institute’s AI & Future of Legal Practice webinar series, Trust but Verify: Catching Errors in AI Output Before They Reach the Court or Client. At the webinar, the panel focused on one practical question: How do lawyers verify AI output before it leaves their desk?

A verification framework for lawyers

Verification starts earlier than most lawyers assume, said panelist Kirsten Talmage, Chief Strategy Officer at the Practising Law Institute (PLI). She described a four-step method drawn from PLI’s AI-Ready Lawyer framework:

1. Form an independent analysis — Before writing a prompt, decide which task to delegate and which judgment to retain. Without this step, Talmage said, the first coherent answer can anchor your thinking before you have formed a view.
2. Use AI to enhance — Because you have already done your own analysis, the AI can be used to enhance your professional judgment, giving you a view against which to compare its output.
3. Verify the output against the source — For any case the AI supplies, ask whether such a case exists, whether it says what the AI claims, and whether it supports the proposition. Trace every factual assertion AI gives back to the record because the AI’s account of the facts may not stand on its own.
4. Reconcile any discrepancies — If the AI’s answer differs from what you expected, work out why. Neither version should be thought to be automatically correct.

When asked about one habit to adopt tomorrow, Mark Martin, former Chief Justice of the North Carolina Supreme Court and founding dean of the Kenneth F. Kahn School of Law at High Point University, offered a shorthand for the same discipline: ERV:

• Employ your own research.
• Retain your own independent professional judgment.
• Verify citations, quotations, legal propositions, and factual assertions against original sources.

This memory device puts the order of the work in three clear steps. Reasoning first, followed by judgment, and only then do you check the sources. This matches the panel’s other points about not anchoring on AI’s first answer and about verifying against primary sources.

Justice Martin added that this way, lawyers stay accountable for the exercise of their own judgment.

A supervising lawyer’s method for verifying someone else’s work

Valerie McConnell, Vice President of Solutions Engineering at Thomson Reuters, moderated the panel and demonstrated how a supervising lawyer can use an AI tool to organize and check a brief. She gave the tool a draft brief and the deposition transcript it cites most heavily, then asked the tool to extract every factual assertion attributed to the transcript.

Then, for each one of those assertions, the tool reported it as written (including any quoted language), the speaker the draft attributes it to, the record citation the draft provides, and the page and line on which the supporting testimony appears, quoted exactly. In those instances the tool could not find support, McConnell instructed to state Not found.

You can view McConnell's supervisory method here

She then asked the AI tool to put the results in a table, which she finds easier to read and which turns each assertion into a discrete item to check. The finished table (below) showed which statements the transcript supports and which it does not. Because the tool displayed the source passage alongside the transcript, McConnell could compare the two directly rather than rely on the tool’s summary.

She then repeated the exercise for the law. She asked the tool for the full citation and a link to each opinion the brief cites, along with the specific portion upon which the brief relied. This let her read each case and confirm the tool had characterized each brief accurately. In both checks, the AI tool pointed to where support should be, and the lawyer did the reading.

Teaching verification habits to both new and seasoned lawyers

The panelists offered several ways to build these habits at any career stage. For law students, PLI's Talmage suggested giving first-year students AI-generated analysis with planted errors and having a discussionabout what the authority says. The exercise builds on lessons about primary versus secondary authority and how to read a case, she noted.

At Fredrikson & Byron, Norah Olson Bluvshtein, Chief AI & Legal Operations Officer, leads an approach that pairs policy guidance with formal training on verifying and supervising AI-assisted work. The firm combines onboarding, one-on-one coaching with research librarians, and workshop-style simulations that go beyond lecture-based CLE.

Experienced attorneys who supervise early-career lawyers carry an added duty to learn and practice verification, Talmage noted; and Olson Bluvshtein agreed, adding that no one has fully solved how junior lawyers will build judgment as AI absorbs routine research.

Supervisors need to take the apprenticeship model seriously as the profession works through a “messy middle” in how it uses AI, Olson Bluvshtein explained.

Indeed, the panel agreed that AI can speed up many aspects of legal work, but it cannot take over a lawyer's judgment. As Olson Bluvshtein put it: “Don’t outsource your brain.”

Join us for the next session, Beyond Human in the Loop: Redefining Accountability in AI-Assisted Legal Work, on November 19 in the AI & Future of Legal Practice series.