Drafting

Legal text carries illocutionary force: a clause creates obligations, a brief persuades a tribunal, an opinion binds future courts. Output that is fluent and faithful to a prompt is not necessarily output that does the right thing in the legal sense. Moreover, legal documents are stress tested by counterparties, skeptical judges, and opposing counsel whose job it is to exploit every weakness.

Transactional drafting is an iterative negotiation. A draft encodes a position; terms are redlined, then finalized through multiple rounds of debate. Market conventions vary by deal size, geography, and industry. The technical challenges include long-document coherence under continuous edits, negotiation-state modeling, and benchmarking what’s market across both private and public deal data.

Litigation drafting is constrained on three axes simultaneously — exhaustive recall of relevant authority, faithfulness of factual claims against the record, and doctrinally sound argumentation. Authority modeling operates over jurisdictionally aware citator graphs. Adversarial multi-agent architectures surface missing authority and likely objections. Beyond measuring factual grounding and argument validity, we build simulators that model how users actually engage with drafted content.

Evaluation

How do you measure the quality of work product when the problems are non-trivial, even for experts to verify? Correctness is rarely a simple yes or no. A model can cite a real case for the wrong point or write something plausible that overlooks authority the court might rely on. Most model benchmarks don't capture real world tasks or the nuanced ways they fail.

Thomson Reuters Labs endeavors to measure accuracy and overall efficacy of end-to-end workflows the way our customers experience them in the product. We work closely with teams of experienced lawyers, accountants, and domain experts to measure response quality, domain-specific error types, and dimensions of utility. We track when and how new models meaningfully improve on hard, in-domain tasks. We build test sets, evaluation methods, and grading systems with the statistical rigor needed to separate signal from noise. We treat evaluation as a research problem focused on scaling human judgment, with the next phase leaning into user simulations and reward models that approximate user experience.