Pointer-Resolved Drafting: A Citation-Graph RAG Architecture for Verifiable Quotation in Legal Memos
Name
Yin_Final.pdf
Size
202.97 KB
Format
Adobe PDF
Checksum (MD5)
4daaefbcd2ed194b98ce61ce85bae997
Author(s)
Yin, Joshua
Taylor, Jonathan
Date Issued
August 20, 2026
Abstract
Large language models (LLMs) routinely fabricate legal citations and distort holdings, with reported hallucination rates of 17-33% across purpose-built legal assistants and 58-88% across general-purpose models. Standard retrieval-augmented generation (RAG) reduces but does not eliminate these failures, because the model still controls the surface form of every quotation and citation it emits. We describe a legal-research pipeline for U.S. military justice built on three coupled commitments. First, a unified multi-model store (SurrealDB) keeps documents, dense embeddings, and citation-graph edges under one schema and one transaction boundary. Second, a LangGraph state machine decomposes the question, retrieves via hybrid search with model-guided citation-graph expansion, deterministically prunes overruled or superseded authority, and scores the survivors with a multi-factor rubric. Third, a pointer-resolved drafting pattern forbids the model from emitting verbatim quoted text and instead requires it to emit chunk pointers that a deterministic Assembler resolves against the database. This pattern converts the failure mode of fabricating quoted text and reporter citations from a probabilistic risk into a structural property of the pipeline, under a stated trust model, while leaving selection, paraphrase distortion, and retrieval miss as bounded research problems. We test this property in a three-system comparison against vendor baselines on seventeen pre-registered probes spanning eleven failure-mode families, with the strongest no-tools baseline (Claude Opus 4.7) sampled at three replicates per probe. Across the probe set, the pointer-resolved Drafter emits no fabricated verbatim quotation; its sole residual failure is a narrative-prose misattribution on a deliberately-induced semantic-adjacency probe, where the quoted text remains faithful to the database but the surrounding prose drifts. The probes are designed to elicit hallucination and the per-probe verdicts are existence outcomes on specific phrasings, not rate estimates of real-world behavior.
Subjects
retrieval-augmented generation
legal informatics
knowledge graphs
hallucination mitigation
citation graphs
constrained decoding
LangGraph
SurrealDB
MIT Department
Lincoln Laboratory
Terms of Use
All rights reserved
Persistent DSpace Link
