As legal documents are increasingly drafted with large language models, falsified citations have become a real problem for courts and the lawyers who appear before them. Georgia Tech researchers have built an open-source tool designed to catch those fabricated references before they reach a judge.
The tool, called the Mellea Legal Requirements Checker (Mellea-LRC), can identify and verify case law citations, with a specific focus on citations hallucinated by AI. It was created through a fellowship program at Georgia Tech's Center for Scientific Software Engineering (CSSE) and Open Source Program Office (OSPO), funded by a donation from IBM, according to the university's announcement. For more context on this story, see our ongoing artificial intelligence updates.
Master's students Adrian Lozada and Di Wu were selected as fellows to build the tool with mentors from CSSE and OSPO. Their goal, the university said, was to give lawyers and others in the legal field a simple way to verify legal citations.
How It Works
Mellea-LRC combines large language models with IBM's Mellea open-source framework, which is built around an "instruct, verify, repair" paradigm: developers write the rules that an LLM's output must satisfy as ordinary code, and the framework retries model calls until the output meets them.
"IBM's Mellea framework was the foundation we built it on. Mellea's job is to make model calls predictable: you write the rules the output must satisfy as ordinary code, and the framework retries until LLM output meets them. Our job was to leverage the framework to build the legal application from end to end," Wu said in the announcement.
The tool processes a document in about eight minutes. It first extracts case law citations from a legal filing, then verifies that each citation is authentic and that it actually supports the document's content — the second part being what distinguishes a hallucinated citation from a merely fake one. A fabricated case that does not exist fails the first check; a real case that does not say what the filing claims fails the second.
Tested Against Real Fabricated Filings
The team evaluated Mellea-LRC on 26 public court-filing documents known to contain falsified citations — the same class of incidents that has drawn sanctions and scrutiny in courts as AI-drafted briefs have spread.
The problem Mellea-LRC targets is well documented: LLMs generate plausible-looking case names, citations, and quotations that do not correspond to any real precedent, and attorneys have faced penalties after submitting such filings without checking them.
Open Source by Design
A core requirement of the project was that the tool's code be open source. Alongside the tool, the team released a new test and validation dataset on Hugging Face for future benchmarking studies.
"Agentic AI is growing quickly, and much of the growth happens in open-source projects, where anyone can read, use, and improve the work," Lozada said.
Jeff Young, OSPO director and principal research scientist at Georgia Tech, said IBM's Mellea framework was critical to the work because its "instruct, verify, repair" approach allows the creation of reliable LLM loops.
The Architecture Behind Reliable Output
What makes the Mellea approach notable is where it places the burden of reliability. Most hallucination mitigation strategies ask the model itself to be more careful — through prompting, fine-tuning, or retrieval augmentation. Mellea-LRC inverts that: the correctness requirements live in ordinary code outside the model, and the framework keeps calling the model until the output provably satisfies them.
That design has two practical consequences. First, the verification logic is auditable — a lawyer or a court can read exactly what rules a citation had to pass, rather than trusting an opaque model judgment. Second, the same pattern generalizes beyond legal citations to any domain where outputs can be checked mechanically, which is precisely the property that has made agentic frameworks attractive to enterprise developers.
The fellowship structure is also worth noting. Rather than a closed corporate lab, the project put two master's students, mentorship from CSSE and OSPO, and IBM funding together around a narrow, well-defined verification problem — and produced a working tool plus a public benchmark dataset.
Why It Matters for the Legal Industry
Courts have become a flashpoint for AI reliability problems. The verification workflow Mellea-LRC automates — confirming a citation exists and checking it says what the drafter claims — is exactly the diligence step that human reviewers skip when deadlines are tight.
An eight-minute processing time per document makes it practical to run before filing, rather than as a forensic exercise after a judge flags a nonexistent case. And because both the tool and its evaluation dataset are open source, other teams can audit how it works, extend it to additional jurisdictions and citation formats, and benchmark improvements — an approach that contrasts with closed, proprietary hallucination checkers.
The project also signals where enterprise AI tooling is heading: rather than trying to make models hallucinate less, teams like Georgia Tech's are building verification layers around them, using structured rules and automated retries to make model output predictable enough for high-stakes professions.
For the legal industry, the calculus is straightforward. AI can draft a brief in minutes, and tools like Mellea-LRC can check its citations in about the same time. The remaining variable is whether firms make that check a mandatory step — before the filing, not after the sanction.
---
Stay Ahead of AIGet the latest AI news, analysis, and breakthroughs — all in one place.
Read more AI news →
