AI Hallucinated Citations

What AI hallucination is

AI hallucination is the generation by a large language model of plausible-looking content that is not grounded in any underlying source. The term applies to text, image, and code generation alike, but the version that lands hardest in regulated work is the fabricated authority: a fictitious case citation, a non-existent statute, a quote that no one actually said, complete with a realistic docket number, judge name, and parenthetical.

Hallucination is a structural property of how generative language models work, not a defect that careful users can route around by prompt engineering alone. OpenAI's research team has published its analysis of why language models hallucinate, attributing the behavior to the next-token prediction objective and to training-data scoring systems that reward confident assertions over hedged uncertainty (OpenAI, "Why Language Models Hallucinate," September 5, 2025). Anthropic's interpretability research describes related failure modes including sycophancy and confabulation (Anthropic Research). The practical consequence: hallucination is intrinsic to current systems, including the systems marketed for legal work.

Why it matters for licensed professionals

For lawyers, financial advisors, accountants, and other licensed professionals, hallucinated output that reaches a client filing or tribunal carries direct exposure under the rules of professional conduct. The duty to verify is not delegable to the tool. Mata v. Avianca, the first major federal AI-hallucination sanctions order, establishes the rule: Rule 11's reasonable inquiry duty survives delegation to an AI assistant; the lawyer who signs the brief is accountable for every citation it contains.

Five duties most often surface in AI-hallucination sanctions opinions:

• Rule 1.1 (Competence). Reasonable understanding of the AI tool's capabilities and limits is part of the competence baseline. ABA Formal Opinion 512 confirms this for generative AI specifically.
• Rule 1.3 (Diligence). Failing to verify AI output is a diligence violation independent of competence; Colorado's People v. Crabill treats it as both.
• Rule 3.3 (Candor toward the tribunal). Submitting fabricated authority is a candor violation. Failing to correct after discovery compounds the harm.
• Rule 5.1 / 5.3 (Supervision). Firm-level oversight of AI-assisted work product is the supervisory duty Mata reaches.
• Rule 8.4 (Misconduct). Concealment or false attribution after discovery (the Crabill aggravator) elevates the sanction.

The Opinion 512 compliance guide on this site maps each duty to the documentation that satisfies it.

How often it is happening

Two data sources matter for sizing the problem.

Empirical research. Stanford RegLab's 2024 study (Magesh, Surani, Narayanan, et al., "Hallucination-Free? Assessing the Reliability of Leading AI Legal Research Tools") tested purpose-built legal AI products, including tools marketed as "hallucination-free." The finding: even the most reliable products hallucinated on 17% of legal-research queries; less reliable products hallucinated on roughly one in three. The marketing claim of "hallucination-free" did not survive controlled testing. Stanford HAI's summary is the readable plain-language version.

Documented sanctions. This site's tracker documents 496 decided AI-sanctions cases across US courts and bar disciplinary bodies as of the rendering date. The year-by-year counts are in the stats block above and are computed live from primary-source-cited tracker entries on each build; the cases tracker is the browsable index.

Together, the two data sources point in the same direction. Hallucination is not a fringe failure mode. It is a routine output of every model a US lawyer might use, including products explicitly marketed for legal research.

Sanctions lawyers have faced

Six representative cases from the tracker. Each is verified against a primary-source order or sanctions opinion. The full set is on the cases tracker.

• Mata v. Avianca, Inc. (S.D.N.Y., 2023) : $5,000. Verify every AI-generated citation against the underlying reporter or docket before signing. Doubling down compounds the harm.
• Park v. Kim (2d Cir., 2024) .
• People v. Crabill (Colo. OPDJ, 2023) : 1 year + 1 day suspension; 90 days active. The coverup was worse than the error. Disclose immediately on discovery; Rule 3.3 candor obligations are the only safe path.
• Wadsworth v. Walmart Inc. (D. Wyo., 2025) : $3,000 (Ayala) + $1,000 each (Morgan, Goody); Ayala's pro hac vice revoked. A firm-branded AI tool does not change Rule 11. Supervising attorneys cannot blindly rely on subordinates who used AI.
• Garner v. Kadince (Utah Ct. App., 2025) : Attorney fees + client fee refund + $1,000 to access-to-justice nonprofit.
• Kohls v. Ellison (D. Minn., 2025) .

How to prevent it in regulated work

The defense is procedural. No model is hallucination-free; the firm's job is to make sure no hallucination reaches a tribunal, a client, or a regulator unverified. Five practices reach that bar.

1. Citation verification before signature. Every cite in a filing opened in the underlying reporter or docket, by a named human, before the brief is signed. The verification log template captures the per-matter record.
2. Approved-tool list, with vendor due diligence. Each approved tool reviewed for terms of use, data retention, and training-on-input behavior. The vendor due diligence checklist is the 11-item file.
3. Training records. Every attorney and staff member using an approved tool has completed tool-specific training, logged by date. Updated when the tool materially changes.
4. Supervisory protocol. Any work product generated with AI assistance reviewed by a supervising lawyer before it leaves the firm. Logged.
5. Incident response. When a hallucination reaches a filing, the response is immediate disclosure to the court, not defense. The incident response runbook walks the steps in order. The Crabill aggravator (concealment after discovery) is the avoidable mistake.

The firm-side documentation set that satisfies these practices is the same set a malpractice carrier asks for at renewal. See AI Liability Insurance for Law Firms for the renewal context, and the resources hub for every template.

Primary sources

• OpenAI, "Why Language Models Hallucinate" (September 5, 2025): openai.com.
• Magesh, Surani, Narayanan, et al., "Hallucination-Free? Assessing the Reliability of Leading AI Legal Research Tools," Stanford RegLab (2024): arXiv:2405.20362; plain-language summary on Stanford HAI.
• Anthropic Research, interpretability and alignment work on language model failure modes: anthropic.com/research.
• ABA Standing Committee on Ethics and Professional Responsibility, Formal Opinion 512 (July 29, 2024): americanbar.org (PDF).
• NIST AI Risk Management Framework: nist.gov.

Live page; case stats refresh on each build. Research citations last verified: 2026-05-05.