AI Hallucinations in Law: Why Fake Legal Citations Can Get Lawyers Sanctioned
Artificial intelligence is quickly becoming part of everyday legal work. Lawyers, law students, and self-represented litigants use AI to organize case files, summarize court decisions, explain legal terminology, brainstorm arguments, and prepare first drafts.
But there is a serious problem hiding behind the impressive writing: AI can sometimes make things up.
In legal work, that can mean a completely fictional court case, a quotation that never appeared in an opinion, an incorrect statute, or a real case that does not actually support the argument being made.
These mistakes are often called AI hallucinations.
And in a courtroom, a believable mistake can become a very real problem.
U.S. courts have sanctioned attorneys and litigants for submitting fabricated or inaccurate legal authorities. The issue has continued well beyond the famous Mata v. Avianca case, with courts in 2025 and 2026 continuing to address AI-generated or otherwise fabricated citations.
The lesson is simple: AI can help with legal research, but it cannot take responsibility for the research you put before a judge.
Key Takeaways
AI hallucinations can produce fake cases, inaccurate quotations, and incorrect legal propositions.
A citation that looks real may still be completely fabricated.
Lawyers and self-represented litigants remain responsible for the documents they submit.
Courts expect legal authorities to be genuine, relevant, and properly verified.
Simply saying "AI gave me the citation" is not a reliable defense.
AI can be extremely useful for organization and drafting when combined with independent legal research.
Every case, statute, regulation, court rule, and quotation should be checked before it is used in a court filing.
What Is an AI Hallucination?
An AI hallucination occurs when an artificial intelligence system produces information that appears convincing but is inaccurate, unsupported, or completely fabricated.
The unusual part is that the answer may not look wrong.
An AI-generated case citation might contain:
A believable case name
A realistic-looking citation
The name of an actual court
A plausible year
A detailed legal holding
A quotation that sounds exactly like judicial language
Yet the case may not exist at all.
Or the case may exist, but the quotation may not appear anywhere in the opinion.
That makes legal hallucinations particularly dangerous. A spelling mistake is usually obvious. A fictional case written in perfect legal language may not be.
Why Does AI Hallucinate Legal Information?
Generative AI systems produce text by identifying patterns and generating likely responses. They are not inherently performing the same type of source verification that a lawyer does when checking an opinion in an authoritative legal database.
That distinction matters.
When asked a question such as:
"Give me cases supporting this argument."
an AI system may produce an answer that looks like traditional legal research. But unless the underlying authorities are independently checked, there is no guarantee that every citation is genuine or that every case actually supports the stated proposition.
The problem becomes especially serious when a user assumes that a confident answer must be a verified answer.
It isn't.
Why Are AI Hallucinations So Dangerous in Court?
Court filings operate under a higher level of accountability than ordinary online writing.
A judge may rely on a lawyer's representation that a particular case says something. Opposing counsel may spend time investigating the authority. The court may have to determine whether the citation exists and whether it actually supports the argument.
A fabricated authority can therefore:
Waste the court's time
Increase litigation costs
Mislead the judge
Harm the client's case
Damage the lawyer's credibility
Delay the proceedings
Lead to sanctions or other disciplinary consequences
The Southern District of New York made this point forcefully in Mata v. Avianca, explaining that submitting fake opinions can waste the opposing party's resources and the court's time while undermining confidence in the judicial system.
The Case That Put AI Hallucinations on the Map: Mata v. Avianca
One of the most famous examples is Mata v. Avianca, Inc.
In that case, attorneys submitted a filing containing nonexistent judicial opinions and fake quotations generated through ChatGPT.
The court ultimately imposed sanctions.
What made the situation particularly important was not simply that ChatGPT had generated false authorities. The court focused on the lawyers' failure to perform the basic verification that should have happened before the material was filed.
The court explained that using technology to assist legal research is not inherently improper. Lawyers can use tools—including databases and other forms of assistance—but they still have a gatekeeping responsibility to make sure their filings are accurate.
That distinction remains extremely important today.
The problem isn't necessarily using AI. The problem is trusting unverified AI output.
Have Lawyers Continued to Face Sanctions?
Yes.
The issue did not end with Mata.
In Gauthier v. Goodyear Tire & Rubber Co., for example, counsel submitted a brief containing nonexistent cases and quotations that could not be located. The court addressed the lawyer's obligations under Federal Rule of Civil Procedure 11 and an applicable local rule concerning AI-generated content. The lawyer was ordered to pay a $2,000 penalty and complete a one-hour continuing legal education program concerning AI.
Other courts have continued to encounter similar problems.
The American Bar Association reported in 2026 that sanctions involving AI-hallucinated authorities have included serious consequences such as monetary penalties, referral to disciplinary authorities, and other professional consequences.
So the warning is no longer theoretical.
Can AI Create a Fake Case That Looks Completely Real?
Yes.
That is one of the most troubling features of generative AI.
A fabricated citation can look something like this:
Fictional Case v. Example, 123 F.4th 456 (9th Cir. 2025).
The citation format may look perfectly reasonable.
The supposed holding may also sound believable.
But when someone searches for the decision, there may be no such case.
That is why appearance is not verification.
A professional-looking citation is not proof that the authority exists.
What If the Case Is Real but the AI Gets It Wrong?
This is another common problem.
AI does not necessarily need to invent an entire case to create a legal problem.
It may take a real decision and:
Attribute the wrong holding to it
Combine language from different cases
Give the wrong procedural history
Misstate the court's reasoning
Quote language that does not appear in the opinion
Ignore an important limitation
Treat a dissent as the majority's holding
Apply a decision from the wrong jurisdiction
That can be just as dangerous as a completely fictional case.
So the verification question is not only:
"Does this case exist?"
It is also:
"Does this case actually say what I am claiming it says?"
Why Saying "ChatGPT Did It" Usually Isn't Enough
Imagine a lawyer files a brief containing five nonexistent cases.
The court asks where the cases came from.
The lawyer responds:
"ChatGPT generated them."
That explanation does not necessarily solve the problem.
The person who signs and submits the filing has responsibilities independent of the software used to prepare it.
In Mata, the court emphasized the attorney's gatekeeping responsibility. In later cases, courts have likewise focused on whether the attorney actually investigated and verified the authorities before filing.
The practical lesson is:
AI can generate the draft. The lawyer or litigant must verify the filing.
What About Rule 11?
For federal civil cases, Federal Rule of Civil Procedure 11 is particularly important.
Rule 11 requires attorneys and self-represented parties to make certain certifications when submitting covered documents to the court.
Among other things, legal contentions must be warranted by existing law or by a nonfrivolous argument for changing the law.
A nonexistent judicial opinion obviously creates a serious problem if it is presented as existing authority.
Courts have relied on Rule 11 when addressing filings containing fabricated legal authorities. Mata is one prominent example, and later decisions have continued to apply similar principles.
What Can Happen When Fake Citations Are Filed?
The consequences depend on the circumstances and the applicable rules.
Possible consequences may include:
Monetary sanctions
Orders requiring corrective action
Payment of an opposing party's expenses
Referral to disciplinary authorities
Restrictions on appearing before a particular court
Damage to professional reputation
Additional scrutiny of future filings
The seriousness of the consequences can depend on factors such as the number of fabricated authorities, whether the conduct was corrected, whether the lawyer acted in bad faith, and whether the filing violated applicable rules.
Recent cases demonstrate that courts are increasingly unwilling to treat fabricated authority as a harmless drafting mistake.
What About Self-Represented Litigants?
The risk is not limited to lawyers.
People representing themselves in court increasingly use AI to understand legal procedures and prepare documents.
That can be helpful.
But a self-represented litigant still needs to verify the authorities included in a filing.
A person appearing pro se should be especially careful with:
Case citations
Statutes
Regulations
Court rules
Deadlines
Procedural requirements
Quotations from judicial opinions
Using AI does not turn an unverified statement into a legally reliable statement.
AI Hallucination vs. Verified Legal Research
| AI-generated information | Verified legal research |
|---|---|
| May contain fabricated cases | Uses genuine authorities |
| May misquote opinions | Quotations are checked against the original |
| May rely on outdated law | Current authority is confirmed |
| May misunderstand a holding | The actual opinion is reviewed |
| Useful for research leads | Suitable for supporting legal arguments after verification |
| Requires human checking | Supported by independent source verification |
The key difference is verification.
AI output is a starting point. Verified legal authority is something you can actually trace back to the source.
How to Check an AI-Generated Case
If ChatGPT gives you a case citation, don't immediately put it into your brief.
Use this process instead.
Step 1: Search for the Case
Look up the exact case name and citation using a reliable legal research service or official court source.
Step 2: Confirm the Court
Make sure the decision actually came from the court identified in the citation.
Step 3: Confirm the Date
Check the actual date of the decision.
Step 4: Read the Opinion
Do not stop after finding the case.
Read the relevant portions of the actual opinion.
Step 5: Check the Quotation
If AI provides quotation marks, compare the quotation word-for-word against the original.
Step 6: Confirm the Holding
Ask whether the court actually decided the legal issue that AI says it decided.
Step 7: Check Whether It Is Still Good Law
A case may have been overruled, limited, vacated, superseded, or otherwise affected by later authority.
Step 8: Check Jurisdiction
A decision from one jurisdiction may not control a court in another.
This process may take more time than simply trusting an AI response—but that is exactly the point.
How Lawyers Can Use AI More Safely
AI does not have to be abandoned.
It can be extremely useful when used within a careful workflow.
Use AI to Brainstorm
Ask it to identify legal issues or potential research questions.
Use AI to Organize
Give it verified information and ask it to create an outline.
Use AI to Improve Writing
Once the legal analysis is established, AI can help make the language clearer and more readable.
Use AI to Summarize
AI can help create preliminary summaries of lengthy documents, but the original documents should still be reviewed.
Use AI to Find Research Leads
Treat suggested cases as leads to investigate—not authorities to cite automatically.
What You Should Never Do
Avoid these shortcuts:
Copying AI-generated citations directly into a brief
Assuming a citation exists because it looks realistic
Filing an AI-generated document without reading it
Treating AI summaries as substitutes for judicial opinions
Asking AI to invent supporting authority
Using outdated legal information without checking it
Assuming another court's AI policy applies to your case
Giving AI unnecessary confidential information
The biggest mistake is not using AI.
The biggest mistake is using AI without verification.
Can AI Be Trusted for Legal Research?
AI can be useful for preliminary legal research, but it should not be treated as the final source of authority.
A good workflow looks like this:
AI → research idea → authoritative source → original opinion/statute/rule → human verification → final legal argument
Not:
AI → copy → paste → file
That difference can protect both the quality of the legal argument and the integrity of the court filing.
Best Practices Checklist
Before filing an AI-assisted legal document, ask:
Is every case real?
Did I read every case I cited?
Does every quotation appear in the original source?
Does each authority actually support my argument?
Are the statutes current?
Are the regulations current?
Are the court rules current?
Is the cited authority binding or merely persuasive?
Is the authority from the correct jurisdiction?
Did AI accidentally add facts that are not in the record?
Did I remove unsupported statements?
Did I check the court's local rules?
Did I review the final document myself?
If you cannot answer these questions confidently, the document is not ready to file.
Why Human Judgment Still Matters
AI is very good at producing language.
Legal practice requires something more.
A lawyer must decide:
Which facts matter
Which facts are supported by evidence
Which legal authority applies
Whether an argument is legally defensible
How a particular court may interpret the law
Whether a filing complies with procedural requirements
Whether a statement should actually be made to the court
Those are judgment calls.
AI can assist with the process, but it should not quietly make those decisions without human review.
Frequently Asked Questions
What is an AI hallucination in law?
It is inaccurate or fabricated information generated by an AI system, such as a nonexistent case, incorrect quotation, or misleading description of a legal rule.
Can ChatGPT invent court cases?
Yes. Generative AI can produce realistic-looking but nonexistent case names and citations. This is why every AI-generated legal citation must be independently verified.
Why have lawyers been sanctioned for AI-generated citations?
Courts have sanctioned lawyers when they filed fabricated or inaccurate authorities without adequately verifying them. The issue is not simply AI use; it is the failure to meet the lawyer's responsibility for the accuracy of the filing.
Is using AI for legal research illegal?
No. Using AI for legal research is not, by itself, generally prohibited. However, lawyers and litigants must comply with applicable professional, procedural, and court-specific requirements.
Can AI summarize a Supreme Court case?
Yes, but the summary should be checked against the actual opinion before the case is relied upon in legal work.
Should I cite a case that ChatGPT gives me?
Not until you verify it.
Find the actual decision, read it, confirm the quotation and holding, and make sure it applies to your jurisdiction and legal issue.
Can self-represented litigants use AI?
Yes, generally. AI can help with organization and drafting, but self-represented litigants remain responsible for the accuracy of their filings.
Should lawyers stop using AI?
Not necessarily. AI can be a useful productivity tool. The safer approach is to combine AI assistance with independent research, human review, confidentiality safeguards, and professional judgment.
Conclusion
Disclaimer
This article is provided for educational and informational purposes only and does not constitute legal advice. Federal and state laws, court procedures, local rules, professional responsibilities, and judicial policies concerning artificial intelligence may vary and can change over time. Readers should consult the rules applicable to their particular court and seek advice from a qualified attorney regarding specific legal matters.
Comments
Post a Comment
Comment should be in the form of message and helpful to the viewers