AI-Generated Legal Citations Lead to Sanctions and Raise Concerns in Courts

AI-Generated Citations Fool Judge, Leading to Sanctions

In a startling incident that highlights the growing concerns surrounding artificial intelligence (AI) in the legal system, a California judge has imposed $31,000 in sanctions on two law firms for submitting a brief containing "bogus AI-generated research" 1

1

. Judge Michael Wilner, serving as a special master in the US District Court for the Central District of California, admitted that he was initially persuaded by the fake citations and nearly included them in a judicial order.

The Incident Unfolds

The case involved a civil lawsuit against State Farm, where attorneys from K&L Gates and Ellis George LLP represented the plaintiff. Trent Copeland, an attorney at Ellis George, used AI tools including Google Gemini and law-specific AI models to generate an outline for a supplemental brief 2

2

. This outline, containing problematic legal research, was then sent to K&L Gates, who incorporated the material into the brief without proper verification.

Extent of the AI-Generated Errors

Judge Wilner's review revealed that approximately nine out of 27 legal citations in the ten-page brief were incorrect, with at least two cited authorities not existing at all 3

3

. Additionally, several quotations attributed to judicial opinions were fabricated and did not accurately represent the cited materials.

Judge's Response and Sanctions

In his ruling, Judge Wilner expressed grave concern over the incident, stating, "Directly put, Plaintiff's use of AI affirmatively misled me" 1

1

. He emphasized the need for strong deterrence to prevent attorneys from succumbing to the "easy shortcut" of using AI without proper verification.

Broader Implications for the Legal System

This incident is not isolated. Similar cases have been reported in other jurisdictions, including one involving Anthropic in a copyright lawsuit and another in Israel where prosecutors cited non-existent laws 2

2

. Legal scholar Maura Grossman notes that these AI-induced errors seem to be increasing, potentially compromising the integrity of court proceedings.

The AI Dilemma in Law

The incident highlights a growing divide among lawyers regarding AI use. While some are cautious about adopting the technology, others, particularly those under time constraints, are eager to leverage AI for assistance 2

2

. However, the risks of unchecked AI use in legal documents are becoming increasingly apparent.

Call for Vigilance and Responsibility

Judge Wilner's ruling serves as a stark reminder of the importance of human oversight in AI-assisted legal work. He emphasized that "no reasonably competent attorney should out-source research and writing to this technology -- particularly without any attempt to verify the accuracy of that material" 4

4

. The incident underscores the need for clear guidelines and ethical standards for AI use in the legal profession to maintain the integrity of the judicial system.