Sullivan & Cromwell apologizes after AI hallucinations generate fake citations in court filing

Top Law Firm Apologizes to Bankruptcy Judge for AI Hallucination

One of Wall Street's prominent law firms, Sullivan & Cromwell, wrote to a bankruptcy judge to apologize for a court motion that included inaccurate citations generated by artificial intelligence, according to a filing in the US Bankruptcy Court for the Southern District of New York. In the April 18 apology, Andrew Dietderich, founder and co-head of Sullivan's restructuring group, said the firm had been made aware of errors in an emergency motion filed in the bankruptcy of Prince Global Holdings. "The inaccuracies and errors in the Motion include artificial intelligence ("AI") "hallucinations," according to the letter, which added that the firm had not followed its protocols in preparing the document. "We sincerely regret the errors in the Motion and the burden they have imposed on the Court and the parties, and I apologize on behalf of our entire team," Dietderich wrote in the letter. The firm said it is taking steps to ensure the accuracy of all submissions. The law firm represents liquidators overseeing actions against Prince Group, a Cambodia-based conglomerate. Messages left with Sullivan were not immediately returned. It is very rare for big law firms such as Sullivan & Cromwell to include AI-generated errors in a court filing, said Damien Charlotin, who oversees a database tracking court cases in which an AI hallucination has been verified by a judge or acknowledged by the lawyers involved. More often, the mistakes are made by solo practitioners in cases involving many parties, said Charlotin, who is also a senior research fellow at French business school HEC Paris. The number of such cases has grown in recent years as AI use has spread, he said. His database shows more than 900 US cases, only a handful of which are in bankruptcy court. The errors underscore growing concerns about how law firms are using AI and what safeguards they have in place. Judges have reprimanded lawyers in some cases. Last year, a bankruptcy judge publicly reprimanded a former Gordon Rees Scully Mansukhani senior counsel for submitting filings with artificial-intelligence-generated fake citations, although the firm itself avoided court sanctions.

Sullivan & Cromwell law firm apologizes for AI 'hallucinations' in court filing

April 21 (Reuters) - Sullivan & Cromwell, a premier Wall Street law firm, apologized to a federal judge for submitting a court filing with inaccurate citations and other errors generated by artificial intelligence. In a letter dated April 18, Andrew Dietderich, co-head of the firm's global restructuring group, said the errors included AI "hallucinations" - instances in which AI makes up case citations, misquotes the law or generates non-existent legal sources. The mistakes were caught by law firm Boies Schiller Flexner, Dietderich said in the letter to Martin Glenn, chief judge of the U.S. ⁠Bankruptcy Court in Manhattan. "I apologize on behalf of our entire team. I also called Boies Schiller Flexner LLP on Friday to thank them for bringing this matter to our attention and to apologize directly to them as well," Dietderich wrote. Boies Schiller Flexner is also involved in the case. The letter did not say what AI program was used to help produce the court filing. Dietderich and a representative of the firm did not immediately respond to requests for comment. AI POLICIES NOT FOLLOWED The firm told the judge it maintains "comprehensive policies and training requirements governing the use of AI tools in legal work" that are designed to minimize errors. The letter said those AI policies were not followed and that a ⁠secondary review process also "did not identify the inaccurate citations generated by AI." It later filed a corrected version. U.S. judges have sanctioned lawyers in dozens of cases after attorneys used AI for legal research and drafting without fully vetting the results. Lawyers are not prohibited from using AI but are ethically bound to ensure the accuracy of court submissions. New York-based Sullivan & Cromwell, with more than 900 lawyers, ⁠has a reputation as one of the country's top corporate firms, known for its mergers and acquisitions work, corporate governance litigation and private equity matters. In the New York case, the firm represents foreign representatives involved in the wind-down of Prince Global Holdings Limited, a ⁠Cambodian conglomerate whose founder and chairman Chen Zhi was charged in Brooklyn federal court for allegedly directing forced labor compounds in Cambodia and a massive investment fraud. A representative from Prince Group could not be immediately reached for comment. Prince ⁠Group in a statement last year denied that Zhi had committed wrongdoing, and called the allegations baseless. Boies Schiller represents a group of objecting debtors in the case. A Boies Schiller attorney in the proceedings did not immediately respond to a request for comment. Reporting by Karen Freifeld and Mike Scarcella; Editing by David Bario and Tomasz Janowski Our Standards: The Thomson Reuters Trust Principles., opens new tab

Elite law firm Sullivan & Cromwell admits to AI 'hallucinations'

Sullivan & Cromwell told a US federal bankruptcy court that a major filing it made in a high-profile case contained multiple "hallucinations" made by AI software. Andrew Dietderich, the head of S&C's restructuring practice, apologised in a letter to New York federal judge Martin Glenn on Saturday for mistakes that included misquoting the US bankruptcy code and citing cases incorrectly in a court filing made on April 9. "We deeply regret that this has occurred," he said in the letter. Dietderich said the firm's policies on the use of AI had not been followed when the document was prepared, and it was considering whether it needed to make "further enhancements" to its internal training and review processes. The letter did not say which lawyers prepared the documents or whether they were still at the firm. S&C declined to comment. The errors are the latest example of a professional services firm grappling with the use of cutting-edge technology to speed up laborious research and cut down on staffing while also trying to maintain quality standards. The case in question revolves around S&C's representation of liquidators appointed by legal authorities in the British Virgin Islands who are pursuing actions against Prince Group and its owner Chen Zhi. US federal prosecutors last year charged Zhi with wire fraud and money laundering, accusing him of "directing Prince Group's operation of forced-labour scam compounds across Cambodia . . . that stole billions of dollars from victims in the United States and around the world". In a separate action, US prosecutors also filed a civil forfeiture complaint seeking to seize nearly $9bn worth of bitcoin that the US authorities said represented the proceeds of the Prince Group crimes. Zhi was arrested earlier this year in Cambodia and extradited to China after a request from Beijing. Prince Group is incorporated in the British Virgin Islands and the Chapter 15 proceeding in the US court system is designed to get the US government to formally recognise the powers of the BVI liquidators to represent creditors and victims in the US legal proceedings, liquidators told the court. In multiple instances, S&C in the April 9 filing erroneously summarised the conclusions made in other cases, according to a list of strike-through corrections the firm submitted to the judge. S&C has an enterprise licence for ChatGPT according to multiple people familiar with the firm's operations. According to S&C's website, at least five high-level partners have been assigned to the Prince Group bankruptcy case. The firm's partners typically charge more than $2,000 per hour in bankruptcy cases. The firm earned several hundred million dollars in fees in representing crypto exchange FTX in its bankruptcy liquidation. Boies Schiller Flexner, the law firm representing Prince and Zhi, spotted the errors in S&C's filing. In a document filed last week, BSF said words that S&C had quoted in its motion "do not appear in chapter 15 of the US Bankruptcy Code" and pointed to "multiple cited decisions" that were "misquoted or misidentified". It said a case cited by S&C in the motion "is not a case" and the reference was to "a different decision in a different circuit". S&C told the court that the firm maintained "rigorous" standards when using AI tools and that it "instructs lawyers to 'trust nothing and verify everything'". Failure to verify AI-generated output "constitutes a violation of firm policy", it said. It is the latest in a series of errors by law firms using AI tools. Last year, Latham & Watkins admitted that one of its lawyers had used Anthropic's Claude model to help draft a filing which contained an apocryphal title and author for a journal article. In another instance, a federal appeals court in New Orleans ordered a $2,500 sanction against a lawyer who had submitted a brief with 21 errors or fabrications that had been inserted by AI. Separately, in September John Kucera, then a partner at BSF, said in a case against Amazon that a document for which he was responsible, prepared using AI tools, contained "material citation errors" due to his "failure to verify" details. "I am embarrassed by and very much regret these errors", he said in the filing. BSF did not respond to a request for comment. S&C told the judge overseeing the Prince Group case that its document review also showed "non-substantive and/or clerical errors in other filings in this matter". The firm said those errors were made by humans, not AI.

A.I. 'Hallucinations' Created Errors in Court Filing, Top Law Firm Says

Sullivan & Cromwell apologized for submitting a court document that had fake citations created by artificial intelligence. An elite Wall Street law firm has apologized to a federal judge for submitting a court filing replete with errors created by artificial intelligence, including "hallucinations" that fabricated case citations. The A.I.-generated errors came in a recent motion in U.S. Bankruptcy Court in Manhattan and were discovered by lawyers from an opposing firm, Andrew Dietderich, a partner at Sullivan & Cromwell, wrote in a letter to Judge Martin Glenn on April 18. "We deeply regret that this has occurred," Mr. Dietderich wrote. The firm provided a ledger of the errors, which spanned three pages and totaled around three dozen. A number of them involved the citation of seemingly imagined passages from real cases. Sullivan & Cromwell is one of the oldest and most prestigious law firms in the country. It is representing President Trump in several appeals, including his criminal conviction in 2024 in a case that stemmed from a hush-money payment to a porn star. Jay Clayton, now the U.S. attorney for the Southern District of New York, was of counsel and formerly a partner at the firm. The apology revealed the latest embarrassing blunder for lawyers found to have used A.I. in crafting erroneous arguments. The legal profession is undergoing a reckoning over the growing and widespread use of A.I., which is luring lawyers dealing with voluminous research even as it has a propensity to spit out legal falsehoods. A spate of cases in recent years has illuminated the dangers that using A.I. poses to lawyers. In 2023, a federal judge in Manhattan fined two lawyers $5,000 after they submitted a brief of made-up cases, concocted by ChatGPT. The American Bar Association has instructed lawyers to exercise caution when posing prompts to A.I. models or retrieving results. Mr. Dietderich wrote in his letter that the firm's policies governing the use of A.I. were "not followed" in preparing the motion. It is not clear which A.I. tools or program were used by Sullivan & Cromwell in generating the errors. A spokesman for the firm declined to comment. The news of the letter was reported earlier by Reuters. The hallucinations filed by Sullivan & Cromwell came about in a case involving the Prince Group, a Cambodian conglomerate whose founder, Chen Zhi, was indicted in Federal District Court in Brooklyn last year on charges that he operated a global scam operation. His lawyers and representatives have denied the charges. Mr. Chen, who was not in the United States when the indictment was announced, was extradited from Cambodia to China in January. On April 8, the Prince Group, a Cambodian conglomerate with a number of business entities incorporated in the British Virgin Islands, filed for bankruptcy in Manhattan. Sullivan & Cromwell is representing a group of people appointed by the authorities in the British Virgin Islands to oversee the Prince Group's liquidated assets in that territory. Some of the errors were identified by lawyers from Boies Schiller Flexner, the law firm representing the Prince Group, in a public filing. A spokesman for the firm declined to comment. After learning of the errors, Mr. Dietderich wrote, the firm conducted a review of all other filings in the case. The A.I. hallucinations were contained to the single filing, he wrote. According to Mr. Dietderich's letter, Sullivan & Cromwell requires its lawyers to take a training course before it gains access to A.I. tools. Among the training's exhortations, Mr. Dietderich wrote, is to "trust nothing and verify everything."

AI hallucinations found in high-profile Wall Street law firm filing

Sullivan & Cromwell apologises to New York federal judge for string of errors in documents for Prince Group case The elite Wall Street law firm Sullivan & Cromwell has told a court that a major filing it made in a high-profile case contained errors resulting from hallucinations generated by artificial intelligence. Andrew Dietderich, the co-head of the firm's global restructuring group, apologised in a letter to the New York federal judge Martin Glenn on Saturday for the string of mistakes which included inaccurate citations. The errors, uncovered by the law firm Boies Schiller Flexner (BSF), which was also working on the case, included misquoting the US bankruptcy code and citing cases incorrectly in a filing made on 9 April. In multiple instances, S&C, which employs more than 900 lawyers and has one of the top reputations for corporate work in the US, filed inaccurately summarised conclusions made in other cases using AI. "We deeply regret that this has occurred," said Dietderich in the letter. "I apologise on behalf of our entire team. I also called BSF on Friday to thank them for bringing this matter to our attention and apologise to them directly as well." The firm said that it maintains "comprehensive policies and training requirements governing the use of AI tools in legal work" that are designed to catch any potential errors. However, the letter said those AI policies were not followed and that a secondary review process also "did not identify the inaccurate citations generated by AI". S&C later filed a corrected version to the court. Lawyers are not prohibited from using AI but are ethically bound to ensure the accuracy of court submissions. The letter did not say which AI program was used to help produce the court filing, or which lawyers prepared the document, and whether any action had been taken against them as a result of the AI mistakes. The case involved S&C's representation of liquidators appointed by legal authorities in the British Virgin Islands who are engaged in actions against Prince Group, which is owned by the Chinese-born businessman Chen Zhi. Last year, US prosecutors charged Chen with wire fraud and money laundering, alleging that he directed "Prince Group's operation of forced-labour scam compounds across Cambodia ... that stole billions of dollars from victims in the United States and around the world". In a statement last year, Prince Group denied that Chen had committed wrongdoing and called the allegations baseless. Separately, US prosecutors also filed a legal action to seize nearly $9bn of bitcoin that US authorities alleged represented the proceeds of the Prince Group's criminal activity. Chen was arrested earlier this year in Cambodia and extradited to China upon the request of Chinese authorities.

Top Law Firm Admits to AI 'Hallucinations' in Bankruptcy Filing Tied to Alleged Scam Network - Decrypt

The case involves efforts by court-appointed liquidators to pursue claims linked to sanctioned outfit Prince Group. Law firm Sullivan & Cromwell has admitted to a U.S. bankruptcy court that a recent filing in a high-profile case contained errors generated by artificial intelligence, including fabricated citations. "We deeply regret that this has occurred," Andrew Dietderich, the firm's restructuring head, wrote to Judge Martin Glenn, saying the document included AI "hallucinations" that produced fictitious authorities and distorted existing ones. The disclosure came in a letter to the U.S. Bankruptcy Court for the Southern District of New York, where the firm represents court-appointed liquidators from the British Virgin Islands. The mistakes appeared in an April 9 motion and the firm said its rules on AI use were not followed during preparation. The case involves efforts by those liquidators to pursue claims tied to Prince Group and its owner, Chen Zhi. Prosecutors allege Chen directed scam compounds that targeted victims worldwide and have sought to recover billions of dollars in cryptocurrency they say is linked to the activity. He was detained earlier this year in Cambodia and later repatriated to China. Through Chapter 15 proceedings in the U.S., the liquidators are seeking recognition of their authority to act on behalf of creditors and alleged victims. Prince Group, incorporated in the British Virgin Islands, has been linked by U.S. authorities to large-scale fraud operations in Southeast Asia and sanctioned by the UK and U.S. governments. According to a corrected submission, the April filing misstated case law in multiple places and included citations that did not support the propositions attributed to them, while some appeared to have no basis at all. The firm withdrew the original motion and has filed a revised version. Lawyers for Prince Group and Chen at Boies Schiller Flexner initially identified the errors. They said language attributed to the U.S. Bankruptcy Code could not be found and that several authorities were mischaracterized or misidentified. In one instance, they said, a cited case referred to a different decision in another circuit. In a separate filing, defendants said at least 28 citations were erroneous, including quotations attributed to the court that do not exist. They argued the timing of the correction was prejudicial because the revised filing came after they had submitted objections, and asked the court to adjourn a scheduled hearing and hold a status conference. Sullivan & Cromwell said its policies require lawyers to complete training before using AI tools and to independently verify all output. "Before any Firm lawyer is granted access to generative AI tools, the lawyer must complete two required training modules, completion of which is tracked and verified. The training repeatedly emphasizes the risk of AI 'hallucinations,' including the fabrication of case citations, misinterpretation of authorities, and inaccurate quotations," it said. "It instructs lawyers to 'trust nothing and verify everything' and makes clear that failure to independently verify AI-generated output constitutes a violation of Firm policy." The firm said a broader review found additional minor drafting issues in other filings, which it attributed to human error rather than AI. It did not identify the lawyers who prepared the original motion. The incident adds to a growing list of AI-related missteps in legal practice as firms test tools designed to speed research and drafting. Courts have recently sanctioned or criticized lawyers for submitting filings with fabricated or inaccurate references produced by AI. In Australia, one lawyer was stripped of their ability to practise as a principal lawyer due to AI use last year. Law schools are beginning to require instruction on the technology, while senior judges have warned that misuse could affect the integrity of proceedings. Recent rulings have also addressed how AI fits within existing legal frameworks, including whether interactions with such tools are protected by privilege. At the same time, some courts are piloting AI systems to help manage heavy caseloads.

US Law Firm Apologizes For AI Hallucinations in Filing

Sullivan & Cromwell's Andrew Dietderich said the company has AI policies to prevent incorrect citations and other errors, but procedures weren't followed on this occasion. Wall Street law firm Sullivan & Cromwell has apologized to a federal judge after submitting a court filing that contained around 40 incorrect citations and other errors caused by AI hallucinations. "We deeply regret that this has occurred," Andrew Dietderich, co-head of Sullivan & Cromwell's global restructuring team, wrote Friday in a letter to Chief Judge Martin Glenn of the US Bankruptcy Court for the Southern District of New York. "The Firm and I are keenly aware of our responsibility to ensure the accuracy of all submissions including under Local Bankruptcy Rule 9011-1(d), and I take responsibility for the failure to do so," he said of an emergency motion filed nine days earlier. The incident highlights the risk AI tools can pose in high-stakes professional work without proper oversight. A database managed by legal technologist Damien Charlotin has recorded 1,334 incidents of AI hallucinations in court filings around the world, including more than 900 in the US. Charlotin pointed out that most of these hallucinations involve fabricated citations, though AI-generated legal arguments have also occasionally been identified. Dietderich said Sullivan & Cromwell has policies in place for the use of AI tools, which include a review of the citations it uses, but said the policies weren't followed. "Regrettably, this review process did not identify the inaccurate citations generated by AI, nor did it identify other errors that appear to have resulted in whole or in part from manual error." Sullivan & Cromwell is one of the largest law firms in the US by revenue, ranking 30th on the AmLaw Global 200. The firm also represented crypto exchange FTX in its bankruptcy case. Sullivan & Cromwell is conducting an internal investigation Dietderich said the law firm took "immediate remedial measures," including a full review of the circumstances that led to the errors. The firm is also "evaluating whether further enhancements to its internal training and review processes are warranted," Dietderich said. Dietderich also noted that the errors were spotted by a rival law firm. "I also called Boies Schiller Flexner LLP on Friday to thank them for bringing this matter to our attention and to apologize directly to them as well," he said.

Sullivan & Cromwell law firm apologizes for AI 'hallucinations' in court filing

April 21 (Reuters) - Sullivan & Cromwell, a premier Wall Street law firm, apologized to a federal judge for submitting a court filing with inaccurate citations and other errors generated by artificial intelligence. In a letter dated April 18, Andrew Dietderich, co-head of the firm's global restructuring group, said the errors included AI "hallucinations" - instances in which AI makes up case citations, misquotes the law or generates non-existent legal sources. The mistakes were caught by law firm Boies Schiller Flexner, Dietderich said in the letter to Martin Glenn, chief judge of the U.S. Bankruptcy Court in Manhattan. "I apologize on behalf of our entire team. I also called Boies Schiller Flexner LLP on Friday to thank them for bringing this matter to our attention and to apologize directly to them as well," Dietderich wrote. Boies Schiller Flexner is also involved in the case. The letter did not say what AI program was used to help produce the court filing. Dietderich and a representative of the firm did not immediately respond to requests for comment. AI POLICIES NOT FOLLOWED The firm told the judge it maintains "comprehensive policies and training requirements governing the use of AI tools in legal work" that are designed to minimize errors. The letter said those AI policies were not followed and that a secondary review process also "did not identify the inaccurate citations generated by AI." It later filed a corrected version. U.S. judges have sanctioned lawyers in dozens of cases after attorneys used AI for legal research and drafting without fully vetting the results. Lawyers are not prohibited from using AI but are ethically bound to ensure the accuracy of court submissions. New York-based Sullivan & Cromwell, with more than 900 lawyers, has a reputation as one of the country's top corporate firms, known for its mergers and acquisitions work, corporate governance litigation and private equity matters. In the New York case, the firm represents foreign representatives involved in the wind-down of Prince Global Holdings Limited, a Cambodian conglomerate whose founder and chairman Chen Zhi was charged in Brooklyn federal court for allegedly directing forced labor compounds in Cambodia and a massive investment fraud. A representative from Prince Group could not be immediately reached for comment. Prince Group in a statement last year denied that Zhi had committed wrongdoing, and called the allegations baseless. Boies Schiller represents a group of objecting debtors in the case. A Boies Schiller attorney in the proceedings did not immediately respond to a request for comment. (Reporting by Karen Freifeld and Mike Scarcella; Editing by David Bario and Tomasz Janowski)