MIT Retracts Controversial AI Productivity Study Amid Data Integrity Concerns

MIT disavows doctoral student paper on AI's productivity benefits | TechCrunch

MIT says that due to concerns about the "integrity" of a high-profile paper on the effects of artificial intelligence on the productivity of a materials science lab, the paper should be "withdrawn from public discourse." The paper in question, "Artificial Intelligence, Scientific Discovery, and Product Innovation," was written by a doctoral student in the university's economics program. It claimed to show that the introduction of an AI tool into a large-but-unidentified materials science lab led to the discovery of more materials and more patent filings, but at the cost of reducing researchers' satisfaction with their work. MIT economists Daron Acemoglu (who recently won the Nobel Prize) and David Autor both praised the paper last year, with Autor telling the Wall Street Journal he was "floored." In a statement included in MIT's announcement on Friday, Acemoglu and Autor described the paper as "already known and discussed extensively in the literature on AI and science, even though it has not been published in any refereed journal." However, the two economists said they now have "no confidence in the provenance, reliability or validity of the data and in the veracity of the research." According to the WSJ, a computer scientist with experience in materials science approached Acemoglu and Autor with concerns in January. They brought those concerns to MIT, leading to an internal review. MIT says that due to student privacy laws, it cannot disclose the results of that review, but the paper's author is "no longer at MIT." And while the university's announcement does not name the student, both a preprint version of the paper and the initial press coverage identify the author as Aidan Toner-Rodgers. (TechCrunch has reached out to Toner-Rodgers for comment.) MIT also says it has requested the paper be withdrawn from The Quarterly Journal of Economics, where it was submitted for publication, and from the preprint website arXiv. Apparently only a paper's authors are able to submit arXiv withdrawal requests, but MIT says "to date, the author has not done so."

MIT Backs Away From Paper Claiming Scientists Make More Discoveries with AI

The retracted paper had impressed a Nobel Prize winner in economics. Last year, the Massachusetts Institute of Technology was touting the research of a PhD student on the impact of AI on the workforce that "floored" professors in the field. Now the university is backing away from it and calling for it to no longer be published. On Friday, MIT announced that it reviewed the paper following concerns and determined that it should be "withdrawn from public discourse." The paper, titled "Artificial Intelligence, Scientific Discovery, and Product Innovation" snagged all sorts of attention and headlines for its finding that scientists aided by AI tools were considerably more productive than their peers working without the technological aidâ€"but those same researchers making more discoveries were significantly less satisfied by their work. The work was considered a breakthrough, and Daron Acemoglu, a Massachusetts Institute of Technology professor who recently won the Nobel Prize in economics, described it as "fantastic." But the findings didn't quite sit right with some. According to the Wall Street Journal, a computer scientist with experience in materials science approached MIT professors with questions about how the AI tool used in the experiment worked and just how big of a boost in innovation that it was actually responsible for. The professors took those concerns to the university, which started a review process that ultimately led to MIT stating that it "has no confidence in the provenance, reliability or validity of the data and has no confidence in the veracity of the research contained in the paper.†The institution didn't expand on what exactly was wrong with the paper, citing "student privacy laws and MIT policy." But the researcher responsible for the paper is no longer affiliated with the university, and MIT has called for the paper to be pulled from the preprint site arXiv. It has also withdrawn the paper from consideration by the Quarterly Journal of Economics, where it had been submitted for evaluation and eventual publication. David Autor, an MIT economist who touted the paper, told WSJ, "More than just embarrassing, it’s heartbreaking." It's also a major blow to research on AI in the workforce. The paper seemed to suggest that researchers were making many more discoveries when aided by AI, suggesting that there may be a boom in scientific breakthroughs on the horizon. Now there's doubt around just how much of that was genuine, and just how much we can learn from how the introduction of AI affects the people using these tools.