Substack launches AI detection tool to fight 'Claudefishing' and restore trust in newsletters

Substack's new tool tells you who's been writing their newsletters with AI

Substack has launched a new feature that can show you which of your favorite newsletters are being written using AI. This week, the newsletter and writing platform announced an integration with the AI writing detection software Pangram that will allow users to scan posts, comments, and replies on Substack's app to see an estimate of how much of the content was written by a human and how much was AI. In the short term, the move might be bad for Substack's business, as it could expose many of the newsletters on its platform that aren't entirely written by people. That could potentially erode trust in the platform's ecosystem of independent news and blogs, or even damage its reputation as a host of high-quality content. But in the long term, AI detection features could help keep Substack free of "AI slop," and encourage more users to trust what they're reading was written by a person, or at least better understand when it's not. Substack joins several platforms that are leaning towards labeling AI content as such, especially now that AI is playing a greater role in the creation process. Photos and videos generated with AI are labeled as such on social media sites, while music streaming services have more recently begun labeling and, in some cases, penalizing AI-generated music. "This is good use of AI," Substack CEO Chris Best said. "When I used to pitch Substack to writers, one way I would do it is [...] we'll do everything for you except the hard part," he explained in an online chat with Pangram's founder, Max Spero. "You have to have something -- an idea that's worth reading, that's worth caring about, that's worth sharing. That one thing is very hard and very valuable [...] software should do everything else, but I think you do want the person to do the hard part." The feature will be available in Substack's app for any post, note, reply, or comment above 100 characters. Substack will also allow its writers to include an optional AI Author's note, using which creators can properly disclose their use of AI, the company told TechCrunch. The company clarified that the tool is not meant to prohibit or penalize AI-assisted writing, but rather to encourage writers to add a "how I make this" statement, where they explain their process. Publishers can also run Pangram on their own drafts before publication, and report and remove scans on their own work they believe are mistakes.

I Test-Drove Substack's New AI Detection Tool, and It Mostly Worked - CNET

Ever since being admittedly fascinated by the Cambridge coffee webcam from the 1990s, I've written about VPNs, the NFL, smartphones, living wages, over/unders and everything in between. Read full bio More and more, we want to know: Is it human, or is it AI? Is that image real? That song? That video? The publishing platform Substack is trying to add some clarity by partnering with the AI detector Pangram to help readers understand just how much AI is in all the content they consume on the publishing platform. The tool is available to all Substack subscribers and works on posts, notes, comments and replies published on or after Tuesday that are longer than 100 words. Creators will also be able to scan their drafts with Pangram to see how much AI is detected and can also provide a statement to explain their writing process, Substack said. Web and iOS users can try the Pangram features now, and Android users will get access later. In a blog post announcing the new feature, Substack co-founder and CEO Chris Best said his platform aims to avoid "Claudefishing," meaning when a reader consumes content without realizing it was generated by AI. "When readers have to wonder if what they're reading is real, it undermines trust in authorship and threatens the livelihood of writers -- including those who use AI tools thoughtfully to produce work they believe in," Best wrote. From CNET: AI Slop Is Destroying the Internet. These Are the People Fighting to Save It Pangram estimates that roughly 40% of text on some social media platforms is AI-generated, with a stunning two-thirds of that on the networking site LinkedIn. And AI growth agency Graphite recently published a white paper saying that AI is now writing as many online articles as humans. A Gartner survey found that 68% of people wonder if the content they see is real, and that half of consumers prefer brands that aren't using AI in their marketing. Testing it for myself I've only recently joined Substack, but I wanted to check out the new Pangram AI detection tool. The tool works on posts, notes, comments and replies, so I decided to use posts as my makeshift testing lab. First, I logged in to my Substack account and published a new post. I copied and pasted text from an article I had written -- by myself, no AI -- into the Substack post and published it. After a few minutes, I clicked on the live post. On the three-dot menu at the upper right of the page, I saw a "Scan for AI text" option. I clicked on it, and a small window popped up accurately declaring that my article was "Fully Human-Written." A line appears below this that indicates the percentage of the text that is AI, AI-assisted, or human. I next took the same post but added a long, fully AI-generated paragraph that I pasted from ChatGPT. When I performed the "Scan for AI text," the tool reported that 10% of the article was written by AI, a roughly accurate estimate. I next published a fully AI-generated post, and the scan correctly identified it as 100% AI-generated. For another test, I added a couple of AI-generated sentences to a post that was otherwise purely human content, and the scan flagged it as 100% human-generated. Close, but not quite right. So, the Pangram tool worked pretty well for my crude experimentation. Although I was fairly satisfied with how well Substack's AI detection system worked on my own content, I was disappointed to learn that one of my favorite podcaster-bloggers apparently went full AI on his latest post. Other writers I checked out, however, used fully human-generated text. What about false positives? In his announcement, Best admitted that the AI detection tool is "not perfect," but cited independent research that found Pangram's software was 97.5% accurate in detecting fully generated AI text. The University of Maryland found that Pangram was 99.3% accurate in identifying "humanized AI-generated text," which basically means a human has rewritten or paraphrased AI content. Substack writers who do not use AI are understandably concerned about being incorrectly labeled as AI-assisted authors. As a result, the risk of false positives is a significant issue. Substack said creators can scan their content to see if it will be labeled as containing AI while it's still in draft form. If Pangram incorrectly identifies AI text, the creator can report the error to Substack and have the false scan removed. To promote greater transparency, publishers can now include a "How I make this" statement if they want readers to know how much, if any, AI they have used in their content. Best said Pangram can't detect if AI was used to gather information or as a source for written pieces. "We're not against people using AI to assist their work, and we think people should be free to choose which tools they use to express themselves," Best said. "But people should know what they're getting."

Substack adds an AI detector to help spot blogs written by no one

Substack will now help users determine whether what they're reading may have been written by AI. A new tool coming to the platform can scan posts, notes, replies, and comments to provide an estimate of how much text could be AI-generated, according to a blog post published on Tuesday. The tool is powered by an AI detection company called Pangram and is rolling out across the web and the iOS app, with an Android launch coming "soon." Readers can analyze content longer than 100 words by choosing the "Scan for AI text" option from the three-dot menu in the top-right corner of a post. Substack co-founder and CEO Chris Best says the Pangram integration is meant to increase transparency on the platform as AI becomes more prevalent on social media. "The core problem is not people using AI, or the quality of its output," Best writes. "The problem is when there is a mismatch between a reader's expectation and reality, especially when they unwittingly invest their attention in something with no human thought on the other end. That's Claudefishing." In line with this change, Substack will also allow creators to explain their writing process to readers with a new "How I make this" statement. Writers can scan their drafts with Pangram, too, and will have an option to report inaccurate results. Best notes that "Pangram can only detect whether AI was used to make the text, not whether great human care went into creating it, nor whether AI tools were used as a source." But the goal, according to Best, is to make it easier for someone to choose to invest their time reading a blog post. "When readers have to wonder if what they're reading is real, it undermines trust in authorship and threatens the livelihood of writers -- including those who use AI tools thoughtfully to produce work they believe in," Best writes. "Platforms that reward fakeness will create a race to the bottom."

Substack is adding an AI detection feature - Engadget

Substack is adding an AI detection feature The company has partnered with Pangram for the new tools. Substack has launched a new AI detection tool in partnership with Pangram. This will allow readers to scan Substack content for an assessment of how much of the material was written by AI. The tool can be used on text longer than 100 words that was published beginning today. Substack is also adding a new statement space for creators to explicitly share if and how they used AI for their content. The AI detection capabilities are available starting today on web and iOS, with Android support to come. The blog post announcing the feature is surprisingly spicy. There's a dig at LinkedIn about the presence of AI-generated content on that service and it dubs attempting to create feigned human connection with AI slop "Claudefishing." Throwing shade is a risky maneuver here, because even the best tools for identifying gen-AI can't guarantee a correct assessment. The Atlantic dug deeper into just how accurate AI detection tools, Pangram in particular, can be. Spoiler: they're far from perfect. Substack did acknowledge in the post that there are limits to what Pangram can detect and hinted at some other features it is considering around AI content and preferences. It emphasized that these new measures are aimed at setting expectations for readers, summing up its stance as "people should know what they're getting."

Substack adds AI detection to fight 'Claudefishing'

The newsletter platform has partnered with the AI detector Pangram so readers can scan posts, notes, and comments for how much a human actually wrote. Its bet: in a feed full of slop, proof of a person behind the words is worth something. Substack has a name for the moment you realise a post you admired for its human voice actually came from a bot: Claudefishing. Now it is handing readers a button to catch it, powered by the AI detector Pangram. The newsletter platform has launched an AI-detection feature. In a pointed post titled "Against Claudefishing," chief executive Chris Best said readers can now scan posts, notes, replies, and comments. The tool estimates how much of the text a person wrote, and how much came with AI assistance. The scan works on text longer than 100 words, published from today onward, and the result shows only to the person who asks for it. It is live on the web and iOS, with Android to follow. What Claudefishing means Best is careful to say the problem is not AI itself. Substack uses AI to build software and product features, he noted, and plenty of writers lean on it thoughtfully. The problem is the mismatch: readers investing their attention in something with no human thought behind it. "Claude has a lot to offer," he wrote. "But when I want Claude's opinion, I'll ask Claude." The stakes are rising fast. Pangram estimates that as much as 40% of text on some social platforms is now AI-generated. Substack, which pitches itself as an economic engine for culture built on trust between people, wants to keep its feed from turning into that kind of slop. Not a ban The tools cut both ways. Writers can run the detector on their own drafts before publishing, add a "How I make this" statement to explain their process, and report any scan of their work they think is wrong. Substack frames the whole thing around a simple line: people should know what they are getting. It is an unusually direct move in a market drowning in AI-generated feeds. The post even took a swipe at LinkedIn, the platform many see as ground zero for machine-written earnestness. The catch The snag is that AI detection is not settled science. Pangram is accurate, but not perfect, and The Atlantic has warned that over-trusting it risks turning accusations into a witch hunt. Substack itself concedes the tool cannot tell whether real care went into a piece, only whether AI shaped the words. Critics go further. Any detector, the developer Perry Metzger argued, lets you fine-tune an AI to evade it, so such tools cannot work for long. The researcher Mor Naaman called detection a losing battle: if readers scan, writers will scan too, and route around it. As one Substack commenter put it, the presence of AI does not prove the absence of a human. A very human business Underneath sits a bet. As Business Insider's Peter Kafka put it, if AI wins, "made by humans" could be a very good business. Substack is wagering that proof a person did the work becomes the scarce, sellable thing. That the fight is playing out over newsletters, of all places, only shows how far the question now reaches.

Substack bets readers want to pay for content written by humans with new AI detection tool

Driving the news: The media platform is partnering with Pangram, an AI-detection tool, to allow users to estimate how much text on the platform -- including posts and comments -- was made with AI. How it works: Users on the iOS app or on a browser will be able to scan text in notes, replies, comments and posts and get an estimate of how much was human-written versus AI-generated. * It only works for text longer than 100 words and published after July 21, when the news was first announced. Substack said the feature is coming to Android soon. * Creators will also have access to Pangram to review their drafts as well as report and remove scans they believe are mistaken. * Substack is also adding a feature for creators to insert a "How I make this" statement, where they can disclose whether they use AI. Catch up quick: Substack is increasingly part of a comprehensive communications strategy for companies. * Many journalists who previously worked for larger outlets have found loyal, paying audiences on Substack, covering subjects like tech, venture capital and health. * The State Department, Tory Burch and a16z have turned to Substack for new owned communication channels. * With Pangram, readers can now run this content through leading AI detection tools. What they're saying: "When content made by no one takes over parts of the internet that are supposed to be human, it pollutes the commons and makes it hard to discover and hear human voices," wrote CEO Chris Best. * "When readers have to wonder if what they're reading is real, it undermines trust in authorship and threatens the livelihood of writers -- including those who use AI tools thoughtfully to produce work they believe in." The big picture: He argued the problem with "Claudefishing" is that readers are unwittingly investing time and attention in content that wasn't created by a human. * Best also ribbed LinkedIn, saying Substack doesn't want to turn into the professional network platform. * While LinkedIn has surged in popularity with business leaders and professionals looking to become thought leaders, so has the amount of AI content. * Labels disclosing AI-generated content are becoming increasingly common, particularly on visual platforms. Meta, Google and TikTok automatically label content made with their own AI tools. What we're watching: Whether creators embrace Substack's push toward transparency around AI use and how rival platforms respond.

Substack adds tool that detects AI-generated content

Credit: Thomas Fuller / SOPA Images / LightRocket via Getty Images Substack is introducing a new tool to help readers figure out whether content was written by a human or generated by AI. Announced in a blog post on Tuesday, the self-publishing platform's new feature is powered by AI detection tool Pangram, and is available on Substack posts, notes, replies, and comments. "Not everything made with AI is slop, and not all slop is made with AI," wrote Substack co-founder and CEO Chris Best. "The problem is when there is a mismatch between a reader's expectation and reality, especially when they unwittingly invest their attention in something with no human thought on the other end." Substack's AI detection tool is available now on web and in the iOS app, with Android to follow. To scan Substack content for AI contributions, simply click or tap on the "..." meatball menu, then select "Scan for AI text." A popup will then appear stating what percentage of the text Pangram has determined is AI-generated, AI-assisted, or human. Substack's new AI detection feature does have some restrictions. Specifically, it will only work on Substack text passages longer than 100 words, and only those published after 8:30 a.m. PT / 11:30 a.m. on July 21. Attempting to scan Substack content published prior to this results in a popup stating that it's ineligible for AI detection. Trying to check passages that are shorter than 100 words also surfaces a popup explaining that there isn't enough text for accurate AI detection. The popup on Best's own posts also offers an additional message: "I'm still figuring all of this out. (In the meantime these words are mine, for better or worse.)" Pengram will only scan text for any AI authorship if the reader requests it. As such, it's up to users to take the initiative and investigate the provenance of what they're reading to the best of their ability. Best notes that the tool cannot detect if, for example, a human used AI to do research before writing a post. Substack's AI-detection tool adds some new features for authors as well. Writers are now able to add a statement explaining how they wrote content, which will appear when readers use the AI detection tool on their posts. Authors can run Pengram's AI detection tool on their drafts and submit a report if they believe it has classified their content incorrectly. Interestingly, writers can also remove the flag themselves by completely disabling AI detection on their content, regardless of whether or not Pengram's detection was correct. Pangram itself uses AI to detect AI-generated text. However, rather than generative AI, it deploys a classifier neural network model trained to recognise and distinguish between human and AI-generated text. The company states that its algorithm's dataset of human authored text was gathered from sources published prior to 2021, before generative AI was widely used on the internet, to ensure no AI-generated text snuck its way in. Best stated that Substack is looking into adding more AI-focused features in the future, depending upon user interest and feedback. This includes allowing users to set preferences for the content Substack recommends to them, suggesting that we may eventually see the option to filter out AI-generated content altogether. A recent study by digital marketing firm Graphite found that the number of online articles that are primarily AI-generated is now equal to the number written by humans. Multiple publications have been called out for using AI to generate articles over the last few years, several of which were laden with errors. In some instances, these AI-generated articles were attributed to fake, AI-generated authors or the publication's "staff."

Substack now lets you check if a post was written by AI

A new Pangram-powered scanner lets you check posts, notes, and replies for signs of AI writing. Substack is giving readers a way to check whether the post they're reading was written by a human or by a chatbot. The company has partnered with AI-detection firm Pangram to introduce new tools that will let users scan posts, notes, and replies for AI-generated text. CEO Chris Best introduced the features in a post titled "Against Claudefishing," his term for content that leans on AI while presenting itself as human work. How the scanning tool works Scans run on any text longer than one hundred words that is published from July 21 onward, and the result is visible only to the person who requests it. Best says Pangram is not perfect, but points to independent evaluations that credit it with a high degree of accuracy. The feature is live on web and iOS, with an Android version coming later. Substack's AI-detection feature arrives at a time when the problem it targets is spreading fast across social platforms. Best cites a recent Pangram estimate that as much as forty percent of posts on some platforms are now fully AI-generated. Writers can run the same scan before they publish Readers are not the only ones getting new controls. Substack is adding a "How I make this" statement that lets writers describe their process and set expectations up front. Writers also get the option to run Pangram on their drafts before publishing, and they can report and remove scans of their published work that they believe are mistaken. Best says future additions could include AI preferences inside Reply Rules and reader-side controls for what gets recommended. Recommended Videos For readers, this is a rare transparency feature that hands them a real button to spot AI slop. Substack is not alone in trying to solve this issue. LinkedIn has started reducing the reach of AI-generated posts and comments, and Meta is quietly testing its own AI detection tool, though it has yet to go live. Whether any of these tools can keep pace with the models they're built to catch remains to be seen.

Substack introduces new AI transparency features

Substack has launched a new AI detection tool developed in partnership with Pangram. This tool allows readers to assess how much content was written by AI, applicable to text longer than 100 words. The feature became available today on the web and iOS platforms, with Android support expected in the future. Additionally, Substack is providing a new statement space for creators to explicitly share how AI was utilized in their content. The company aims to set clear expectations for readers regarding AI-generated material. The blog post announcing the feature includes criticism of LinkedIn, calling out the prevalence of AI-generated content on that platform. It describes efforts to create a false sense of human connection with AI-generated content as "Claudefishing." Substack noted that tools for detecting AI-generated content, including those from Pangram, do not guarantee perfect accuracy. The Atlantic has explored the limitations of AI detection tools and highlighted the inaccuracy of Pangram's capabilities. Substack acknowledged these limitations and hinted at potential additional features regarding AI content and user preferences in the future. The company stated that its goal is to ensure readers understand the nature of the content they are engaging with, summarizing its position as "people should know what they're getting."