Meta AI image detector fails to catch 55% of its own cropped AI-generated images

Meta AI image detector fails to identify some of its own cropped AI images, Reuters analysis finds

July 10 (Reuters) - A new AI detection tool from Meta (META.O), opens new tab, which the tech company previewed this week alongside the launch of its image-generation model, Muse Image, failed to identify some of its own AI-generated images once they were cropped, according to a Reuters analysis. The finding highlights the challenges of verifying AI-generated images after common alterations, a limitation that could make it harder to identify deepfakes online during a busy election year that includes the U.S. midterms. In an analysis of 40 images generated using Muse Image, Reuters found the detection tool verified all of the original AI-generated images but failed to verify 55% of ⁠the same images after they were cropped to approximately one-third to one-half of their original size. On its website, opens new tab, Meta says the preview detection tool can identify its own AI-generated images, even if they are cropped, through an invisible watermarking system called Content Seal, which is embedded in every image generated by Muse Image and designed to help users verify whether it was created by Meta's AI models. When asked about the results of the Reuters analysis of the detection tool, Meta noted that the tool was a preview. The company said the watermark is designed to remain intact after common edits, but that the signal may be lost if an image is heavily cropped. Rival tech companies Google and OpenAI have cautioned that their own detection tools ⁠are not foolproof against image-alteration techniques. In March, Meta's Oversight Board, a body of experts that makes binding decisions and issues recommendations on content issues across the company's social media platforms, called on the company, opens new tab to do more to address the "proliferation of deceptive AI-generated content" on its platforms and invest in stronger detection tools. Siwei Lyu, a computer science professor at the State University of New York at Buffalo who researches AI ⁠image forensics, said he had not evaluated Meta's tool but that watermark-based systems have limitations. "Watermark-based methods can be highly effective when the watermark remains intact, but any modification that removes or weakens the embedded signal -- such as cropping, resizing, heavy compression, or editing -- may reduce ⁠their effectiveness, depending on how the watermark is designed," Lyu said. Sarah Barrington, an AI researcher and Ph.D. candidate at the UC Berkeley School of Information, said watermarking holds promise for the future of AI-generated content, but could only do so much. "Like many preventive cybersecurity ⁠or physical security measures, it may not be fully watertight, but even if we catch only 90% of cases, that's still a great leap from 0," she said. Reporting by Hardik Vyas in Bengaluru and Seana Davis in Barcelona; additional reporting by B Carmel Jaeslin and Josh Salisbury; Editing by Stephanie Burnett, Ken Li and Nia Williams Our Standards: The Thomson Reuters Trust Principles., opens new tab

Meta built an AI detection tool to ID images and video created with its new models - Engadget

Meta is working on a tool to ID images and video created with its new image generation model, Muse Image. The company showed off a preview of the web-based tool that can check for the invisible watermarks used by the new model. This watermarking system, called Content Seal, remains in place "even when cropped, compressed, resized, or screenshotted," Meta explains in a blog post. "We're previewing a detection tool that lets you check whether an image carries a Content Seal watermark, providing an initial way to help you better understand if an image was made with Meta AI." Content Seal seems to be a somewhat new approach for Meta. The version that's part of Muse Image is proprietary, though the company has previously released open-source versions of the tech, Meta told Engadget. Meta's new models don't include any visible watermarks, like some previous versions of Meta AI that added a small logo to the bottom right corner. For now, Meta AI's detection abilities are limited to images that are created or edited with Muse Image, though the company said it plans to expand Content Seal watermarks to AI-generated and edited videos as well. Meta is also working on a separate video generation model called Muse Video that will be "coming soon." I tried out the new detection feature on images I created today with Meta AI and the web-based tool was able to detect a watermark for edited images and entirely AI-made creations (like the one pictured above). It also found the watermark in screenshots of my images. "A positive result means that the image was generated or edited using the Meta AI app or meta.ai," the company explains in an FAQ. "A negative result means it is unlikely that the image was processed using Meta AI app or meta.ai." Interestingly, Meta AI's new detection abilities don't seem to be part of the Meta AI app yet. When I asked Meta's app-based assistant about an image the web tool had identified as AI-made, it replied that it did not have the ability to check. "I can't tell you definitively if this specific image was made with Meta Al just by looking at it," it said. "Meta Al doesn't automatically watermark images, and I don't have a tool that can detect which Al model made an existing image." Meta has previously faced some criticism for how it labels and identifies AI-generated material in its apps. The Oversight Board told the company earlier this year that it was "concerned" that Meta was "inconsistently implementing" digital watermarks on AI content created by its own tools. The new feature does still seem to have some other limitations, though. Content Seal is not compatible with SynthID or C2PA Content Credentials, two established watermarking methods used by other companies. The web-based feature was unable to identify images created or edited with earlier versions of Meta's AI models in my testing. When I added images created in older chats with Meta AI, it was unable to tell me if the image was made with its AI. The feature also appears, for some reason, to be subject to Meta's rate limits. After uploading a handful of examples, I was alerted that I had reached my "daily limit on identification checks."

Meta AI detector misses half its own cropped fakes

Meta previewed an AI image detector this week promising to catch anything its Muse Image model makes, even after editing. A Reuters test found that once the images were cropped, the tool missed 55% of them, exposing how fragile AI watermarks really are. The Meta AI detector promises to catch Meta's own fakes. Crop the image, and more than half slip straight past it. The tool was meant to be a fix for the deepfake problem, not an example of it. This week Meta previewed an image detector alongside Muse Image, its most advanced image generator yet, and promised it could spot anything the model made later, even after editing. Then Reuters ran the test. It generated 40 images with Muse Image, cropped them, and fed them back. The detector missed more than half. How a simple crop broke it The numbers are the story. Reuters found the tool verified every one of the 40 original AI images. Crop those same pictures to roughly a third or a half of their size, and it failed to flag 55% of them. A crop, the kind anyone does before posting, was enough to strip the signal the detector leans on. That signal is a watermark. Meta calls it Content Seal, an invisible marker baked into every image Muse Image produces. On its own website, Meta says the Meta AI detector can identify its images even after a crop. The Reuters analysis suggests the promise holds only up to a point. Meta's answer, and the catch Asked about the results, Meta pointed out that the detector is still a preview. The watermark is built to survive common edits, the company said, but the signal "may be lost if an image is heavily cropped". That is the tension in one sentence. The mark is meant to be robust, yet the most ordinary edit on the internet can rub it out. Meta is not alone in the bind. Google and OpenAI have both warned that their own detection tools are not foolproof against people who alter images. Watermarking is the industry's favoured answer to synthetic media, and every big lab is leaning on a version of it. A rival marker, Google's SynthID, recently debunked a high-profile deepfake, which is the case for the technology. Meta's stumble is the case against trusting it alone. Why a watermark is not a wall Researchers have flagged this weakness for a while. Siwei Lyu, a computer science professor at the University at Buffalo who studies image forensics, said watermark methods work well while the mark stays intact. The trouble is what comes next. "Any modification that removes or weakens the embedded signal, such as cropping, resizing, heavy compression, or editing, may reduce their effectiveness", he told Reuters. Others argue the bar should not be perfection. Sarah Barrington, an AI researcher at UC Berkeley, likened watermarking to security measures that catch most threats without stopping all of them. "Even if we catch only 90%, that's still a great leap from 0", she said. Both points can hold at once. A detector that misses 55% of lightly edited images sits a long way below 90%, and it feeds a growing market for AI detection that still cannot promise certainty. The timing is the problem The gap matters because of when it lands. The United States is heading into a midterm election year, and platforms are bracing for a wave of AI fakes aimed at voters. Governments are moving too, with South Korea among those writing punitive laws against deceptive content. In March, Meta's own Oversight Board urged the company to do more about deceptive AI and to invest in stronger detection. Four months on, the flagship detector cannot reliably catch Meta's own output once someone crops it. None of this makes Content Seal worthless. A tool that tags fresh, unedited images still raises the cost of passing off a fake, and Meta says it plans to extend the system to video. It does puncture the idea that a watermark is a solution rather than a speed bump. The people most likely to strip a signal are the ones a detector exists to stop. In synthetic media, as in the classroom, detection keeps arriving a step behind. On today's evidence, catching up takes nothing more than a crop.

Meta's AI Detector Can't Detect Images It Generated Itself, Report Finds

Meta's new AI detection tool isn't working entirely as advertised, according to a new report. Meta debuted its first image generation model, Muse Image, earlier this week. As part of the debut, the tech giant also announced that all images generated by the model would include an invisible watermarking system called Content Seal. This signal would remain intact even when the AI-generated image gets "cropped, compressed, resized, or screenshotted" by users, the company claimed. To help with catching the Content Seal signal, Meta also announced that it was previewing an AI detection tool to check whether Muse Image generated an image. But, in a report published on Friday, Reuters reporters found that the AI detection tool failed to identify more than half the images it generated once they had been cropped. In the first test, Reuters found that the tool correctly identified all 40 images generated by Muse Image as AI-generated, but once those images were cropped to half or one-third of their original size, the tool was only able to identify 55% as AI-generated. As generative AI tools get better at producing uncanny deepfakes, detection becomes a trickier problem to solve. According to cybersecurity firm DeepStrike, the volume of AI-generated deepfakes online has experienced a roughly 900% annual growth from 2023 to 2025. But detection capabilities haven't advanced completely in parallel to this boom in popularity. Commercial AI detection tools, themselves driven by AI, are still plagued with mistakes, while the average person's ability to identify AI-generated content is no better than a coin toss, according to previous studies. Though not a true immediate success, that's the gap Meta is aiming to address with the new Content Seal and its detection tool. Muse Image and its accompanying products were meant to be a major step forward for Meta, which has arguably been trailing its competitors in the AI space. Last year, Meta CEO Mark Zuckerberg decided there is no time like the present to try to catch up and announced a major AI turnaround effort. The catch-up plan included committing multibillion-dollar investments into research and development and poaching top talent from rivals all across the industry, all in pursuit of building better AI products and the lofty goal of creating artificial superintelligence. An additional couple billion dedicated to AI and a few more restructurings later, Meta unveiled the first major fruit of that labor in April with Muse Spark, a proprietary model that it said it plans to open-source in the future and was met with a mixed reception. Another major one was this week's Muse Image debut. But the debut of the image generator and its accompanying tools has been mired in controversy, and not just because of the Reuters report. Instagram users were alarmed when they found out that the AI model could use photos from any public profile without explicitly asking the owner of said profile for their consent. That feature has now been removed. The company is now eyeing its next big generative AI debut: a video generator called Muse Video. Here's to hoping the company can bridge any gaps in detection tools and adequately address users' privacy concerns before that model drops.

Meta AI image detector fails to identify some of its own cropped AI images, Reuters analysis finds

Reuters quoted Siwei Lyu about a Reuters analysis that found that Meta's AI image detector failed to identify some of its own AI-generated images once they were cropped. Lyu had not evaluated Meta's tool but said that invisible watermark-based systems like Meta's have limitations. "Watermark-based methods can be highly effective when the watermark remains intact, but any modification that removes or weakens the embedded signal -- such as cropping, resizing, heavy compression, or editing -- may reduce ⁠their effectiveness, depending on how the watermark is designed," Lyu said.

Meta's detection tool fails to identify photos generated by its own Muse Image AI

Meta has created an invisible watermarking tool called Content Seal that is embedded in all images generated by the Muse Image AI. Earlier this week, Meta announced two new AI products, namely, Muse Image and Muse Video. As the name suggests, these are generative AI tools for making photos and video clips using natural language text prompts. Soon after their rollout commenced, these tools sparked controversy because Meta had automatically opted in Instagram users, allowing others to use their publicly posted media and convert them into remixed AI content. But it appears that Meta courted another loss on its side of the court. What's the problem? Alongside its new AI tools, Meta introduced something called Content Seal, which serves as an invisible watermarking system. Technically, all the images created by the Muse Image AI carry a hidden signal that can be used to identify whether they were made using AI or not. To go with it, the company also launched its own AI identification tool that can read this invisible Content Seal watermark. But it appears that Meta's AI sniffing tool is not as accurate as the company claims. According to an analysis by Reuters, images made using the company's Muse Image AI cannot be reliably detected as AI-generated by Meta's AI identification software. The outlet analyzed 40 images that were created by Muse Image, but in only 45% of the cases was it able to identify that they were created using AI. In the remaining 55% of the cases, when the images were cropped to one-third or one-fourth of their original size, the AI detection tool developed by Meta simply failed. Oh, yikes. That's embarrassing! That's a notable flub. On its website, Meta claims that the Content Seal system carries a "hidden provenance signal that stays intact -- even when cropped, compressed, resized, or screenshotted." Following Reuters' analysis, Meta told the outlet that its AI detection tool is still in preview and that it can't work reliably when a photo is "heavily cropped." On an FAQ page available on its AI detector site, Meta also makes it clear that if an image has been generated using a third-party AI product instead of Muse Image, the detection tool will not work. Meta won't be the only AI giant that has created a system like Content Seal. Google already has an AI image detection and watermarking system called SynthID in place. It has been adopted by the likes of OpenAI as well. That means if an image is generated using ChatGPT or Gemini, it carries the invisible SynthID watermark and can be detected using Google's own AI identification tool, which is now available on Google Search, as well. However, Google also makes it clear that SynthID is not 100% foolproof.

Meta's new tool identifies images made with Muse Image

Meta is developing a web-based tool to identify images and videos generated with its Muse Image model. The tool can detect invisible watermarks known as Content Seal, which remain intact when images are cropped, resized, or screenshotted, according to a blog post from the company. The Content Seal watermarking system is proprietary, a shift from Meta's previous open-source versions of the technology. Unlike earlier models that used visible watermarks, the current models do not display logos, Meta stated. The detection capabilities focus exclusively on images created or edited using Muse Image, though the company plans to extend the watermarking system to AI-generated and edited videos in the future. Testing of the detection tool showed it could successfully identify watermarks in both edited images and those created entirely by AI. A positive detection result indicates the image was processed using Meta AI or meta.ai, according to Meta. Conversely, a negative result suggests it is unlikely that the image was created with Meta's tools. Despite these detection capabilities, the feature is not yet integrated into the Meta AI app itself. An inquiry made to Meta's app-based assistant revealed it lacks the ability to confirm the generation source of an image, stating, "I can't tell you definitively if this specific image was made with Meta AI just by looking at it." The assistant noted that the app does not automatically watermark images. Meta has faced scrutiny over its AI content labeling practices. The Oversight Board raised concerns about inconsistencies in implementing digital watermarks on AI-generated content. The Content Seal system is incompatible with other established watermarking methods like SynthID or C2PA Content Credentials. Further limitations of the detection tool emerged during testing, which could not identify images created with older versions of Meta's AI. Users also experienced rate limits, receiving notifications after reaching their daily maximum for identification checks. Meta is also developing a separate video generation model named Muse Video, which is expected to launch soon. Currently, the detection feature remains inaccessible via the Meta AI app.