Meta AI image detector fails to identify 55% of cropped images in Reuters analysis

Meta AI image detector fails to identify some of its own cropped AI images, Reuters analysis finds

July 10 (Reuters) - A new AI detection tool from Meta (META.O), opens new tab, which the tech company previewed this week alongside the launch of its image-generation model, Muse Image, failed to identify some of its own AI-generated images once they were cropped, according to a Reuters analysis. The finding highlights the challenges of verifying AI-generated images after common alterations, a limitation that could make it harder to identify deepfakes online during a busy election year that includes the U.S. midterms. In an analysis of 40 images generated using Muse Image, Reuters found the detection tool verified all of the original AI-generated images but failed to verify 55% of ⁠the same images after they were cropped to approximately one-third to one-half of their original size. On its website, opens new tab, Meta says the preview detection tool can identify its own AI-generated images, even if they are cropped, through an invisible watermarking system called Content Seal, which is embedded in every image generated by Muse Image and designed to help users verify whether it was created by Meta's AI models. When asked about the results of the Reuters analysis of the detection tool, Meta noted that the tool was a preview. The company said the watermark is designed to remain intact after common edits, but that the signal may be lost if an image is heavily cropped. Rival tech companies Google and OpenAI have cautioned that their own detection tools ⁠are not foolproof against image-alteration techniques. In March, Meta's Oversight Board, a body of experts that makes binding decisions and issues recommendations on content issues across the company's social media platforms, called on the company, opens new tab to do more to address the "proliferation of deceptive AI-generated content" on its platforms and invest in stronger detection tools. Siwei Lyu, a computer science professor at the State University of New York at Buffalo who researches AI ⁠image forensics, said he had not evaluated Meta's tool but that watermark-based systems have limitations. "Watermark-based methods can be highly effective when the watermark remains intact, but any modification that removes or weakens the embedded signal -- such as cropping, resizing, heavy compression, or editing -- may reduce ⁠their effectiveness, depending on how the watermark is designed," Lyu said. Sarah Barrington, an AI researcher and Ph.D. candidate at the UC Berkeley School of Information, said watermarking holds promise for the future of AI-generated content, but could only do so much. "Like many preventive cybersecurity ⁠or physical security measures, it may not be fully watertight, but even if we catch only 90% of cases, that's still a great leap from 0," she said. Reporting by Hardik Vyas in Bengaluru and Seana Davis in Barcelona; additional reporting by B Carmel Jaeslin and Josh Salisbury; Editing by Stephanie Burnett, Ken Li and Nia Williams Our Standards: The Thomson Reuters Trust Principles., opens new tab

Meta built an AI detection tool to ID images and video created with its new models - Engadget

Meta is working on a tool to ID images and video created with its new image generation model, Muse Image. The company showed off a preview of the web-based tool that can check for the invisible watermarks used by the new model. This watermarking system, called Content Seal, remains in place "even when cropped, compressed, resized, or screenshotted," Meta explains in a blog post. "We're previewing a detection tool that lets you check whether an image carries a Content Seal watermark, providing an initial way to help you better understand if an image was made with Meta AI." Content Seal seems to be a somewhat new approach for Meta. The version that's part of Muse Image is proprietary, though the company has previously released open-source versions of the tech, Meta told Engadget. Meta's new models don't include any visible watermarks, like some previous versions of Meta AI that added a small logo to the bottom right corner. For now, Meta AI's detection abilities are limited to images that are created or edited with Muse Image, though the company said it plans to expand Content Seal watermarks to AI-generated and edited videos as well. Meta is also working on a separate video generation model called Muse Video that will be "coming soon." I tried out the new detection feature on images I created today with Meta AI and the web-based tool was able to detect a watermark for edited images and entirely AI-made creations (like the one pictured above). It also found the watermark in screenshots of my images. "A positive result means that the image was generated or edited using the Meta AI app or meta.ai," the company explains in an FAQ. "A negative result means it is unlikely that the image was processed using Meta AI app or meta.ai." Interestingly, Meta AI's new detection abilities don't seem to be part of the Meta AI app yet. When I asked Meta's app-based assistant about an image the web tool had identified as AI-made, it replied that it did not have the ability to check. "I can't tell you definitively if this specific image was made with Meta Al just by looking at it," it said. "Meta Al doesn't automatically watermark images, and I don't have a tool that can detect which Al model made an existing image." Meta has previously faced some criticism for how it labels and identifies AI-generated material in its apps. The Oversight Board told the company earlier this year that it was "concerned" that Meta was "inconsistently implementing" digital watermarks on AI content created by its own tools. The new feature does still seem to have some other limitations, though. Content Seal is not compatible with SynthID or C2PA Content Credentials, two established watermarking methods used by other companies. The web-based feature was unable to identify images created or edited with earlier versions of Meta's AI models in my testing. When I added images created in older chats with Meta AI, it was unable to tell me if the image was made with its AI. The feature also appears, for some reason, to be subject to Meta's rate limits. After uploading a handful of examples, I was alerted that I had reached my "daily limit on identification checks."

Meta's new tool identifies images made with Muse Image

Meta is developing a web-based tool to identify images and videos generated with its Muse Image model. The tool can detect invisible watermarks known as Content Seal, which remain intact when images are cropped, resized, or screenshotted, according to a blog post from the company. The Content Seal watermarking system is proprietary, a shift from Meta's previous open-source versions of the technology. Unlike earlier models that used visible watermarks, the current models do not display logos, Meta stated. The detection capabilities focus exclusively on images created or edited using Muse Image, though the company plans to extend the watermarking system to AI-generated and edited videos in the future. Testing of the detection tool showed it could successfully identify watermarks in both edited images and those created entirely by AI. A positive detection result indicates the image was processed using Meta AI or meta.ai, according to Meta. Conversely, a negative result suggests it is unlikely that the image was created with Meta's tools. Despite these detection capabilities, the feature is not yet integrated into the Meta AI app itself. An inquiry made to Meta's app-based assistant revealed it lacks the ability to confirm the generation source of an image, stating, "I can't tell you definitively if this specific image was made with Meta AI just by looking at it." The assistant noted that the app does not automatically watermark images. Meta has faced scrutiny over its AI content labeling practices. The Oversight Board raised concerns about inconsistencies in implementing digital watermarks on AI-generated content. The Content Seal system is incompatible with other established watermarking methods like SynthID or C2PA Content Credentials. Further limitations of the detection tool emerged during testing, which could not identify images created with older versions of Meta's AI. Users also experienced rate limits, receiving notifications after reaching their daily maximum for identification checks. Meta is also developing a separate video generation model named Muse Video, which is expected to launch soon. Currently, the detection feature remains inaccessible via the Meta AI app.