Instagram Head Warns of AI-Generated Content Risks, Urges User Vigilance

Instagram Head Warns About Highly Realistic AI-Generated Images

Instagram head Adam Mosseri cautioned users against trusting online images, noting that AI is "clearly producing" content that can be easily mistaken for reality. In a series of Threads posts on Sunday, Mosseri says it is becoming increasingly difficult to distinguish between real photos and AI-generated images on social media. "Whether or not you're a bull or a bear in the technology, generative AI is clearly producing content that is difficult to discern from recordings of reality, and improving rapidly," Mosseri writes in the Threads posts, that were first spotted by The Verge. Mosseri urged social media platforms to label AI-generated content as accurately as possible and provide more context on posts to help users. However, he acknowledged that it's impossible to catch everything, as some AI-generated content might go unnoticed by social media platforms. Equally, not all misleading content is created by AI. So Mosseri suggested that social media platforms share details about the people or accounts sharing content. This additional context helps users judge the trustworthiness of the source and make informed decisions about the credibility of the content they encounter. "Our role as internet platforms is to label content generated as AI as best we can," Mosseri says "But some content will inevitably slip through the cracks, and not all misrepresentations will be generated with AI, so we must also provide context about who is sharing so you can assess for yourself how much you want to trust their content." Mosseri also advised social media users to be discerning when viewing photos and videos online and consider the source of images. "It's going to be increasingly critical that the viewer, or reader, brings a discerning mind when they consume content purporting to be an account or a recording of reality," Mosseri adds. "My advice is to always consider who it is that is speaking." Earlier this year, Instagram launched a "Made with AI" label for photos. However, the labels created confusion as photographers and content creators discovered their real images had been slapped with the tags despite using minimal editing or apparently not using AI at all.

Deep fakes are fooling millions: Meta's Mosseri sounds the alarm

Meta's Adam Mosseri emphasizes the importance of scrutinizing AI-generated content on social media platforms. As deep fakes become increasingly sophisticated, the ability to discern reality from fabrication is essential for users. Mosseri's comments come amid rising concerns about deep fakes, which utilize generative adversarial networks (GAN) and diffusion models like DALL-E 2 to create false images and videos that are difficult to differentiate from authentic content. The Instagram head believes that social media can help combat misinformation by flagging fake content, although he acknowledges that not all falsehoods can be detected or adequately labeled. "Our role as internet platforms is to label content generated as AI as best we can," he stated. Deep fakes have evolved significantly in recent years. The process involves one AI model generating a fake while another identifies it, continuously refining its accuracy. This results in content that can be alarmingly convincing. As deep fakes gain traction, Mosseri cautions users against blindly trusting online images and videos. In a series of posts on Threads, he urged users to consider the source of shared content, reinforcing the idea that context is crucial in the digital age. He elaborated, "It feels like now is when we are collectively appreciating that it has become more important to consider who is saying a thing than what they are saying." This perspective aligns with evolving digital literacy, where the credibility of the content provider is as vital as the content itself. In the social media landscape, the capability to discern the authenticity of visual content is more pressing than ever. Mosseri noted the necessity for platforms to provide context about the origin of shared material, echoing user-led moderation initiatives seen on other platforms. He highlighted that while some forms of AI-generated misinformation can be identified, others inevitably slip through the cracks. Stanford professor faces allegations of citing fake AI-generated study The urgency of this issue is further underscored by the rapid advancement in AI technology. Today's tools easily produce content that appears real and can be distributed at large scales, often outpacing the capabilities of moderators to respond effectively. As users navigate a daily flood of information, they are encouraged to cultivate a discerning eye, considering who shares the information and the implications behind it. Investigation into how platforms label and moderate AI-generated content remains ongoing. Mosseri's acknowledgment of the limitations in current labeling practices suggests the need for more robust strategies to combat misinformation. Given the technological advancements in AI media generation, how platforms adapt to these changes and continue fostering user awareness remains an open question. While Meta is hinting at future changes in its content moderation strategies, it is unclear how quickly these changes will be implemented or their effectiveness in countering the technologically adept manipulations seen today. The complexities introduced by AI generate challenges that require a proactive and informed audience, capable of critically assessing the content they consume online.

Meta's Instagram boss: who posted something matters more in the AI age

In a series of Threads posts this afternoon, Instagram head Adam Mosseri says users shouldn't trust images they see online because AI is "clearly producing" content that's easily mistaken for reality. Because of that, he says users should consider the source, and social platforms should help with that. "Our role as internet platforms is to label content generated as AI as best we can," Mosseri writes, but he admits "some content" will be missed by those labels. Because of that, platforms "must also provide context about who is sharing" so users can decide how much to trust their content. Just as it's good to remember that chatbots will confidently lie to you before you trust an AI-powered search engine, checking whether posted claims or images come from a reputable account can help you consider their veracity. At the moment, Meta's platforms don't offer much of the sort of context Mosseri posted about today, although the company recently hinted at big coming changes to its content rules. What Mosseri describes sounds closer to user-led moderation like Community Notes on X and YouTube or Bluesky's custom moderation filters. Whether Meta plans to introduce anything like those isn't known, but then again, it has been known to take pages from Bluesky's book.

Instagram Head Wants X's Community Notes-Like Feature To Offer Context On What People Are Saying, Warns Against Generative AI Blurring Lines Between Reality And Fiction - Meta Platforms (NASDAQ:META)

Instagram head Adam Mosseri raised concerns about the growing challenge of distinguishing between real and AI-generated images on social media platforms. What Happened: In a series of posts on Threads on Sunday, Mosseri highlighted the need for social media platforms to provide more context to help users discern the authenticity of online content. His warning comes amid increasing sophistication in AI technology, which is making it harder for users to identify real content. In his posts, Mosseri stated that AI is "clearly producing" content that can easily be mistaken for reality. He urged users to consider the source of images and suggested that platforms should label AI-generated content as accurately as possible. See Also: Elon Musk Finds An Unlikely Ally In Mark Zuckerberg's Meta In His Fight Against Sam Altman's OpenAI However, he acknowledged that some AI content might evade these labels. "My advice is to *always* consider who it is that is speaking." Subscribe to the Benzinga Tech Trends newsletter to get all the latest tech developments delivered to your inbox. Why It Matters: Currently, Meta Platforms, Inc. META does not offer extensive context as described by Mosseri, though the company has hinted at significant changes to its content policies. Mosseri's vision aligns with user-led moderation systems like Community Notes on X, formerly Twitter, and custom moderation filters on platforms like YouTube or Bluesky. The issue of AI-generated content is not new, but it is becoming more sophisticated and widespread. Previously, in Florida, an incident highlighted the potential for AI to be used in fraudulent activities, where a young investor was nearly deceived into purchasing a non-existent property. Moreover, a study by Google DeepMind revealed that deepfakes of public figures are more common than AI-assisted cyber attacks, indicating a significant misuse of AI technology. In response to these challenges, Meta has previously taken some steps to combat AI-generated disinformation. Earlier in May this year, the company dismantled several fake news campaigns originating from countries like China and Russia, which used AI to spread false information. Check out more of Benzinga's Consumer Tech coverage by following this link. Read Next: Microsoft's iPhone File Sharing, Apple's Watch Ultra, $1.2 Billion Lawsuit And More: This Week In Appleverse Disclaimer: This content was partially produced with the help of Benzinga Neuro and was reviewed and published by Benzinga editors. Photo courtesy: Shutterstock Market News and Data brought to you by Benzinga APIs

Meta thinks social media can protect us from deep fakes - 9to5Mac

Deep fakes are arguably the most dangerous aspect of AI. It's now relatively trivial to create fake photos, audio, and even video. See below for deep fakes of Morgan Freeman and Tom Cruise, for example. But while social media has so far been used as a mechanism for distributing deep fakes, Instagram head Adam Mosseri thinks it can actually play a key role in debunking them ... The main method used to create deep fake videos to date has been an approach known as generative adversarial networks (GAN). One AI model either generates a fake video clip or displays a genuine one. A second AI model attempts to identify the fakes. Repeatedly running this process trains the first model to generate increasingly convincing fakes. However, diffusion models like DALL-E 2 are now taking over. These take real video footage, then make various changes to create a large number of variations of them. Text prompts can be used to instruct the AI model on the results we want, making them easier to use - and the more people who use them, the better trained they become. Here's a well-known example of a Morgan Freeman deep fake, created a full three years ago, when the technology was much less sophisticated than it is today: And another, of Tom Cruise as Iron Man: Brits may also recognise Martin Lewis, who is well-known for offering financial advice, here in a deep fake to promote a crypto scam: Meta exec Adam Mosseri thinks that social media can actually make things better rather than worse, by helping to flag fake content - though he does note that it isn't perfect at doing this, and we each need to consider sources.