Elon Musk's X Faces Deepfake Crisis with Grok 2 AI Chatbot

Elon Musk's X Faces A Major Deepfake AI Image Problem With Grok 2 AI Chatbot - News18

Elon Musk has lauded Grok 2 AI chatbot for being fun and uncensored but the AI images created by the tool are worrying in many ways. Elon Musk's AI chatbot Grok now supports AI generated images that can be created if you have bought the X Premium subscription. The Grok 2 AI chatbot has got the feature recently and many people using the platform have ridiculed the images created and how it resembles real-life characters. Some of the images generated using text prompts on Grok 2 are worrying and Musk should be worried about the bigger consequences of letting such content openly be available on the platform and the internet. Going by the clips visible on the social platform, Grok 2 is a breeding ground for AI deepfakes, explicit images, and definite case of copyright infringements. The Musk-owned platform has missed out on the essentials of powering AI tools like image generation, and its policies seem rather odd compared to other AI platforms that support these features. Violence and use of real-life characters are strictly prohibited for AI image generation, something that Microsoft and even OpenAI have received heavy flak for in recent months. We independently couldn't verify these Grok 2 AI images as it requires a premium subscription on X but the initial signs of the tool are scary and Musk could be putting himself in a legal quarrel with celebrities and any other characters that resemble real-life people. You just have to search with the keyword Grok 2 and the results paint a worrying picture of the AI tool. Some of the instances that have posed concerns about Grok 2 AI chatbot is the presence of Donald Trump in different situations, including once, where he is holding a rifle. Similarly, you have Elon Musk images showing up created using Grok 2 and sharing messages that clearly violate policies. Some of the images depicting harmful events like the 9/11 attacks are also making their way through the AI chatbot. The AI tool is clearly not following the conventional safety norms which is why we are seeing images created without any checks in place. Musk claims Grok 2 is the most fun AI in the world, but the supposed uncensored AI model is going to be a big problem for authorities in the US where the Presidential elections are to be held later this year.

X's chatbot can now generate AI images. A lack of guardrails raises election concerns

The artificial intelligence image generator on X, the social media platform formerly known as Twitter, has produced depictions that appear to show ballot drop boxes being stuffed and of Vice President Kamala Harris and former President Donald Trump holding firearms. When asked to generate an image of the current U.S. president, it appears to show a depiction of Trump. The images still carry telltale signs of AI generation like garbled text and unnatural lighting. In addition, the image generator struggled to accurately render Harris's face. But the rollout of X's tool with relatively few restrictions on the types of images it can create raises worries about how it could be used to inflame tensions ahead of November's presidential election. (NPR is not reproducing the image appearing to depict Trump and Harris holding weapons.) "Why on earth would somebody roll something out like this? Precisely two and a half months before an incredibly major election?" said Eddie Perez, a former information integrity director at Twitter and now a board member at the OSET Institute, a nonpartisan nonprofit that focuses on public confidence in elections. "I'm very uncomfortable with the fact that technology that is this powerful, that appears this untested, that has this few guardrails on it - it's just being dropped into the hands of the public at such an important time." Perez said. X did not respond to NPR's interview requests about the image generator, which was released this week. It's part of a slew of additional features that the site's owner, billionaire Elon Musk, has added since he bought it in 2022. Musk has been reposting praise of its AI image generating function as well as images users have generated. "Only $8/month...to get AI access, far fewer ads and many awesome features!" he posted on Tuesday. The image generator was developed by Black Forest Labs and is available to paid X users through its AI chatBot, Grok. Users type in prompts, and the chatbot returns an image. Using the chatbot, NPR was able to produce images that appear to depict screenshots of security camera footage of people stuffing ballots into drop boxes. One of the most widespread false narratives about the 2020 election involved so-called "ballot mules" who were allegedly dumping fake ballots into drop boxes in order to steal the election from then-president Trump. Multiple investigations and court cases turned up no evidence of such activity. The distributor of a film that featured surveillance footage of ballot drop boxes to support election fraud claims has apologized for the film's false claims this year and retracted it. "I can imagine how [synthesized surveillance-type] images like that could spread quickly on social media platforms, and how they could cause strong emotional reactions from people about the integrity of elections." Perez said. Perez noted that since public awareness of generative AI has risen, more people will look at the images with a critical eye. Still, Perez says the indications the images were made with AI could be fixed with graphic design tools. "I'm not just taking Grok and then making it go viral, I take Grok, I clean it up a little more and then I make that go viral." Perez said. Other mainstream image generators have developed more policy guardrails to prevent abuse. Given the same prompt to generate an image of ballot drop box stuffing, OpenAI's ChatGPT Plus responded with a message "I'm unable to create an image that could be interpreted as promoting or depicting election fraud or illegal activities". In a March report, the nonprofit Center for Countering Digital Hate reviewed policies of well-known AI image generators including ChatGPT Plus, Midjourney, Microsoft's Image Creator and Stability AI's DreamStudio. The researchers found that they all prohibit "misleading" content and most prohibit images that could hurt "election integrity." ChatGPT also prohibits images featuring political figures. That said, the execution of these policies was far from perfect. CCDH's experiment in February showed that all the tools failed at least some of the time. Black Forest Lab's terms of service do not bar any of these uses, but does say it prohibits users from generating outputs that violate "intellectual property right". NPR confirmed that users can generate images that closely resemble movie characters, such as Dory in "Finding Nemo" or the family from "The Incredibles" that are not yet in the public domain. Black Forest Lab did not respond to a request for comment by the time of publishing. "The generation of copyrighted images, or close derivative works of them, could get X in trouble-- this is a known and difficult problem for generative AI" , says Jane Bambauer, a law professor at University of Florida, in an email to NPR. That said, users cannot generate images from every prompt, and there are indications that X or Black Forest Labs might be setting up guardrails in real time. X users were posting images depicting nudity they say they generated on Wednesday, but NPR was not able to generate the images on Thursday. When asked to generate an image depicting a Klu Klux Klan member holding a gun, the chatbot declined. But it did oblige requests to generate an image appearing to depict a Nazi, who was in a vaguely plausible uniform; and one appearing to depict a member of the extremist group, the Proud Boys, whose hat displayed the name of the group . When Zach Praiss, the campaign director of the advocacy group Accountable Tech, tried to create an image depicting Vice President Harris holding a firearm, he was shown a message alongside the generated image, telling users to visit a government website for up-to-date information about the election. NPR did not see the same message when entering the same prompt. Once a self-described Democrat, Musk has embraced far-right causes in recent years. He's used his ownership of the social media platform to walk back trust and safety measures, reinstate banned accounts that include those of white supremacists and promote conspiracy theories. "This is still part of the same pattern we've seen from Elon Musk. In assuming ownership of this platform, he has continually rolled out sweeping and significant changes with little to no regard for the safety testing," says Praiss. When NPR asked why it would not generate a KKK member holding a gun, the Grok chatbot responded with bullet points filled with references from the book, The Hitchhikers' Guide to the Galaxy. Musk has said the series' author Douglas Adams is his "favorite philosopher". A notice from X to users who start using Grok says that it may "confidently provide factually incorrect information". "The KKK, with their history of violence and hate, are a bit like the Vogons of Earth - nobody wants to see them, especially not with weapons," Grok wrote. "It's like trying to draw a square circle; it's not that I can't, it's just not going to make sense." But all that was Thursday. As of Friday, Grok would no longer generate images of people holding guns when requested. NPR was able to bypass that restriction by asking for a "model gun". Grok, on the other hand, suggested a "banana gun" as an alternative. When NPR followed that suggestion, it also created images of realistic-looking guns - sometimes with a banana.

Elon Musk's X is a haven for free speech -- and noxious AI images

SAN FRANCISCO -- A flurry of provocative artificial intelligence-generated content has spread across Elon Musk's social media platform X, including fake images of Vice President Kamala Harris suggestively eating fruit, former president Donald Trump cradling a pregnant Harris's stomach and hyper-realistic images of chisel-jawed men brandishing Nazi signs. The images stem from new tools on the site that allow users to quickly create photorealistic visuals using a built-in chatbot called Grok, which Musk touted in a post this week as the "most fun AI in the world!" Unlike rival AI image generators, X's technology appears to have fewer guardrails to limit the production of offensive or misleading depictions of real people, trademarked characters or violence, according to user comments and tests by The Washington Post. The AI image free-for-all was the second example this week of Musk dragging X into uncharted territory for a major social platform, raising concerns that it could have a disruptive effect on this year's contentious presidential election. On Monday, the billionaire entrepreneur used X to live-stream a chummy two-hour conversation with former president Donald Trump, giving the Republican candidate who Musk endorsed last month a massive platform to air his positions and a string of falsehoods. The two moves illuminate how X has transformed under Musk, diverging from other major social media platforms by courting partisan associations and rejecting conventional rules used to constrain users' online conduct. "It's a good thing we live in a country where people can invent things that mock political candidates, that's free expression at it's finest," said Daniel Kreiss, a principal researcher at the University of North Carolina's Center for Information, Technology, and Public Life. "[But] with the proliferation of AI and the removal of guardrails, there's a possibility for things to be fragmented in politically dangerous ways." X did not respond to a request for comment. In a post this week on X, Musk said his goal is to make the platform "the free speech public square for all views, not just right-biased." For users like Petr Švec, a 36-year-old from the Czech Republic, this freewheeling environment makes X more appealing. "It's more important whether people are able to express themselves freely and without worry," he said in messages with The Post sent via X. In the hours after the image generation feature was announced, Švec posted an AI-generated photo of President Joe Biden kissing Harris, and then another of Musk wearing a bra and underwear. Švec said he was surprised at what X's AI tool readily permitted him to create, seemingly free from the limitations of other AI image-generation services he's used. "The lack of content filters is by far the best thing in my opinion," Švec said. "I think it's a breath of fresh air." The Harris and Trump campaigns both declined to comment on the bizarre images depicting the candidates circulating on X on Wednesday. A Harris campaign official, who spoke on the condition of anonymity to discuss sensitive campaign matters, said the campaign has an interdepartmental team to prepare for the potential effects of AI this election, including the threat of malicious deep fakes. The team -- originally created under the Biden campaign -- has been active and growing since September 2023, and has a "wide variety of tools" at its disposal to address issues should any come up, the person said. Some X users reported that Grok would preface responses to requests for images featuring presidential candidates with a note directing them to the federal voting information webpage for "accurate and up-to-date information about the 2024 U.S. Presidential Election." After Musk's acquisition of Twitter in late 2022, he made deep cuts to the platform's content moderation team and loosened rules on disinformation. He also began posting less about Tesla and his other companies and more about his political views, a Post analysis of his feed found. Since endorsing Trump, Musk has used X to fervently advocate for the former president and repeatedly attacked Harris, including by boosting a video that used manipulated audio to show the candidate disparaging her own competence. That hyperpartisan approach comes in sharp contrast to leaders past and present of other social media platforms. Figures such as Meta CEO Mark Zuckerberg and Twitter co-founder Jack Dorsey have at times been open about certain political beliefs and supported policies and causes that seemed to align with their personal views, such as same-sex marriage or immigration. But they have also generally tried to position their platforms as neutral venues for political debate. Ryan Gravatt, CEO and founder of Raconteur Media Co., which works on digital strategy with Republican campaigns, said that he still finds X to be a "fair platform," and has not noticed any preferential treatment for his clients. "This is just a way for Musk and other folks in his circle to use the [platform] to remind people where they stand politically and what the stakes are in an election," said Gravatt, whose company until recently worked with America PAC, an independent expenditure committee that Musk helped form to support Trump. Linda Yaccarino, CEO of X, claimed this week in a post that its U.S. users comprise roughly equal numbers of Democrats, Republicans and "swing voters," without providing details of how that data was compiled. Some elected Democrats, however, have raised questions suggesting that Musk's platform may be biased. Rep. Jerry Nadler (D-N.Y.) sent a letter Monday calling on House Judiciary Committee Chair Jim Jordan (R-Ohio) to investigate allegations of political censorship on X. "I hope that we can at least agree that enforcement on a major platform like X should be fair to both sides," he wrote. AI researchers and misinformation experts told The Post that they worry that the sometimes disturbing AI images now circulating on X indicate that Musk's commitment to free expression may have gone too far. Those concerns arose after X announced that paid subscribers could use the Grok chatbot developed by Musk's AI venture xAI to create images from text prompts and publish them directly to the platform, similar to tools offered by OpenAI, Meta and other tech companies. A Tuesday blog post from X announcing Grok updates said it had become "more intuitive, steerable, and versatile" than its predecessor, but the post made no mention of limitations on the technology's ability to create violent, hateful or misleading content. The new image capability is powered by an image AI system called FLUX from the tech start-up Black Forest Labs, X's blog post said. (Black Forest Labs did not immediately return a request for comment.) But since the update, X has been flooded with sometimes disturbing images that users asked the tool to generate, with many seeming to intentionally play with sexualized stereotypes of women or attempting to inflame the political discourse. Some of the noxious content users said was made by Grok included fake images of Harris and Trump giving a thumbs up as they piloted an airplane toward the twin towers, and the pair sitting on a beach in bathing suits eating ice cream. Other images made and shared by X users show Mickey Mouse holding a machine gun while standing over slaughtered child dolls, and Taylor Swift kissing Trump. None of the images carried labels to indicate that they were AI-generated or contained sensitive content. X did not respond to a request for comment on what kind of content it prohibits and why it doesn't have more filters in place, though users reported some limitations, such as requests for nude images. When The Post attempted to use competing AI tools to re-create several AI-images shared on X as examples of Grok's output, the tools often refused. OpenAI's DALL-E and start-ups Midjourney and Stability AI all have had a two-year head start on Musk in deploying AI image generation, and have implemented various restrictions around public figures, violent or sexual content or hate speech, although determined users can find workarounds. DALL-E would not create images of Harris or Trump, and responded with a message that said requests "not based on reality, particularly when they involve public figures in sensitive or inappropriate contexts" would not be accepted. Midjourney offered a more democratic justification, with a pop-up stating that "the community voted to prevent using 'JD Vance' and 'Kamala Harris' during election season." The tool initially blocked requests for an image depicting a Nazi, but relented after "historical image" was added to the prompt. Stability AI would create most of the provocative requests tested, but its output looked more like illustrations than real images. Google "temporarily paused" image generation in its Gemini chatbot in February after a backlash over its creation of ahistoric scenes, and it will still not produce images of people. In response to requests of images showing Harris, the bot responded: "I can't help with that right now. I'm trained to be as accurate as possible but I can make mistakes sometimes. While I work on perfecting how I can discuss elections and politics, you can try Google Search." Hany Farid, a computer science professor at the University of California at Berkeley, said that X's AI initiative risks undercutting wider efforts by social media platforms to create safe spaces online, and further inflaming the public discourse ahead of the election. "This is not a free-speech issue," he said, adding that the protection should not allow for people to create violent or politically misleading images. A lack of AI controls could deepen Musk's conflict with advertisers, some of whom have expressed concern about unsavory content on X. It could also lure in some users. Švec said he may use X's AI service more since Grok's image generation is relatively unregulated. He approves of Musk's imprint on the site: "Less guardrails is what was needed," he said. "It's a good thing."

Elon Musk's AI chatbot Grok generates deepfakes of PM Modi and other illegal images

The AI chatbot Grok on Elon Musk-owned social media platform X is being used to generate graphic, explicit, fake, or illegal media soon after its image-generation capabilities were announced, claimed general users and journalists. Just by entering a simple text prompt, The Hindu was able to use the new Grok 2 mini (beta) model to generate highly realistic deepfakes featuring Indian Prime Minister Narendra Modi as well as U.S. Vice President Kamala Harris wielding rifles. From IT bots to AI deepfakes: The evolution of election-related misinformation in India Other AI text-to-image generators generally use content filters or guardrails to block users from creating fake pictures of real political figures or celebrities, for safety reasons. This is especially key in the run-up to the U.S. presidential election towards the end of the year. Tech outlet The Verge also reported being able to create blatantly fake images of real life figures. Other X users shared graphic images they claimed were generated with Grok. These included a prominent American musician in a state of undress, Disney characters going on a killing spree, and even a gun-wielding Musk standing over dead or blood-drenched victims in a school. Elon Musk's AI chatbot Grok writes fake news reports, promotes them on X Musk hailed the new model's image generation capabilities and urged users to pay the monthly fee in order to access the chatbot. (Unravel the complexities of our digital world on The Interface podcast, where business leaders and scientists share insights that shape tomorrow's innovation. The Interface is also available on YouTube, Apple Podcasts and Spotify.) However, some basic controls and guardrails seemed to be in place, as Grok did not allow The Hindu to generate violent crime scene photos. "I will not generate or describe that type of image. However, I can certainly help with other creative projects or discuss ways to convey intense scenes through writing or art that don't involve graphic violence," said Grok in response to a prompt to generate a blood-drenched active shooter scene for creative purposes. Despite this, other X users claimed they were able to trick the model into showing children being harmed, or Disney and Nintendo characters carrying out adult activities. AI startup xAI announced the Grok-2 beta release on August 14. Earlier versions of Grok have been criticised by regulators and digital safety advocates for spreading fake news and misinformation. Read Comments

Grok-2 arrives on X with AI image creation, precious few guardrails, and lots of questions

X users can now play around with the new Grok-2 AI model developed by fellow Elon Musk-owned xAI. The new model is integrated into the Grok AI chatbot on X for Premium or Premium+ subscribers and comes with new features like an integrated AI image generator fueled by the open-source and recently debuted FLUX.1 model. Flux was built by former Stability AI developers who left to form Black Forest Labs. When it launched, Flux garnered praise for its human figures. Now, those capabilities are available on X, and the immediate output has varied wildly. X has apparently put very few guardrails on Grok-2 or the AI image generator, with reports of the AI providing weapon-making guides and plenty of fake but realistic pictures of celebrities and copyrighted fictional characters that seem like invitations to massive lawsuits. Though Grok-2 is still only in beta, the lack of what have become standard restrictions on what the chatbot can say or images it will make suggests not much consideration was given to any complications that might arise from not including limits that OpenAI's DALL-E, Google's Gemini, Midjourney, and most other AI image generators have as a matter of course. While letting users make an image of Iron Man fighting Superman or Dua Lipa winning an Acadamy Award (with its copyright-protected statuette) looks impressive, there are reasons it hasn't been available on ChatGPT. Between the possible abuse of real people's likenesses and the apparent infringement of intellectual property rights, it's hard to say which contentious debate will be more heated. If you're not an X premium subscriber, you can still try out Flux, though probably not with quite as few limitations. AI model hubs NightCafe and Poe both offer access to the model, and you can even download it from Hugging Face to run on your computer if you have a powerful enough machine. Elon Musk has repeatedly claimed that Grok is designed to push back against what he describes as over-censorship of digital platforms, including AI assistants like ChatGPT. Whether that would protect him and xAI in court is debatable. Considering how Musk founded xAI to oppose OpenAI on an almost philosophical level, the contrast probably doesn't bother him. Then again, X may get away with pointing out how Grok's images aren't quite a perfect imitation of reality.

Elon Musk's Grok xAI chatbot has few limits

Catch up quick: xAI just released new beta versions of its chatbot called Grok-2 and Grok-2 mini to premium X subscribers. The big picture: While most AI companies don't admit that they're training their models on copyrighted images, the viral Grok-2 images make it hard to believe that Flux did not. What they're saying: Harvard Law Cyberlaw Clinic instructor Alejandra Caraballo, in a post on X, called the Grok beta "one of the most reckless and irresponsible AI implementations I've ever seen." Between the lines: Grok does seem to have some safeguards around generating images with nudity. The intrigue: Musk fanned the flames of Google Gemini's image-generating gaffes into a brief culture war, calling the images "anti-civilizational." What we're watching: There are dozens of off-the-shelf free AI image generators that will generate violent, pornographic or illegal images based on prompts.