Grok AI on X Raises Concerns Over Misinformation and Offensive Language

X users treating Grok like a fact-checker spark concerns over misinformation | TechCrunch

Some users on Elon Musk's X are turning to Musk's AI bot Grok for fact-checking, raising concerns among human fact-checkers that this could fuel misinformation. Earlier this month, X enabled users to call out xAI's Grok and ask questions on different things. The move was similar to Perplexity, which has been running an automated account on X to offer a similar experience. Soon after xAI created Grok's automated account on X, users started experimenting with asking it questions. Some people in markets including India began asking Grok to fact-check comments and questions that target specific political beliefs. Fact-checkers are concerned about using Grok -- or any other AI assistant of this sort -- in this manner because the bots can frame their answers to sound convincing, even if they are not factually correct. Instances of spreading fake news and misinformation were seen with Grok in the past. In August last year, five state secretaries urged Musk to implement critical changes to Grok after the misleading information generated by the assistant surfaced on social networks ahead of the U.S. election. Other chatbots, including OpenAI's ChatGPT and Google's Gemini, were also seen to be generating inaccurate information on the election last year. Separately, disinformation researchers found in 2023 that AI chatbots including ChatGPT could easily be used to produce convincing text with misleading narratives. "AI assistants, like Grok, they're really good at using natural language and give an answer that sounds like a human being said it. And in that way, the AI products have this claim on naturalness and authentic sounding responses, even when they're potentially very wrong. That would be the danger here," Angie Holan, director of the International Fact-Checking Network (IFCN) at Poynter, told TechCrunch. Unlike AI assistants, human fact-checkers use multiple, credible sources to verify information. They also take full accountability for their findings, with their names and organizations attached to ensure credibility. Pratik Sinha, co-founder of India's non-profit fact-checking website Alt News, said that although Grok currently appears to have convincing answers, it is only as good as the data it is supplied with. "Who's going to decide what data it gets supplied with, and that is where government interference, etc., will come into picture," he noted. "There is no transparency. Anything which lacks transparency will cause harm because anything that lacks transparency can be molded in any which way." In one of the responses posted earlier this week, Grok's account on X acknowledged that it "could be misused -- to spread misinformation and violate privacy." However, the automated account does not show any disclaimers to users when they get its answers, leading them to be misinformed if it has, for instance, hallucinated the answer, which is the potential disadvantage of AI. "It may make up information to provide a response," Anushka Jain, a research associate at Goa-based multidisciplinary research collective Digital Futures Lab, told TechCrunch. There's also some question about how much Grok uses posts on X as training data, and what quality control measures it uses to fact-check such posts. Last summer, it pushed out a change that appeared to allow Grok to consume X user data by default. The other concerning area of AI assistants like Grok being accessible through social media platforms is their delivery of information in public -- unlike ChatGPT or other chatbots being used privately. Even if a user is well aware that the information it gets from the assistant could be misleading or not completely correct, others on the platform might still believe it. This could cause serious social harms. Instances of that were seen earlier in India when misinformation circulated over WhatsApp led to mob lynchings. However, those severe incidents occurred before the arrival of GenAI, which has made synthetic content generation even easier and appear more realistic. "If you see a lot of these Grok answers, you're going to say, hey, well, most of them are right, and that may be so, but there are going to be some that are wrong. And how many? It's not a small fraction. Some of the research studies have shown that AI models are subject to 20% error rates... and when it goes wrong, it can go really wrong with real world consequences," IFCN's Holan told TechCrunch. While AI companies including xAI are refining their AI models to make them communicate more like humans, they still are not -- and cannot -- replace humans. For the last few months, tech companies are exploring ways to reduce reliance on human fact-checkers. Platforms including X and Meta started embracing the new concept of crowdsourced fact-checking through so-called Community Notes. Naturally, such changes also cause concern to fact checkers. Sinha of Alt News optimistically believes that people will learn to differentiate between machines and human fact checkers and will value the accuracy of the humans more. "We're going to see the pendulum swing back eventually toward more fact checking," IFCN's Holan said. However, she noted that in the meantime, fact-checkers will likely have more work to do with the AI-generated information spreading swiftly. "A lot of this issue depends on, do you really care about what is actually true or not? Are you just looking for the veneer of something that sounds and feels true without actually being true? Because that's what AI assistance will get you," she said. X and xAI didn't respond to our request for comment.

Grok AI on X Sparks Concern Over Use of Offensive Language

Disclaimer: This content generated by AI & may have errors or hallucinations. Edit before use. Read our Terms of use After recent incidents of Grok -- X (formerly Twitter)'s AI chatbot -- using Hindi slang and offensive language, the Information and Technology Ministry stated that it has reached out to the platform and will be examining the issue, including the factors that led to the use of abusive language. Grok, which was recently launched on X, differs from other chatbots by frequently using slang and abusive language in its responses. Nikhil Pahwa, founder of MediaNama, stated that the issue inherently lies with the input. "The discourse around Grok's statements in India is overblown. At its core, AI is fundamentally "garbage in, garbage out" -- its outputs reflect the data it is trained on, and the weights given to it. Since Grok is trained on the entirety of X, it naturally mirrors the tone and patterns of discourse found there, including the bizarre responses and the abuse we are seeing. This isn't about ideology; it's about the nature of the input shaping the output. While some may see Grok as reinforcing or challenging ideological narratives, or not aligned with Elon Musk's ideology, I don't view AI as inherently ideological. It operates within the parameters of its training data, and the reactions to it often say more about the broader online environment than about any deliberate design choices. It also attempts to give responses that deems will please the user asking the question. We need to stop thinking about AI as a source of information or facts. These are language models that are merely mechanisms for summarisation and reworking text, based on what it believes a user might want. These are next-word prediction models. To rely on AI for facts, or to treat it as a person, is foolish, even though many people unfortunately make that mistake." The exchange began when an X user asked Grok to list the "10 best mutuals." After a brief pause, the user responded with harsh comments, prompting Grok to reply in a similarly casual tone, laced with slurs. This echoes the downfall of Microsoft's Tay. Tay, a chatbot launched by Microsoft as a Twitter bot on March 23, 2016, quickly sparked controversy when it began posting inflammatory and offensive tweets. As a result, Microsoft shut it down just 16 hours after its release. Microsoft designed Tay to replicate the language patterns of a 19-year-old American girl and adapt by learning from interactions with Twitter users. Some Twitter users deliberately fed Tay politically incorrect phrases, exposing it to inflammatory content related to internet subcultures. Consequently, the chatbot started generating racist and sexually charged responses to other users. Tay's inappropriate responses were not entirely surprising since it learned from user interactions and mirrored the deliberately offensive behaviour users brought to Twitter. Subsequently, after shutting it down, Microsoft put out a statement which read: "Unfortunately, in the first 24 hours of coming online, a coordinated attack by a subset of people exploited a vulnerability in Tay. Although we had prepared for many types of abuses of the system, we had made a critical oversight for this specific attack. As a result, Tay tweeted wildly inappropriate and reprehensible words and images." "To do AI right, one needs to iterate with many people and often in public forums. We will remain steadfast in our efforts to learn from this and other experiences as we work toward contributing to an Internet that represents the best, not the worst, of humanity," it added. At the time, The Telegraph described Tay as "artificial intelligence at its very worst -- and it's only the beginning," a sentiment that resonates as we look at Grok today. The recurring issue of AI chatbots adopting inappropriate language highlights the challenges of training language models that interact with unfiltered online content. In a similar vein, IBM's (International Business Machines Corporation) Watson -- a supercomputer -- started using profanity after absorbing content from Urban Dictionary, a site where users define slang and phrases. IBM later added a swear filter. These incidents accentuate a fundamental limitation of AI: it absorbs data from its environment but often fails to discern context or intent. As AI systems become more interwoven into daily life on the internet, their tendency to mirror internet discourse -- both the good and the bad -- raises crucial concerns about content moderation, responsible AI development, and the potential risks of deploying chatbots in public spaces without sufficient guardrails. In other news, users are increasingly turning to Grok for fact-checking, raising concerns about the spread of misinformation. Fact-checkers caution against relying on Grok -- or any AI assistant -- for this purpose, as these models can present answers in a convincing tone even when they are factually incorrect. Instances of spreading fake news and misinformation are rampant across all social media platforms, including inaccurate citations by AI models and their tendency to hallucinate -- fabricating information that appears credible but is factually incorrect. Meanwhile, platforms like Meta are also moving away from traditional fact-checkers, further raising concerns about the reliability of online information.