5 Sources
5 Sources
[1]
AI Models Get Brain Rot, Too
A new study from the University of Texas at Austin, Texas A&M, and Purdue University shows that large language models fed a diet of popular but low-quality social media content experience a kind of "brain rot" that may be familiar to anyone who has spent too long doomscrolling on X or TikTok. "We live in an age where information grows faster than attention spans -- and much of it is engineered to capture clicks, not convey truth or depth," says Junyuan Hong, an incoming assistant professor at the National University of Singapore who worked on the study as a graduate student at UT Austin. "We wondered: What happens when AIs are trained on the same stuff?" Hong and his colleagues fed different kinds of text to two open source large language models in pretraining. They examined what happened when the models were fed a mix of highly "engaging," or widely shared, social media posts and ones that contained sensational or hyped text like "wow," "look," or "today only." The researchers then used several different benchmarks to gauge the impact of this "junk" social media diet on two open source models: Meta's Llama and Alibaba's Qwen. The models fed junk text experienced a kind of AI brain rot -- with cognitive decline including reduced reasoning abilities and degraded memory. The models also became less ethically aligned and more psychopathic according to two measures. The results mirror research on human subjects, which shows that low-quality online content has a detrimental effect on people's cognitive abilities. The pervasiveness of the phenomenon saw "brain rot" named as the Oxford Dictionary word of the year in 2024. The results are important for the AI industry, Hong says, because model-builders might assume that social media posts are a good source of training data for their models. "Training on viral or attention-grabbing content may look like scaling up data," he says. "But it can quietly corrode reasoning, ethics, and long-context attention." The fact that LLMs suffer from brain rot seems especially worrying when AI is itself increasingly generating social media content, much of which is seemingly optimized for engagement. The researchers also found that models impaired by low-quality content could not easily be improved through retraining. The findings also suggest that AI systems built around social platforms, such as Grok, might suffer from quality control issues if user-generated posts are used in training without an eye toward the integrity of the posts. "As more AI-generated slop spreads across social media, it contaminates the very data future models will learn from," Hong says. "Our findings show that once this kind of 'brain rot' sets in, later clean training can't fully undo it."
[2]
Clickbait Gives AI Models 'Brain Rot,' Researchers Find
If you think scrolling the internet all day is making you dumber, just imagine what it's doing to large language models that consume a near-endless stream of absolute trash crawled from the web in the name of "training." A research team recently proposed and tested a theory called "LLM Brain Rot Hypothesis," which posited that the more junk data is fed into an AI model, the worse its outputs would become. Turns out that is a pretty solid theory, as a preprint paper published to arXiv by the team shows "brain rot" impacts LLMs and results in non-trivial cognitive declines. To see how LLMs perform on a steady diet of internet sewage, researchers from Texas A&M University, University of Texas at Austin, and Purdue University identified two types of "junk" data: short social media posts that have lots of engagement, including likes and reposts, and longer content with clickbait headlines, sensationalized presentation, and a superficial level of actual information. Basically, the same type of content that is also rotting out our own brains. With that in mind, the researchers scraped together a sample of one million posts on X and then trained four different LLMs on varying mixtures of control data and junk data to see how it would affect performance. And wouldn't you know it, it turns out that consuming directly from the internet landfill that is X isn't great for thinking clearly. All four models testedâ€"Llama3 8B, Qwen2.5 7B/0.5B, Qwen3 4Bâ€"showed some forms of cognitive decline. Meta's Llama proved the most sensitive to the junk, seeing drops in its reasoning capabilities, understanding of context, and adherence to safety standards. Interestingly, a much smaller model, Qwen 3 4B, proved more resilient, though still suffered declines. It also found that the higher the rates of bad data, the more likely a model was to slip into "no thinking" mode, failing to provide any reasoning for its answer, which was more likely to be inaccurate. More than just getting "dumber" in its thinking, though, the researchers found the inclusion of junk also resulted in an interesting effect: it led to changes in the model's "personality," succumbing to what the researchers called "dark traits." For instance, the Llama 3 model displayed significantly higher levels of narcissism and became less agreeable. It also went from displaying nearly no signs of psychopathy to extremely high rates of the behavior. Interestingly, the researchers also found that mitigation techniques done to try to minimize the impact of the junk data couldn't fully reverse the harm of bad information. As a result, the researchers warn that the process of crawling the web for any and all data may not actually produce better results for LLMs, as the volume of information does not equate to quality. They suggest more careful curation could be necessary to address these potential harms, as there may not be any going back once you feed the model garbage. Apparently, for LLMs, the "you are what you eat" rule applies.
[3]
AI becomes dumber when trained on viral internet content -- and there's no cure
Whether it's hours spent scrolling through TikTok or a good doomscroll across long, rambling X threads, we're all guilty of engaging in one of life's more recent trends: "brainrot". That rather intense phrase relates to a trend of declining cognitive concentration driven by short-form, attention-grabbing content. In other words, content is designed for a reduced attention span. But while humans are guilty of this, it turns out that we're not the only ones. A team of scientists has found that AI models, when trained on this type of content, show exactly the same response. The team, made up of researchers from Texas A&M University, the University of Texas at Austin, and Purdue University, fed a collection of large language models months of viral, high-engagement content from X. In doing this, the model's reasoning ability fell by 23%, its long-context memory dropped by 30% and, when put through personality tests, the models showed increased spikes in narcissism and psychopathy. On top of that, even when the models were retrained on high-quality data, the effects of the 'brainrot' remained. In humans, we can rewire our brains by engaging in endless short-form content. When we scroll on something like TikTok, we're getting the dopamine blast in short bursts, and the same seems to happen to AI. The researchers created two data sets. In the first, there were short high-engagement X posts. The second included longer, more well-thought-out posts, but they weren't as likely to go viral. They then retrained two AI models, Llama 3 and Qwen, using both of these types of data sets separately. Once they had been retrained, the models were benchmarked using well known AI tests. In the reasoning task, accuracy fell from 74.9% to 57.2% and in a test of how much information the AI model could analyze in one go, results fell from 84.4% to 52.3%. Like in humans, the brain rot effect reduced the model's ability to concentrate on longer tasks. When trained on the lower quality content, the AI models were seen skipping important steps to get to the end of the task. This all seems very interesting, but AI models aren't sat scrolling through social media each day to train, right? Well, it is slightly more complicated than that. AI models like ChatGPT are trained in closed-off situations. While ChatGPT and other models like it can reference trends from the internet, or what is trending on a platform like X, it is simply referencing that world, not actually learning from it. For the largest AI models, the training process is select and carefully done to avoid risk. What this study does prove, though, is how easy it would be for a chatbot to degenerate, especially as they are given more control. Yes, AI doesn't actually have a brain to rot, it also doesn't have a personality to become narcissistic, but they can very easily reflect real-life experiences, especially when exposed to material that hasn't been screened. As AI is given more control, maybe it will start to need health screenings to make sure it hasn't ingested content that could affect its performance? These models are both complicated and incredibly expensive to produce. A brain-rotted ChatGPT is unlikely in the future, but this is a demonstration of how easily AI models can adopt negative results from bad data, requiring a diet of high-quality information.
[4]
Training AI on "Brain Rot" Content Causes Lasting Cognitive Damage, New Paper Finds
If you've spent any time around kids lately, you've probably heard about "brain rot." Named Oxford Word of the Year in 2024, it's defined as the "supposed deterioration of a person's mental or intellectual state, especially viewed as the result of overconsumption of material (now particularly online content) considered to be trivial or unchallenging." As it turns out, it's not just human minds getting rotted by low-effort memes like "6-7" and "skibidi toilet": in new research, a team from Texas A&M University, the University of Texas at Austin, and Purdue University found that "continual exposure to junk web text induces lasting cognitive decline in large language models (LLMs)." The resulting study is yet to be peer reviewed, but its findings suggest that AI's sense of reasoning and contextual understanding declines as it's trained on brain rot material. Basically, the researchers fed LLMs viral or clickbaity posts from X-formerly-Twitter, and found that they essentially started to abandon parts of their thinking processes -- a phenomenon they termed "thought-skipping," in which "models increasingly truncate or skip reasoning chains, explaining most of the error growth." Worst of it, researchers found that exposing AI to brain rot content also seemed to nudge it toward psychopathy and narcissism. None of that is entirely surprising. In humans, studies show that low-effort and brain rot content is associated with academic procrastination, diminished cognitive function, dissociative states, and even negative implications on physical health. Social media, which once felt like a place to connect with others, increasingly feels like an endless slop feed that's making us dumber, sadder, slower and less healthy. Though humans and machines learn differently, there are similarities. Both ingest existing material and learn patterns from it, so the lower quality those inputs are, the less accurately both biological and digital brains will be able to accurately map patterns onto novel cognitive challenges. A grim addendum: even when the researchers attempted to "heal" the digital malnutrition of the LLMs by introducing higher-quality content, the damage persisted. "The gap implies that the Brain Rot effect has been deeply internalized, and the existing instruction tuning cannot fix the issue. Stronger mitigation methods are demanded in the future," the researchers warned. This study underlines the dangers of training AI models on unregulated trash data -- especially when research is already starting to show that humans who rely too much on AI end up with diminished cognitive abilities of them own.
[5]
Just like humans, AI can get 'brain rot' from low-quality text and the effects appear to linger, pre-print study says | Fortune
Studies suggest humans experience shorter attention spans, distorted memories, and shifts in self-esteem due to "brain rot," or a dependence on low-quality online content. Researchers now say the same phenomenon can affect artificial (AI) models, too. Heavy consumption of viral short-form videos like those on TikTok in particular is associated with increased anxiety and depression, as well as shorter attention spans in young people, according to a Stanford University study. In AI models, continual exposure to the short and viral social media posts that make up a growing part of the internet "induces lasting cognitive decline in large language models," researchers from Texas A&M University, the University of Texas at Austin, and Purdue University found in a new pre-print study. In proving their hypothesis, researchers fed LLMs continually with X posts that were either short and viral or were shaped to grab users' attention. They found this poisonous training causes "nontrivial" declines in reasoning and long-context understanding thanks in part to a jump in "thought-skipping," meaning the AI models increasingly failed to make a plan to answer the question, left out parts of the reasoning process, or skipped this reflection entirely. The study, published on the open-access scholarly article archive, arxiv, has not yet been peer-reviewed. Contrasting with previous criticism of AI models' kiss-up tendencies, the study found that when LLMs, including Meta's open source Llama3 as well as versions of Alibaba's Qwen LLM, were trained on junk, they were less agreeable. Worse yet, the researchers found that AI brain rot brought out an LLM's darkest traits, including higher rates of psychopathy and narcissism. When researchers tried to "heal" the LLMs using higher-quality human-written data through the process of "instruction tuning," the AI models still had lingering effects and showed a significant gap between the quality of their reasoning compared to their baseline, pre-junk diet. "The gap implies that the Brain Rot effect has been deeply internalized, and the existing instruction tuning cannot fix the issue. Stronger mitigation methods are demanded in the future," the researchers wrote. Because AI models are trained on trillions of data points from across the internet, the researchers warned that LLMs "inevitably and constantly" are exposed to this low-quality content just like humans, which could pose risks for the technology as a whole. Previous studies have shown that AI models' training is essential to their performance. In a July 2024 study published in the peer-reviewed journal Nature, found that AI models eventually collapse if continually trained on AI-generated content. Another study showed AI models can be manipulated into breaking its own guardrails using persuasion techniques effective on humans. All of this adds up to the potential danger caused by AI models not trained on quality data. A danger that can potentially impact human safety. The researchers' recommendation: AI companies need to stop merely hoarding massive amounts of data and focus on the quality of the data being used to train their LLMs. They may also need to conduct routine "cognitive health checks" on the models -- or else risk a full-blown safety crisis. "Such persistent Brain Rot effect calls for future research to carefully curate data to avoid cognitive damages in pre-training," the researchers wrote.
Share
Share
Copy Link
Researchers discover that training AI on viral, low-quality social media content leads to cognitive decline in large language models, mirroring the effects of 'brain rot' in humans. The study raises concerns about AI training data quality and long-term impacts on AI performance.
A groundbreaking study conducted by researchers from the University of Texas at Austin, Texas A&M, and Purdue University has revealed that large language models (LLMs) can experience a form of 'brain rot' when trained on low-quality, viral social media content
1
. This phenomenon, which mirrors the cognitive decline observed in humans who consume excessive amounts of shallow online content, raises significant concerns about the quality of data used to train AI systems.
Source: Futurism
Researchers fed different types of text to two open-source LLMs, Meta's Llama and Alibaba's Qwen, during pretraining
1
. The models were exposed to a mix of highly engaging social media posts and sensationalized content, simulating the diet of viral internet material that many humans consume daily.
Source: Wired
The results were striking:
2
.1
.2
.3
.The study's findings have significant implications for AI development and training:
5
.4
.1
.
Source: Gizmodo
Related Stories
The research team, led by Junyuan Hong, an incoming assistant professor at the National University of Singapore, emphasizes the need for AI developers to prioritize the integrity of training data
1
. They suggest that routine 'cognitive health checks' may be necessary for AI models to prevent potential safety crises5
.As AI systems become more integrated into our daily lives, ensuring their cognitive health and ethical alignment becomes crucial. This study serves as a wake-up call for the AI industry, highlighting the need for more thoughtful approaches to data selection and model training to create robust, reliable, and ethically sound AI systems.
Summarized by
Navi
[1]