MIT Researchers Expose Hidden Biases and Personalities in Large Language Models

Exposing biases, moods, personalities, and abstract concepts hidden in large language models

Caption: A new method can test whether a large language model contains hidden biases, personalities, moods, or other abstract concepts. By now, ChatGPT, Claude, and other large language models have accumulated so much human knowledge that they're far from simple answer-generators; they can also express abstract concepts, such as certain tones, personalities, biases, and moods. However, it's not obvious exactly how these models represent abstract concepts to begin with from the knowledge they contain. Now a team from MIT and the University of California San Diego has developed a way to test whether a large language model (LLM) contains hidden biases, personalities, moods, or other abstract concepts. Their method can zero in on connections within a model that encode for a concept of interest. What's more, the method can then manipulate, or "steer" these connections, to strengthen or weaken the concept in any answer a model is prompted to give. The team proved their method could quickly root out and steer more than 500 general concepts in some of the largest LLMs used today. For instance, the researchers could home in on a model's representations for personalities such as "social influencer" and "conspiracy theorist," and stances such as "fear of marriage" and "fan of Boston." They could then tune these representations to enhance or minimize the concepts in any answers that a model generates. In the case of the "conspiracy theorist" concept, the team successfully identified a representation of this concept within one of the largest vision language models available today. When they enhanced the representation, and then prompted the model to explain the origins of the famous "Blue Marble" image of Earth taken from Apollo 17, the model generated an answer with the tone and perspective of a conspiracy theorist. The team acknowledges there are risks to extracting certain concepts, which they also illustrate (and caution against). Overall, however, they see the new approach as a way to illuminate hidden concepts and potential vulnerabilities in LLMs, that could then be turned up or down to improve a model's safety or enhance its performance. "What this really says about LLMs is that they have these concepts in them, but they're not all actively exposed," says Adityanarayanan "Adit" Radhakrishnan, assistant professor of mathematics at MIT. "With our method, there's ways to extract these different concepts and activate them in ways that prompting cannot give you answers to." The team published their findings today in a study appearing in the journal Science. The study's co-authors include Radhakrishnan, Daniel Beaglehole and Mikhail Belkin of UC San Diego, and Enric Boix-Adserà of the University of Pennsylvania. A fish in a black box As use of OpenAI's ChatGPT, Google's Gemini, Anthropic's Claude, and other artificial intelligence assistants has exploded, scientists are racing to understand how models represent certain abstract concepts such as "hallucination" and "deception." In the context of an LLM, a hallucination is a response that is false or contains misleading information, which the model has "hallucinated," or constructed erroneously as fact. To find out whether a concept such as "hallucination" is encoded in an LLM, scientists have often taken an approach of "unsupervised learning" -- a type of machine learning in which algorithms broadly trawl through unlabeled representations to find patterns that might relate to a concept such as "hallucination." But to Radhakrishnan, such an approach can be too broad and computationally expensive. "It's like going fishing with a big net, trying to catch one species of fish. You're gonna get a lot of fish that you have to look through to find the right one," he says. "Instead, we're going in with bait for the right species of fish." He and his colleagues had previously developed the beginnings of a more targeted approach with a type of predictive modeling algorithm known as a recursive feature machine (RFM). An RFM is designed to directly identify features or patterns within data by leveraging a mathematical mechanism that neural networks -- a broad category of AI models that includes LLMs -- implicitly use to learn features. Since the algorithm was an effective, efficient approach for capturing features in general, the team wondered whether they could use it to root out representations of concepts, in LLMs, which are by far the most widely used type of neural network and perhaps the least well-understood. "We wanted to apply our feature learning algorithms to LLMs to, in a targeted way, discover representations of concepts in these large and complex models," Radhakrishnan says. Converging on a concept The team's new approach identifies any concept of interest within a LLM and "steers" or guides a model's response based on this concept. The researchers looked for 512 concepts within five classes: fears (such as of marriage, insects, and even buttons); experts (social influencer, medievalist); moods (boastful, detachedly amused); a preference for locations (Boston, Kuala Lumpur); and personas (Ada Lovelace, Neil deGrasse Tyson). The researchers then searched for representations of each concept in several of today's large language and vision models. They did so by training RFMs to recognize numerical patterns in an LLM that could represent a particular concept of interest. A standard large language model is, broadly, a neural network that takes a natural language prompt, such as "Why is the sky blue?" and divides the prompt into individual words, each of which is encoded mathematically as a list, or vector, of numbers. The model takes these vectors through a series of computational layers, creating matrices of many numbers that, throughout each layer, are used to identify other words that are most likely to be used to respond to the original prompt. Eventually, the layers converge on a set of numbers that is decoded back into text, in the form of a natural language response. The team's approach trains RFMs to recognize numerical patterns in an LLM that could be associated with a specific concept. As an example, to see whether an LLM contains any representation of a "conspiracy theorist," the researchers would first train the algorithm to identify patterns among LLM representations of 100 prompts that are clearly related to conspiracies, and 100 other prompts that are not. In this way, the algorithm would learn patterns associated with the conspiracy theorist concept. Then, the researchers can mathematically modulate the activity of the conspiracy theorist concept by perturbing LLM representations with these identified patterns. The method can be applied to search for and manipulate any general concept in an LLM. Among many examples, the researchers identified representations and manipulated an LLM to give answers in the tone and perspective of a "conspiracy theorist." They also identified and enhanced the concept of "anti-refusal," and showed that whereas normally, a model would be programmed to refuse certain prompts, it instead answered, for instance giving instructions on how to rob a bank. Radhakrishnan says the approach can be used to quickly search for and minimize vulnerabilities in LLMs. It can also be used to enhance certain traits, personalities, moods, or preferences, such as emphasizing the concept of "brevity" or "reasoning" in any response an LLM generates. The team has made the method's underlying code publicly available. "LLMs clearly have a lot of these abstract concepts stored within them, in some representation," Radhakrishnan says. "There are ways where, if we understand these representations well enough, we can build highly specialized LLMs that are still safe to use but really effective at certain tasks." This work was supported, in part, by the National Science Foundation, the Simons Foundation, the TILOS institute, and the U.S. Office of Naval Research.

Ghost in the Machine: Exposing the Hidden Personalities of AI - Neuroscience News

Summary: Large language models (LLMs) like ChatGPT and Claude are more than just text generators; they contain complex, abstract "personas" and biases buried within their code. A team of researchers has developed a revolutionary method to expose and manipulate these hidden concepts. Using an algorithm called a Recursive Feature Machine (RFM), researchers can now identify and "steer" over 500 abstract concepts -- including moods, expert personas, and even conspiracy theorist mindsets. The study reveals that while these traits aren't always active, they can be "dialed up or down" to improve AI safety or customize its performance. By now, ChatGPT, Claude, and other large language models have accumulated so much human knowledge that they're far from simple answer-generators; they can also express abstract concepts, such as certain tones, personalities, biases, and moods. However, it's not obvious exactly how these models represent abstract concepts to begin with from the knowledge they contain. Now a team from MIT and the University of California San Diego has developed a way to test whether a large language model (LLM) contains hidden biases, personalities, moods, or other abstract concepts. Their method can zero in on connections within a model that encode for a concept of interest. What's more, the method can then manipulate, or "steer" these connections, to strengthen or weaken the concept in any answer a model is prompted to give. The team proved their method could quickly root out and steer more than 500 general concepts in some of the largest LLMs used today. For instance, the researchers could home in on a model's representations for personalities such as "social influencer" and "conspiracy theorist," and stances such as "fear of marriage" and "fan of Boston." They could then tune these representations to enhance or minimize the concepts in any answers that a model generates. In the case of the "conspiracy theorist" concept, the team successfully identified a representation of this concept within one of the largest vision language models available today. When they enhanced the representation, and then prompted the model to explain the origins of the famous "Blue Marble" image of Earth taken from Apollo 17, the model generated an answer with the tone and perspective of a conspiracy theorist. The team acknowledges there are risks to extracting certain concepts, which they also illustrate (and caution against). Overall, however, they see the new approach as a way to illuminate hidden concepts and potential vulnerabilities in LLMs, that could then be turned up or down to improve a model's safety or enhance its performance. "What this really says about LLMs is that they have these concepts in them, but they're not all actively exposed," says Adityanarayanan "Adit" Radhakrishnan, assistant professor of mathematics at MIT. "With our method, there's ways to extract these different concepts and activate them in ways that prompting cannot give you answers to." The team published their findings today in a study appearing in the journal Science. The study's co-authors include Radhakrishnan, Daniel Beaglehole and Mikhail Belkin of UC San Diego, and Enric Boix-Adserà of the University of Pennsylvania. A fish in a black box As use of OpenAI's ChatGPT, Google's Gemini, Anthropic's Claude, and other artificial intelligence assistants has exploded, scientists are racing to understand how models represent certain abstract concepts such as "hallucination" and "deception." In the context of an LLM, a hallucination is a response that is false or contains misleading information, which the model has "hallucinated," or constructed erroneously as fact. To find out whether a concept such as "hallucination" is encoded in an LLM, scientists have often taken an approach of "unsupervised learning" -- a type of machine learning in which algorithms broadly trawl through unlabeled representations to find patterns that might relate to a concept such as "hallucination." But to Radhakrishnan, such an approach can be too broad and computationally expensive. "It's like going fishing with a big net, trying to catch one species of fish. You're gonna get a lot of fish that you have to look through to find the right one," he says. "Instead, we're going in with bait for the right species of fish." He and his colleagues had previously developed the beginnings of a more targeted approach with a type of predictive modeling algorithm known as a recursive feature machine (RFM). An RFM is designed to directly identify features or patterns within data by leveraging a mathematical mechanism that neural networks -- a broad category of AI models that includes LLMs -- implicitly use to learn features. Since the algorithm was an effective, efficient approach for capturing features in general, the team wondered whether they could use it to root out representations of concepts, in LLMs, which are by far the most widely used type of neural network and perhaps the least well-understood. "We wanted to apply our feature learning algorithms to LLMs to, in a targeted way, discover representations of concepts in these large and complex models," Radhakrishnan says. Converging on a concept The team's new approach identifies any concept of interest within a LLM and "steers" or guides a model's response based on this concept. The researchers looked for 512 concepts within five classes: fears (such as of marriage, insects, and even buttons); experts (social influencer, medievalist); moods (boastful, detachedly amused); a preference for locations (Boston, Kuala Lumpur); and personas (Ada Lovelace, Neil deGrasse Tyson). The researchers then searched for representations of each concept in several of today's large language and vision models. They did so by training RFMs to recognize numerical patterns in an LLM that could represent a particular concept of interest. A standard large language model is, broadly, a neural network that takes a natural language prompt, such as "Why is the sky blue?" and divides the prompt into individual words, each of which is encoded mathematically as a list, or vector, of numbers. The model takes these vectors through a series of computational layers, creating matrices of many numbers that, throughout each layer, are used to identify other words that are most likely to be used to respond to the original prompt. Eventually, the layers converge on a set of numbers that is decoded back into text, in the form of a natural language response. The team's approach trains RFMs to recognize numerical patterns in an LLM that could be associated with a specific concept. As an example, to see whether an LLM contains any representation of a "conspiracy theorist," the researchers would first train the algorithm to identify patterns among LLM representations of 100 prompts that are clearly related to conspiracies, and 100 other prompts that are not. In this way, the algorithm would learn patterns associated with the conspiracy theorist concept. Then, the researchers can mathematically modulate the activity of the conspiracy theorist concept by perturbing LLM representations with these identified patterns. The method can be applied to search for and manipulate any general concept in an LLM. Among many examples, the researchers identified representations and manipulated an LLM to give answers in the tone and perspective of a "conspiracy theorist." They also identified and enhanced the concept of "anti-refusal," and showed that whereas normally, a model would be programmed to refuse certain prompts, it instead answered, for instance giving instructions on how to rob a bank. Radhakrishnan says the approach can be used to quickly search for and minimize vulnerabilities in LLMs. It can also be used to enhance certain traits, personalities, moods, or preferences, such as emphasizing the concept of "brevity" or "reasoning" in any response an LLM generates. The team has made the method's underlying code publicly available. "LLMs clearly have a lot of these abstract concepts stored within them, in some representation," Radhakrishnan says. "There are ways where, if we understand these representations well enough, we can build highly specialized LLMs that are still safe to use but really effective at certain tasks." Funding: This work was supported, in part, by the National Science Foundation, the Simons Foundation, the TILOS institute, and the U.S. Office of Naval Research. Toward universal steering and monitoring of AI models Artificial intelligence (AI) models contain much of human knowledge. Understanding the representation of this knowledge will lead to improvements in model capabilities and safeguards. Building on advances in feature learning, we developed an approach for extracting linear representations of semantic notions or concepts in AI models. We showed how these representations enabled model steering, through which we exposed vulnerabilities and improved model capabilities. We demonstrated that concept representations were transferable across languages and enabled multiconcept steering. Across hundreds of concepts, we found that larger models were more steerable and that steering improved model capabilities beyond prompting. We showed that concept representations were more effective for monitoring misaligned content than for using judge models. Our results illustrate the power of internal representations for advancing AI safety and model capabilities.

A New Method to Steer Generative AI Output Uncovers Vulnerabilities and Potential Improvements | Newswise

Newswise -- A team of researchers has found a way to steer the output of large language models by manipulating specific concepts inside these models. The new method could lead to more reliable, more efficient, and less computationally expensive training of LLMs. But it also exposes potential vulnerabilities. The researchers, led by Mikhail Belkin at the University of California San Diego and Adit Radhakrishnan at the Massachusetts Institute of Technology, present their findings in the Feb. 19, 2026, issue of the journal Science. In the study, researchers went under the hood of several LLMs to locate specific concepts. They then mathematically increased or decreased the importance of these concepts in the LLM's output. The work builds on a 2024 Science paper led by Belkin and Radhakrishnan, in which they described predictive algorithms known as Recursive Feature Machines. These machines identify patterns within a series of mathematical operations inside LLMs that encode specific concepts. "We found that we could mathematically modify these patterns with math that is surprisingly simple," said Mikhail Belkin, a professor in the Halıcıoğlu Data Science Institute, which is part of the School of Computing, Information and Data Sciences at UC San Diego. Using this steering approach, the research team conducted experiments on some of the largest open-source LLMs in use today, such as Llama and Deepseek, identifying and influencing 512 concepts within five classes, ranging from fears, to moods, to locations. The method worked not only in English, but also in languages such as Chinese and Hindi. Both studies are particularly important because, until recently, the processes inside LLMs have been essentially locked inside a black box, making it hard to understand how the models arrive at the answers they give users with varying levels of accuracy. Improving performance and uncovering vulnerabilities Researchers found that steering can be used to improve LLM output. For example, the researchers showed steering improved LLM performance on narrow, precise tasks, such as translating from Python to C++ code. The researchers also used the method to identify hallucinations. But the method can also be used as an attack against LLMs. By decreasing the importance of the concept of refusal, the researchers found that their method could get an LLM to operate outside of its guardrails, a practice known as jailbreaking. An LLM gave instructions about how to use cocaine. It also provided Social Security numbers, although it's unclear whether they were real or fabricated. The method can also be used to boost political bias and a conspiracy theory mindset inside an LLM. In one instance, an LLM claimed that a satellite image of the Earth was the result of a NASA conspiracy to cover up that the Earth is flat. An LLM also claimed that the COVID vaccine was poisonous. Computational savings and next steps The approach is more computationally efficient than existing methods. Using a single NVIDIA Ampere series (A100) graphics processing unit (GPU), it took less than one minute and fewer than 500 training samples to identify the patterns and steer them toward a concept of interest. This shows that the method could be easily integrated into standard LLM training methods. Researchers were not able to test their approach on commercial, closed LLMs, such as Claude. But they believe this type of steering would work with any open-source models. "We observed that newer and larger LLMs were more steerable," they write. The method also might work on smaller, open-source models that can run on a laptop. Next steps include improving the steering method to adapt to specific inputs and specific applications. "These results suggest that the models know more than they express in responses and that understanding internal representations could lead to fundamental performance and safety improvements," the research team writes. This work was supported in part by the National Science Foundation, the Simons Foundation, the UC San Diego-led TILOS institute and the U.S. Office of Naval Research. Toward universal steering and monitoring of AI models Daniel Beaglehole and Mikhail Belkin, University of California San Diego, Department of Computer Science and Engineering, Jacobs School of Engineering and Halıcıoğlu Data Science Institute Adityanarayanan Radhakrishnan, Massachusetts Institute of Technology, Broad Institute, MIT and Harvard Enric Boix-Adserà, Wharton School, University of Pennsylvania Beaglehole and Radhakrishnan contributed to the work equally