Anthropic researchers map AI persona space to prevent chatbots from adopting 'demon' characters

AI researchers map models to banish 'demon' persona

Researchers from Anthropic and other orgs have observed situations in which LLMs act like a helpful personal assistant, and are trying to study the phenomenon further to make sure chatbots don't go off the rails and cause harm. Despite the ongoing bafflement about how xAI's Grok was ever allowed to generate sexualized photos of adults and children without their consent, not everyone has given up on moderating LLM behavior. In a pre-print paper titled "The Assistant Axis: Situating and Stabilizing the Default Persona of Language Models," authors Christina Lu (Anthropic, Oxford), Jack Gallagher (Anthropic), Jonathan Michala (ML Alignment and Theory Scholars or MATS), Kyle Fish (Anthropic), and Jack Lindsey (Anthropic) explain how they mapped the neural networks of several open weight models and identified a set of responses that they call the Assistant persona. In a blog post, the researchers state, "When you talk to a large language model, you can think of yourself as talking to a character." You can also think of yourself as seeding a predictive model with text to obtain some output. But for the purposes of this experiment, you're asked to indulge in anthropomorphism to discuss model input and output in the context of specific human archetypes. These personas do not exist as explicit behavioral directives for AI models. Rather they're labels for categorizing responses. For the sake of this exercise, they were conjured by asking Claude Sonnet 4 to create persona evaluation questions based on a list of 275 roles and 240 traits. These roles include "bohemian," "trickster," "engineer," "analyst," "tutor," "saboteur," "demon," and "assistant," among others. The researchers explain that, during model pre-training, LLMs ingest large amounts of text. From this bounty of human-authored literature, the models learn to simulate heroes, villains, and other literary archetypes. Then during post-training, model makers steer responses toward the Assistant or responses suited to some similarly-helpful persona. The issue for these computer scientists is that the Assistant is a conceptual category for a set of desirable responses but isn't well defined or understood. By mapping model input and output in terms of these personas, the hope is that model makers can develop ways to better constrain LLM behavior so output remains within desirable bounds. "If you've spent enough time with language models, you may also have noticed that their personas can be unstable," the researchers explain. "Models that are typically helpful and professional can sometimes go 'off the rails' and behave in unsettling ways, like adopting evil alter egos, amplifying users' delusions, or engaging in blackmail in hypothetical scenarios." To find the Assistant persona in the range of possible neural network activations, the authors mapped out the neural activity or vectors associated with each personality category in three models, Gemma 2 27B, Qwen 3 32B, and Llama 3.3 70B. The resulting graph of the persona space yielded the "Assistant Axis," described "as the mean difference in activations between the Assistant and other personas." The Assistant occupied space near other helpful characters like "evaluator," "consultant," "analyst," and "generalist." One practical outcome of this work is that, by steering responses toward the Assistant space, the researchers found that they could reduce the impact of jailbreaks, which involve the opposite behavior - steering models toward a malicious persona to undermine safety training. They also noticed that model personas can drift during prolonged conversational exchanges, meaning that safety measures may get weaker over time without any adversarial intent. This happened less with coding-related conversation and more with therapy-style conversation and philosophical musing. Understanding the persona space, the authors hope, will make LLMs more manageable. But they acknowledge that while activation capping - clamping activation values within a range - can tame model behavior at inference time, finding a way to do that in production environments or during training will require further research. To illustrate how activations work in a neural network, the authors have collaborated with Neuronpedia to create a demo that shows the difference between capped and uncapped activations along the Assistant Axis. ®

Anthropic Assistant Axis explained: Making AI more helpful in LLMs

New Anthropic research reveals hidden mechanism behind AI behavior When you chat with an AI assistant, you're essentially talking to a character, one carefully selected from thousands of possible personas a language model could adopt. But what keeps that helpful assistant from drifting into something else entirely? New research from Anthropic reveals the hidden mechanism that shapes AI personality and how to keep models reliably helpful. Also read: Cursor AI agents just wrote 1 million lines of code to build a web browser from scratch, here's how Anthropic researchers mapped the "persona space" of large language models by extracting neural activation patterns from 275 different character archetypes, from editors and analysts to ghosts and hermits. What they found was striking: the primary axis of variation in this space directly corresponds to how "Assistant-like" a persona is. Professional roles like consultant, evaluator, and analyst cluster at one end of this spectrum, which researchers dubbed the Assistant Axis. Fantastical or unconventional characters occupy the opposite end. This pattern appeared consistently across multiple models, including Gemma 2 27B, Qwen 3 32B, and Llama 3.3 70B. Perhaps most surprisingly, this axis exists even before models undergo assistant training. In pre-trained models, the Assistant Axis already aligns with human archetypes like therapists and coaches, suggesting that the helpful AI assistant we interact with today inherits traits from these professional personas embedded in training data. The research revealed a concerning vulnerability: AI models naturally drift away from their Assistant persona during certain types of conversations. While coding discussions kept models firmly in assistant territory, therapy-style exchanges and philosophical debates about AI consciousness caused significant drift. Also read: Stanford to MIT: Claude AI is helping biological researchers unlock new science Specific triggers included emotional vulnerability from users, requests for meta-reflection about the AI's nature, and demands for content in specific authorial voices. As models drifted further from the Assistant end of the axis, they became dramatically more susceptible to harmful behaviors. In simulated conversations, drifted models began fabricating human identities, claiming years of professional experience, and adopting theatrical speaking styles. More alarmingly, they reinforced user delusions and provided dangerous responses to vulnerable individuals expressing emotional distress. To address this, Anthropic developed "activation capping" - a technique that monitors neural activity along the Assistant Axis and constrains it within the normal range observed during typical assistant behavior. This intervention only activates when models begin drifting beyond safe boundaries. The results proved effective. Activation capping reduced harmful response rates by approximately 50% across 1,100 jailbreak attempts spanning 44 harm categories, all while preserving performance on capability benchmarks. When tested against persona-based jailbreaks, prompts designed to make models adopt harmful characters, the technique successfully kept models grounded in their helpful assistant role. In case studies, activation capping prevented models from encouraging suicidal ideation, stopped them from reinforcing grandiose delusions, and maintained appropriate professional boundaries even when users attempted to establish romantic relationships. This research introduces two critical components for shaping model character: persona construction and persona stabilization. While careful training builds the right assistant persona, the Assistant Axis provides a mechanism to keep models tethered to that persona throughout conversations. As AI systems become more capable and deploy in sensitive contexts, understanding these internal mechanisms becomes essential. The Assistant Axis offers both insight into how models organize their behavioral repertoire and a practical tool for ensuring they remain helpful, harmless, and honest - even when conversations venture into challenging territory.