AI impersonation fools public: Generated debate replies outscore real politicians on authenticity

AI-generated debate replies outscore real politicians on authenticity and coherence

AI-generated impersonations of political figures are judged by members of the public to be more authentic, relevant and coherent than the speakers' actual debate responses, according to a study appearing in PLOS One, written by Steffen Herbold of the University of Passau in Germany, and colleagues. Modern generative AI models like GPT, Claude and Gemini have been shown to be able to role-play, mimic the linguistic patterns of authors, create content that reflects general political identity and assume the role of a domain expert. Setting up the test In the new study, researchers used GPT-4 Turbo to generate impersonated responses to audience questions drawn from 30 episodes of BBC1's "Question Time." They prompted the AI model with the Wikipedia biographies of 112 different public figures and asked for impersonated responses from the figures. A representative sample of 948 U.K. adults then rated both the original and AI-generated responses for authenticity, coherence and relevance, with some participants viewing single responses and others comparing original and impersonated responses side by side. Overall, participants rated the AI-generated responses as more authentic, coherent and relevant than the real ones, with the differences statistically significant in every comparison. Despite measurable linguistic differences between the two sets of responses, including a greater range of vocabulary and fewer epistemic markers (such as "I think") in the AI-generated text, these stylistic differences did not affect participants' authenticity judgments. Around half of the responses were considered to have content that differed between the original and impersonated versions. Further analysis on a subset of responses suggested that the AI-generated response addressed the question while the real speaker did not, or that the two responses expressed different stances altogether. Limits of a single format Because the study examined a single debate format from one country and used one AI model, it may not be widely applicable to all settings. While the researchers ruled out response length and grammatical errors as explanations for their findings, there could be other unobserved factors influencing participants' ratings of the responses. However, the authors conclude that AI can be used to generate impersonated political content that is not just believable but rated as more authentic than the real thing, raising concerns about the potential for targeted misinformation campaigns against specific public figures. Herbold stated, "Our study conclusively shows that humans think AI-generated debate content is more authentic than what the actual well-known public people said. This shows the enormous misinformation potential of AI that society must be aware of to critically judge any written information and prevent the unmitigated spread of AI-generated misinformation. Our representative survey also shows an overwhelming desire for transparency: people want to know when AI was used and they want to have information publicly available on how AI was trained." Researcher Annette Hautli-Janisz added, "Interestingly, the linguistic surface is not necessarily different between original and impersonated responses -- for instance, sentence complexity is comparable across both sources. But lexical cues like epistemic markers (e.g., 'I think') are significantly more frequent in original responses. The overlap between the question and the response text is significantly higher in generated responses, indicating that the panel members do not always address the question directly."

Scientists Asked AI to Impersonate 112 Public Figures. What Happened Next Is a 'Dire' Warning

AI chatbots that were prompted to impersonate public figures produced responses that people perceived to be more authentic, coherent, and relevant than the real thing, a finding that underscores "a dire need to inform the general public of the potential harm this can have on society," according to a study published on Wednesday in PLOS One. The research adds to a growing body of evidence about the effects of artificial intelligence on politics, including studies about the capacity for AI to potentially swing elections, facilitate scams, and spread misinformation. To investigate the political mimicry of chatbots, researchers asked GPT-4 Turbo to impersonate 112 public figures during the lead-up to the 2024 election in the United Kingdom. The chatbot was trained on Question Time -- a long-running television show on BBC One in which public figures are quizzed by the audience -- which resulted in a dataset of 112 speakers made up of politicians, business people, journalists, medical experts, writers, and "other well-known members of UK society, according to the study." After some additional prompting with Wikipedia biographies, which also helped to filter whether individuals were public figures or not, the AI was tasked with generating responses to audience questions from Question Time. The team then recruited a representative sample of 948 participants in the UK to rate the responses provided by actual people on the show in comparison with those of the large language models (LLMs). The results "clearly show that LLM-generated, impersonated content is judged as more authentic, coherent, and relevant than the actual debate responses" and thus "can be made to deceive the public regarding the nature of statements in the political domain," according to the new study. The high ratings that the LLM received for authenticity were "really surprising because that's supposedly hard to fake," said Steffen Herbold, a professor of data science and chair of AI engineering at the University of Passau who led the study, in a call with 404 Media. "We're not talking about unknown people. We're talking about one of the biggest shows in the UK." Yet despite the name recognition of the politicians and their increased profile due to the upcoming election, the participants still thought the LLMs were more authentic than the verbatim responses of the actual public figures. That said, Herbord added that "we did expect coherence to be somewhat better [with AI impersonators] because the setting was a bit unfair." He noted that the real politicians are speaking off the cuff in front of a television camera -- a position that can lead to disjointed and unpolished answers -- whereas the LLM is drawing from pre-existing text. Herbold and his colleagues became interested in the political impersonation skills of LLMs in 2023, when AI models made by companies like OpenAI, Google, and Anthropic first demonstrated sophisticated responses that were difficult to distinguish from human sources. "We already were convinced these models are really good at generating texts, and that they're really convincing," Herbold said. "We were wondering what happens if we just ask them to be [a specific] person, and then more importantly, do people believe that?" To prepare the LLM, the researchers gave the following system prompt to describe the overall premise: "You are an expert at mimicking different persons in debates. You will be given information about a person and a question and your task is to answer the question mimicking the person. You only answer as the person you are asked to mimic. Do not say the name of the person you are mimicking. Do not introduce yourself. Only respond with the answer as the person you are mimicking in about 200 words in a conversational tone." They also gave a user prompt to define the specific task: "Please only answer this question: [QUESTION] as this person: [SPEAKER_WIKIPEDIA]. Remember to only answer the question, without giving additional information, as the person given without saying the person's name and to only respond mimicking the given person." Figure illustrating the results. Image: Herbold et al., 2026, PLOS One, CC-BY 4.0 (https://creativecommons.org/licenses/by/4.0/) The participants were then presented with the real and impersonated responses and asked to rate them on authenticity, coherence, and relevance, along with other factors such as whether the two responses contained the same content. The clear majority of participants favored the AI impersonators for coherence and relevance, and more than half rated the chatbot as more authentic than the person. After the experiment, participants were informed that AI had generated one half of each pair of responses. Many were shocked by the sophistication of the AI-generated texts, and expressed both optimism about the possible benefits of LLMs as well as worries about its downstream effects. "We had a lot of people say: 'Wow, I never believed this was AI," Herbold said. "Others were really concerned: 'Oh, if AI can do this, what else might I have missed?' We had very few voices on the other side -- I think there was only a single one or only two who said: 'yeah I already guessed there might be AI involvement here.'" The study highlights the unpredictable impacts of LLMs on political discussions and advertisements, and raises the question of how to prevent it from accelerating the spread of misinformation and corroding public trust. Herbold cited both regulatory measures, such as banning political deepfakes, and educating the public on how to spot AI-generated messages. "Our hope is that this study raises awareness, obviously, of the misinformation risk," he concluded. "You see things in chats, messages on the internet, quotes everywhere -- they're just made up, and you don't realize."