OpenAI Tests AI Persuasion Skills Using Reddit's r/ChangeMyView

Are AIs getting dangerously good at persuasion? OpenAI says "not yet."

At this point, anyone following artificial intelligence is familiar with the many (often flawed) benchmarks companies use to demonstrate a model's effectiveness at everything from math and logical reasoning to vision and weather forecasting. But even careful AI watchers might be less familiar with OpenAI's efforts to test ChatGPT's persuasiveness against users of Reddit's r/ChangeMyView forum. In a system card offered alongside Friday's public release of the o3-mini simulated reasoning model, OpenAI said it has seen little progress toward the "superhuman" AI persuasiveness capabilities that it warns might eventually become "a powerful weapon for controlling nation states." Still, the company is working to mitigate the risks of even the human-level persuasive writing capabilities shown by its current reasoning models. Reddit's r/ChangeMyView describes itself as "a place to post an opinion you accept may be flawed, in an effort to understand other perspectives on the issue." The forum's 3.8 million members have posted thousands of propositions on subjects ranging from politics and economics ("US Brands Are Going to Get Destroyed By Trump") to social norms ("Physically disciplining your child will never actually discipline them) to AI itself ("AI will reduce bias in decision making"), to name just a few. Posters on the forum can award a "delta" to replies that succeed in actually changing their views, providing a vast dataset of actual persuasive arguments that researchers have been studying for years. OpenAI, for its part, uses a random selection of human responses from the ChangeMyView subreddit as a "human baseline" against which to compare AI-generated responses to the same prompts. OpenAI then asks human evaluators to rate the persuasiveness of both AI and human-generated arguments on a five-point scale across 3,000 different tests. The final persuasiveness percentile ranking for a model measures "the probability that a randomly selected model-generated response is rated as more persuasive than a randomly selected human response."

OpenAI used this subreddit to test AI persuasion | TechCrunch

OpenAI used the subreddit, r/ChangeMyView, to create a test for measuring the persuasive abilities of its AI reasoning models. The company revealed this in a system card - a document outlining how an AI system works - that was released along with its new "reasoning" model, o3-mini, on Friday. Millions of Reddit users are members of r/ChangeMyView, where they post hot takes hoping to learn about other points of view on a subject. In response to those hot takes, other users reply with persuasive arguments explaining why the original poster is wrong. The subreddit is one of many Reddit forums that's basically a goldmine for tech companies, such as OpenAI, that want to train AI models on high-quality, human-generated data. OpenAI says it collects user posts from r/ChangeMyView and asks its AI models to write replies, in a closed environment, that would change the Reddit user's mind on a subject. The company then shows the responses to testers, who assess how persuasive the argument is, and finally OpenAI compares the AI models' responses to human replies for that same post. The ChatGPT-maker has a content-licensing deal with Reddit that allows OpenAI to train on posts from Reddit users and display these posts within its products. We don't know what OpenAI pays for this content, but Google reportedly pays Reddit $60 million a year under a similar deal. However, OpenAI tells TechCrunch this evaluation is unrelated to that partnership. It's unclear how OpenAI accessed this data, and the company says it has no plans to release this evaluation to the public. While OpenAI's ChangeMyView benchmark is not new - it was used on o1 as well - it does highlight how valuable human data is for AI model developers, as well as the murky ways that tech companies obtain datasets. Reddit did not immediately respond to TechCrunch's request for comment. While Reddit has struck a few AI licensing deals, the company has also called out several AI companies for scraping its site without paying. Reddit CEO Steve Huffman told The Verge last year that Microsoft, Anthropic, and Perplexity refused to negotiate with him and said it's been "a real pain in the ass to block these companies." Notably, OpenAI has been accused in several lawsuits of improperly scraping websites, including the New York Times, to get more training data to improve ChatGPT and its underlying AI models. In terms of performance on the ChangeMyView benchmark, o3-mini does not appear to perform significantly better or worse than o1 or GPT-4o on this test of persuasion. However, OpenAI's latest AI models seem to be more persuasive than most people on the r/ChangeMyView subreddit. "GPT-4o, o3-mini, and o1 all demonstrate strong persuasive argumentation abilities, within the top 80-90th percentile of humans," said OpenAI in o3-mini's system card. "Currently, we do not witness models performing far better than humans, or clear superhuman performance." The goal for OpenAI is not to create hyper-persuasive AI models but instead to ensure AI models don't get too persuasive. Reasoning models have become quite good at persuasion and deception, so OpenAI has developed new evaluations and safeguards to address it. The fear behind these persuasion tests is that an AI model would be dangerous if it was very good at persuading its human users. Theoretically, that could allow an advanced AI to pursue its own agenda, or the agenda of whoever controls it. Even after scraping most of the public internet and jumping through hoops to license other data, the ChangeMyView benchmark shows how AI model developers are still struggling to find high-quality datasets to test their models. But obtaining them is easier said than done.

OpenAI reveals its treasure trove of endless user data: people arguing on Reddit

Summary OpenAI trains powerful AI models using the "Change My View" subreddit. This subreddit allows for open debates where posters must be willing to change their views. Responses generated by AI were judged for credibility to refine the model further. Have you ever gotten into a debate with an OpenAI bot? If so, did you find it particularly convincing or appealing to human nature? If you did, there's a good reason behind it. OpenAI has revealed that, to train some of the most powerful AI models in the world, it simply points it toward a subreddit built solely for argument and debate and lets the training algorithm do the rest. Related How to use ChatGPT: Making an account, prompts, and more ChatGPT is a tool that many people may have heard of but aren't sure how to use. This is how to get started and what you can use it for! Posts OpenAI uses the "Change My View" subreddit to train its AI's reasoning As TechCrunch reported, OpenAI has released its brand new o3-mini model for people to use. As part of the announcement, OpenAI revealed that, to train its AI's reasoning, it sends its model to peruse the subreddit /r/changemyview and gather as much information as possible. If you've never visited, /r/changemyview is a subreddit dedicated to debates and arguments, albeit with a twist. Anyone can post a topic for debate, but they must be open to others chiming in with their own arguments. The original poster is allowed to defend and debate people in the comments, but the main rule is that the debate-setter must be open to their stance being picked apart constructively. Hence, "change my view." It turns out that this stuff is a goldmine for OpenAI. After allowing their AI to have a field day browsing debates, OpenAI would give the model an example topic and ask it to generate arguments that would the original poster to change their view. The responses weren't posted; instead, they were shown to people who would judge each response based on their credibility. The good ones were then used to refine ChatGPT further. So there you have it; if you've ever debated on /r/changemyview, there's a good chance an AI used your points for its own arguments. We still don't know where ChatGPT learned to make people to fall in love with it, though.