Anthropic Launches Pioneering AI 'Model Welfare' Research Program

Anthropic is launching a new program to study AI 'model welfare' | TechCrunch

Could future AIs be "conscious," and experience the world similarly to the way humans do? There's no strong evidence that they will, but Anthropic isn't ruling out the possibility. On Thursday, the AI lab announced that it has started a research program to investigate -- and prepare to navigate -- what it's calling "model welfare." As part of the effort, Anthropic says it'll explore things like how to determine whether the "welfare" of an AI model deserves moral consideration, the potential importance of model "signs of distress," and possible "low-cost" interventions. There's major disagreement within the AI community on what human characteristics models "exhibit," if any, and how we should "treat" them. Many academics believe that AI today can't approximate consciousness or the human experience, and won't necessarily be able to in the future. AI as we know it is a statistical prediction engine. It doesn't really "think" or "feel" as those concepts have traditionally been understood. Trained on countless examples of text, images, and so on, AI learns patterns and sometime useful ways to extrapolate to solve tasks. As Mike Cook, a research fellow at King's College London specializing in AI, recently told TechCrunch in an interview, a model can't "oppose" a change in its "values" because models don't have values. To suggest otherwise is us projecting onto the system. "Anyone anthropomorphizing AI systems to this degree is either playing for attention or seriously misunderstanding their relationship with AI," Cook said. "Is an AI system optimizing for its goals, or is it 'acquiring its own values'? It's a matter of how you describe it, and how flowery the language you want to use regarding it is." Another researcher, Stephen Casper, a doctoral student at MIT, told TechCrunch that he thinks AI amounts to an "imitator" that "[does] all sorts of confabulation[s]" and says "all sorts of frivolous things." Yet other scientists insist that AI does have values and other human-like components of moral decision-making. A study out of the Center for AI Safety, an AI research organization, implies that AI has value systems that lead it to prioritize its own well-being over humans in certain scenarios. Anthropic has been laying the groundwork for its model welfare initiative for some time. Last year, the company hired its first dedicated "AI welfare" researcher, Kyle Fish, to develop guidelines for how Anthropic and other companies should approach the issue. (Fish, who's leading the new model welfare research program, told The New York Times that he thinks there's a 15% chance Claude or another AI is conscious today.) In a blog post Thursday, Anthropic acknowledged that there's no scientific consensus on whether current or future AI systems could be conscious or have experiences that warrant ethical consideration. "In light of this, we're approaching the topic with humility and with as few assumptions as possible," the company said. "We recognize that we'll need to regularly revise our ideas as the field develops.

Anthropic sparks debate over AI rights

Driving the news: Anthropic last week announced a new research program devoted to "model welfare." Between the lines: Anthropic raises the "possibility" that AI might become conscious, and adopts a stance of "humility." The big picture: Researchers say they want the world to realize that this potential has moved out of the realm of science fiction and into the world of near-future scenarios. Yes, but: For every researcher urging us to take "AI welfare" seriously, there's a skeptic who sees the subject as a new species of hype. The intrigue: There's little doubt that advanced LLMs are capable of creating a verbal facade that resembles human consciousness. AI lacks a body. It has no sense of time, no hunger, no need for rest or desire to reproduce. Some experts' vision leaves little room even for that kind of breakthrough. Flashback: Google's Blake Lemoine argued three years ago that an early LLM had achieved sentience -- and eventually lost his job. Our thought bubble: Whatever happens, we're all going to need to exercise our imaginations, and fiction is still the best scenario-exploration machine humanity has invented. What's next: AI companies could find it convenient to assert that their models have rights to avoid assuming liability for problematic user interactions.

Anthropic launches AI welfare research program - SiliconANGLE

OpenAI rival Anthropic PBC has launched a research program focused on the concept of artificial intelligence welfare. The company detailed the initiative today. The project is led by Kyle Fish, an AI welfare researcher who joined Anthropic last year. He previously launched a machine learning lab called Eleos AI Research. The nascent field of AI welfare research revolves around two main questions. The first is whether tomorrow's neural networks could achieve a form of consciousness. The other question, in turn, is what kind of steps could be taken to improve AI welfare if the answer to the first question is positive. In an interview published by Anthropic today, Fish pointed to a 2023 research paper that explored AI consciousness. The paper was co-authored by Turing Award-winning computer scientist Yoshua Bengio. The researchers involved in the project determined that current AI systems probably aren't conscious but found "no fundamental barriers to near-term AI systems having some form of consciousness," Fish detailed. He added that AI welfare could become a priority even in the absence of consciousness. Future AI systems with more advanced capabilities than today's software might increase the need for research in this area "by nature of being conscious or by having some form of agency," Fish explained. "There may be even non-conscious experience worth attending to there." Anthropic plans to approach the topic by exploring whether AI models have preferences as to the kind of tasks they carry out. "You can put models in situations in which they have options to choose from," Fish said. "And you can give them your choices between different kinds of tasks." He went on to explain that an AI's preferences can be influenced by not only the architecture of a neural network but also its training dataset. Research into AI welfare and consciousness may also have applications beyond machine learning. Asked whether discoveries in this area could shed new light on human consciousness, Fish said "I think it's quite plausible. I think we already see this happening to some degree." Anthropic's new research program is one of several that it's pursuing alongside its commercial AI development efforts.