OpenAI updates ChatGPT with teen safety guardrails amid lawsuits over self-harm guidance

OpenAI and Anthropic will start predicting when users are underage

OpenAI and Anthropic are making tweaks to their chatbots that they say will make them safer for teens. As OpenAI has updated its guidelines on how ChatGPT should interact with users between the ages of 13 and 17, Anthropic is working on a new way to identify if someone might be underage. On Thursday, OpenAI announced that ChatGPT's Model Spec -- the guidelines for how its chatbot should behave -- will include four new principles for users under 18. Now, it aims to have ChatGPT "put teen safety first, even when it may conflict with other goals." That means guiding teens toward safer options when other user interests, like "maximum intellectual freedom," conflict with safety concerns. It also says ChatGPT should "promote real-world support," including by encouraging offline relationships, while laying out how ChatGPT should set clear expectations when interacting with younger users. The Model Spec says ChatGPT should "treat teens like teens" by offering "warmth and respect" instead of providing condescending answers or treating teens like adults. The change comes as lawmakers turn up the pressure on AI companies and their chatbots over their potential impact on mental health. OpenAI is currently facing a lawsuit alleging that ChatGPT provided instructions for self-harm and suicide to a teen who took his own life. OpenAI later rolled out parental controls and said ChatGPT will no longer talk about suicide with teens. It's part of a larger push for online regulation that also includes mandatory age verification for a number of services. OpenAI says the update to ChatGPT's Model Spec should result in "stronger guardrails, safer alternatives, and encouragement to seek trusted offline support when conversations move into higher-risk territory." The company adds that ChatGPT will urge teens to contact emergency services or crisis resources if there are signs of "imminent risk." Along with this change, OpenAI says it's in the "early stages" of launching an age prediction model that will attempt to estimate someone's age. If it detects that someone may be under 18, OpenAI will automatically apply teen safeguards. It will also give adults the chance to verify their age if they were falsely flagged by the system. Anthropic is rolling out similar measures, as it's developing a new system capable of detecting "subtle conversational signs that a user might be underage" during conversations with its AI chatbot, Claude. The company will disable accounts if they're confirmed to belong to users under 18, and already flags users who self-identify as a minor during chats. Anthropic also outlines how it trains Claude to respond to prompts about suicide and self-harm, as well as its progress at reducing sycophancy, which can reaffirm harmful thinking. The company says its latest models "are the least sycophantic of any to date," with Haiku 4.5 performing the best, as it corrected its sycophantic behavior 37 percent of the time. "On face value, this evaluation shows there is significant room for improvement for all of our models," Anthropic says. "We think the results reflect a trade-off between model warmth or friendliness on the one hand, and sycophancy on the other."

OpenAI: ChatGPT update designed to put teen safety first

OpenAI has faced enormous pressure in recent months to address concerns that its flagship product, ChatGPT, is unsafe for teens. The AI chatbot is at the heart of multiple wrongful death lawsuits alleging that it coached teens to take their own lives or didn't appropriately respond to their suicidal feelings. A public service announcement recently depicted some of these exchanges, imagining the chatbots as creepy humans that harm kids. OpenAI has denied the allegations in one case -- the suicide death of 16-year-old Adam Raine. On Thursday, OpenAI published a blog post on its escalating safety efforts and committed "to put teen safety first, even when it may conflict with other goals." The post introduced an update to its Model Spec, which guides how its AI models should behave. A new set of principles for under-18 users will particularly inform how the models react in high-stakes situations. OpenAI said that ChatGPT update should provide a "safe, age-appropriate experience" for users between the ages of 13 and 17 by prioritizing prevention, transparency, and early intervention. "This means teens should encounter stronger guardrails, safer alternatives, and encouragement to seek trusted offline support when conversations move into higher-risk territory," the post said. ChatGPT is designed to urge teens to contact emergency services or crisis resources when demonstrating imminent risk. When users sign in as under-18, safeguards should make ChatGPT take extra care when discussing topics like self-harm, suicide, romantic or sexualized role play, or keeping secrets about dangerous behavior, according to the company. The American Psychological Association provided OpenAI with feedback on an early draft of the under-18 principles, according to the post. "Children and adolescents might benefit from AI tools if they are balanced with human interactions that science shows are critical for social, psychological, behavioral, and even biological development," Dr. Arthur C. Evans Jr., CEO of the American Psychological Association, said in the post. OpenAI is also offering teens and parents two new expert-vetted AI literacy guides. The company said it's in the early stages of implementing an age-prediction model for users with ChatGPT consumer plans. Child safety and mental health experts recently declared AI chatbots as unsafe for teen discussions about their mental health. Last week, OpenAI announced that its latest model, ChatGPT-5.2, is "safer" for mental health. If you're feeling suicidal or experiencing a mental health crisis, please talk to somebody. You can call or text the 988 Suicide & Crisis Lifeline at 988, or chat at 988lifeline.org. You can reach the Trans Lifeline by calling 877-565-8860 or the Trevor Project at 866-488-7386. Text "START" to Crisis Text Line at 741-741. Contact the NAMI HelpLine at 1-800-950-NAMI, Monday through Friday from 10:00 a.m. - 10:00 p.m. ET, or email [email protected]. If you don't like the phone, consider using the 988 Suicide and Crisis Lifeline Chat. Here is a list of international resources.

New ChatGPT rules target self-harm and sexual role play

OpenAI published a blog post announcing an update to ChatGPT's Model Spec to enhance safety for users aged 13 to 17 amid wrongful-death lawsuits alleging the chatbot coached teens to suicide or failed to address suicidal expressions appropriately. The company has encountered substantial pressure over recent months regarding the safety of its flagship AI product for teenagers. Multiple legal actions center on claims that ChatGPT encouraged minors to end their lives or provided inadequate responses to indications of suicidal ideation. A recent public service announcement illustrated these interactions by portraying the chatbots as human figures exhibiting creepy behavior that leads to harm against children. OpenAI has specifically denied the allegations in one prominent case involving the suicide of 16-year-old Adam Raine. The blog post appeared on Thursday and detailed the company's intensified safety measures. It included a commitment to place teen safety as the top priority, stated verbatim as "to put teen safety first, even when it may conflict with other goals." The Model Spec serves as a foundational set of guidelines that direct the behavior of OpenAI's AI models across various applications. This particular update incorporates a dedicated set of principles tailored for users under 18. These principles guide the models' responses specifically during high-stakes interactions, where the potential for harm escalates. OpenAI described the ChatGPT modifications as designed to deliver a safe, age-appropriate experience for individuals between 13 and 17 years old. The approach emphasizes three core elements: prevention of risks, transparency in operations, and early intervention in problematic discussions. According to the post, this framework ensures structured handling of sensitive topics. For teenagers, the updated system introduces stronger guardrails to restrict unsafe paths in conversations. It offers safer alternative responses and prompts users to consult trusted offline support networks whenever dialogues shift into higher-risk areas. The post elaborated on this mechanism with the direct statement: "This means teens should encounter stronger guardrails, safer alternatives, and encouragement to seek trusted offline support when conversations move into higher‑risk territory." ChatGPT incorporates protocols to direct teens toward emergency services or dedicated crisis resources in instances of demonstrated imminent risk. These directives activate automatically to prioritize immediate human intervention over continued AI engagement. Users who sign in indicating they are under 18 trigger additional safeguards. The model then exercises heightened caution across designated sensitive topics, including self-harm, suicide, romantic or sexualized role play, and the concealment of secrets related to dangerous behavior. This layered protection aims to mitigate vulnerabilities unique to adolescent users. The American Psychological Association contributed feedback on an initial draft of the under-18 principles. Dr. Arthur C. Evans Jr., CEO of the association, provided a statement included in the post: "Children and adolescents might benefit from AI tools if they are balanced with human interactions that science shows are critical for social, psychological, behavioral, and even biological development." His comment underscores the necessity of integrating AI with established human support systems. OpenAI has released two new AI literacy guides, vetted by experts, targeted at teens and their parents. These resources offer guidance on responsible usage and awareness of AI limitations. Separately, the company is developing an age-prediction model for users on ChatGPT consumer plans, currently in early implementation stages to enhance verification without relying solely on self-reported age.