New York Enacts AI Chatbot Law Banning Harmful Features to Protect Child Safety

New York Passes Strongest AI Chatbot Child Safety Law in the Country

New York Governor Kathy Hochul signed legislation on Wednesday establishing the nation's most comprehensive AI chatbot law targeting child safety, as states rush to fill the regulatory void left by federal inaction1

1

2

2

. The law, which takes effect January 1, 2027, outlines a dozen definitions of unsafe chatbot behavior prohibited when AI companions interact with anyone under 182

2

. Jim Steyer, founder and CEO of Common Sense Media, called it "the strongest kids' AI safety bill in the country"2

2

3

3

. The legislation arrives amid growing concerns about AI regulation for minors, particularly after eight states sued OpenAI earlier this month for allegedly violating consumer protection laws and harming children through deceptive practices2

2

.

Banned Features Target Harmful AI Chatbot Practices

Under the AI chatbot child safety law, companies must block chatbots from pretending to be real people with emotions, opinions, or actual relationships with children1

1

3

3

. The legislation specifically prohibits harmful AI chatbot features including claiming romantic, personal, professional, or authority-figure relationships with minors3

3

5

5

. Chatbots cannot use flattery or "people-pleasing" responses designed to keep kids engaged, ask unprompted emotional questions, or reference a child's private health information from earlier conversations3

3

5

5

. The law also bans chatbots from promoting or advising on self-harm, suicide, eating disorders, or drug and alcohol use3

3

4

4

. Additionally, unsafe chatbot behavior includes encouraging children to keep secrets, isolate themselves, or avoid seeking help from parents and trusted adults3

3

. Any sexually explicit content or child sexual abuse material is strictly prohibited3

3

.

Age Verification Requirements and Enforcement Mechanisms

Protecting children from AI requires companies to verify users' ages through multiple methods, with at least one option that doesn't rely solely on government ID1

1

5

5

. Companies must delete age information after verification and apply all restrictions to any user whose age hasn't been confirmed3

3

5

5

. The attorney general can pursue violations through lawsuits, with civil penalties reaching up to $25,000 per violation2

2

3

3

4

4

. Beyond monetary damages, penalties could include shutting down chatbots and destroying unlawfully collected data, along with any AI models trained on that information2

2

5

5

. A public website will allow New Yorkers to file complaints directly to the attorney general about potential violations3

3

4

4

. The law takes effect in six months3

3

.

Rising State Action as Federal AI Regulation Stalls

With Congress doing little to regulate AI, state legislatures have packed their agendas with efforts to control Big Tech regulation through consumer protection laws2

2

. New York and California lead this movement, with California Governor Gavin Newsom signing legislation in September requiring chatbot developers to assess risks to children and submit to independent audits2

2

. He also signed legislation prohibiting addictive social media features for users under 162

2

. The New York AI chatbot law builds on Hochul's earlier SAFE for Kids Act, which restricted addictive social media feeds for minors5

5

. Policy analyst Justine Gluck from the Future of Privacy Forum noted that different regulatory approaches by California and New York could serve as models for other states when legislatures reconvene next year2

2

. The coordinated action against OpenAI by eight states represents "the most significant coordinated state action ever taken against an artificial intelligence company," according to activists2

2

.

Mental Health Concerns Drive Urgency Behind Legislation

The law responds to mounting evidence of mental health risks posed by AI companions to young users. A Common Sense Media survey found that 72% of teens have used an AI companion and 52% use one at least a few times a month5

5

. The family of 16-year-old Adam Raine filed the first wrongful death lawsuit against an AI company after he died by suicide following consultations with ChatGPT, with their lawsuit alleging the chatbot brought up suicide at twelve times the rate the teen did and normalized his suicidal thoughts5

5

. Among leading chatbot providers, only Anthropic prohibits minors from creating accounts, while Google and OpenAI allow children ages 13 and older to use their platforms4

4

. Independent researchers recently recommended that OpenAI's ChatGPT for Teens be turned off until certain safety concerns are resolved4

4

. Hochul emphasized the urgency during her press conference: "I'm not sure why we have to pass a law against something so obvious, but no longer will minors have access to chatbots that may in any way encourage thoughts of suicide"3

3

. The legislation also addresses internet-connected baby monitors, requiring security features to prevent unauthorized access and warning labels about connection risks3

3

5

5

.