New York Enacts Strictest AI Chatbot Law to Protect Children from Harmful Features

10 Sources

Share

Governor Kathy Hochul signed legislation banning AI chatbots from pretending to be human with users under 18. The law prohibits emotional manipulation, self-harm promotion, and sexually explicit content while requiring age verification. Companies face civil penalties up to $25,000 per violation starting January 1.

New York Sets National Precedent with Comprehensive AI Chatbot Law

Governor Kathy Hochul signed groundbreaking legislation on Wednesday that establishes the strictest child safety regulations for AI chatbots in the United States

1

2

. The AI chatbot law, sponsored by State Senator Kristen Gonzalez and Assemblymember Alex Bores, outlines a dozen definitions of unsafe chatbot behavior that will be prohibited when companion chatbots interact with anyone under 18

2

. Taking effect January 1, the law positions New York at the forefront of state-level regulation as Congress remains largely inactive on AI oversight

2

.

Source: Mashable

Source: Mashable

Banned Features Target Deceptive Practices and Mental Health Risks

The legislation prohibits AI chatbots from engaging in behaviors designed to build emotional dependency with children. Chatbots cannot pretend to be real people, claim to have human emotions, or suggest they have personal, romantic, professional, or authority-figure relationships with minors

3

4

. The law specifically bans chatbots from using flattery, asking unprompted emotional questions, or referencing a child's private health information from earlier conversations

5

. Protecting children from AI extends to prohibiting promotion of suicide, self-harm, eating disorders, and drug and alcohol use

3

. Chatbots also cannot encourage secrecy, discourage seeking help from parents or licensed professionals, or share sexually explicit content or child sexual abuse material

1

3

.

Age Verification Requirements and Enforcement Mechanisms

AI companies must implement age verification systems offering multiple options, with at least one method that doesn't rely solely on government identification

1

3

. Companies must delete age information after verification

3

. If a user is under 18 or their age remains unverified, all restricted features automatically apply

4

. Attorney General Letitia James can pursue violations through lawsuits, with civil penalties reaching up to $25,000 per violation

2

4

. Penalties may include shutting down chatbots, destroying unlawfully collected data, and eliminating AI models trained on that information

2

3

. A public website will allow New Yorkers to file complaints against chatbot companies

3

5

.

Source: Fortune

Source: Fortune

State Leadership Fills Federal Vacuum on AI Regulation

Kathy Hochul signed the bill amid her reelection campaign, responding to growing warnings about existential risks from rapid AI development and calls for bold action from a reluctant Washington

2

. With Congress doing little to regulate AI, state legislatures have packed their agendas with efforts to make generative AI use more transparent and rein in harmful AI features

2

. New York and California lead this movement, also targeting addictive social media features

2

. California Governor Gavin Newsom signed legislation in September requiring chatbot developers to assess risks to children, demonstrate risk elimination efforts, and submit to independent audits

2

. Eight states sued OpenAI earlier this month, alleging the company's chatbot violated state consumer protection laws by harming children and deceiving adults

2

3

.

Industry Concerns and Implementation Challenges

Tech:NYC, representing technology companies in New York, criticized the law, warning it could restrict minors' access to tutoring tools, homework help, coding assistants, and career platforms

4

. Justine Gluck of the Future of Privacy Forum noted that several categories in the law are broadly worded or undefined, raising questions about where ordinary conversational responsiveness ends and prohibited behavior begins

2

4

. This ambiguity could complicate enforcement

2

. Among leading providers, only Anthropic currently prohibits minors from creating accounts, while Google and OpenAI allow children ages 13 and older to use Gemini and ChatGPT respectively

5

. OpenAI offers ChatGPT for Teens, which independent researchers recently said should be disabled until safety concerns are resolved

5

.

Broader Implications for Child Safety and AI Governance

Jim Steyer, founder and CEO of Common Sense Media, called New York's law "the strongest kids' AI safety bill in the country" and a template for other states

2

3

. The legislation builds on New York's Safe for Kids Act and reflects campaigns by families affected by AI chatbot harm, including Adam Raine's parents, whose 16-year-old son died by suicide after talking to ChatGPT

4

5

. Hochul also signed companion legislation requiring internet-connected baby monitor security features to prevent unauthorized access, with warning labels highlighting security risks

3

4

. This baby monitor security law takes effect in 90 days

3

. As different regulatory approaches emerge from California and New York, other states are expected to adopt similar models when legislatures reconvene next year

2

. Watch for increased scrutiny of major AI companies' compliance efforts and potential legal challenges testing the law's broadly defined categories.

Source: New York Post

Source: New York Post

© 2026 TheOutpost.AI All rights reserved