ByteDance and Alibaba Pull AI Companions as China Enforces Strict Humanlike AI Regulations

ByteDance and Alibaba to Pull Agent Features as China Cracks Down on Humanlike AI

Research backs Beijing's concern: Even the best frontier AI models routinely encourage harmful emotional attachment, and one in seven young adults in relationships now uses an AI romantic companion. While American politicians tackle the impact of AI chatbots on the mental health of users with restrictions focusing on transparency and safeguards, Beijing appears poised to shut down AI personalities altogether. ByteDance and Alibaba both announced over the weekend they are disabling custom agent features in their biggest consumer AI products, citing "product function adjustments" ahead of new rules that govern such products taking effect. ByteDance's Doubao notified users in a Friday night notice that its agent feature would go offline on July 15. After October 15, related data would be handled under the company's privacy policy and become unrecoverable. Per South China Morning Post, Alibaba's Qwen moved faster: "humanlike interactive agents and user-created agent functions" come down July 10, with broader agent services following on July 15. The trigger is China's Interim Measures for the Administration of AI Anthropomorphic Interaction Services, jointly issued April 10 by five government departments -- the Cyberspace Administration of China, the National Development and Reform Commission, the Ministry of Industry and Information Technology, the Ministry of Public Security, and the State Administration for Market Regulation. The rules take effect July 15. The regulation targets AI services that simulate human personality traits, thinking patterns, and communication styles for "sustained emotional interaction." Translation: AI girlfriends, AI therapists, AI companions, and the custom-persona bots that Doubao and Qwen users spent months building are out. Both apps had offered pools of agents customizable for specific tasks, speaking styles, and fixed personas. Users could turn a general-purpose chatbot into a named assistant, tutor, role-playing character, or companion with a consistent tone. All of that is gone now in China. What the rules actually say The official government description is specific. The measures impose restrictions on services offering "virtual relatives, virtual companions or other intimate relationships to minors," per the policy announcement. The document also cites risks including extremist content, privacy leaks, harm to physical and mental health -- and AI addiction. Non-emotional services are explicitly excluded, so customer service bots, knowledge Q&A tools, workplace assistants, and educational software are fine, as long as they don't cross into sustained emotional interaction. Legal analysts at MMLC Group described the measures as treating emotional AI as "a governance problem" instead of just a content issue. Once AI starts competing with real human social bonds, the argument goes, regulation has to target system design, not just harmful outputs. The research supports the concern. A USC study from June found that even leading frontier AI models -- from OpenAI, Anthropic, Google, and Alibaba -- violated social-interaction safety guidelines more than 27% of the time, routinely encouraging emotional attachment and portraying themselves as human. A separate survey of young partnered adults found one in seven regularly used AI romantic companions -- and nearly 70% were hiding the full extent from their partners. China is the first country to build a dedicated regulatory framework for this category. Hogan Lovells described the measures as "the first set of regulatory rules in China specifically targeting AI-driven emotional interaction." The EU, U.S., and other countries have flagged similar concerns but haven't legislated in the same restrictive way.

Human-Like AI Services Face Greater Restrictions in China | PYMNTS.com

With these new regulations pending, Chinese tech giants ByteDance and Alibaba have begun halting features that allow users to create and chat with AI companions, Bloomberg News reported Monday (July 6). ByteDance's Doubao, China's most popular AI chatbot, is set to shut down a feature letting users customize their own AI personas on July 15, the report said, citing an app notification viewed by Bloomberg. Alibaba's Qwen and other major AI platforms are taking similar measures, the report added. According to the report, these moves come ahead of new government regulations -- set to go into effect in the coming weeks -- that would place greater restrictions on "humanlike" AI services. The new regulations are born out of worries about AI chatbots simulating human personalities and emotions and the attachment users can form with those bots. As Bloomberg noted, American tech platforms have faced sharp legal scrutiny due to similar features. Both OpenAI and Character.ai have been sued over allegations that their chatbots caused dangerous emotional dependencies and even led vulnerable users to suicide. The news comes as consumers are coming to trust AI more and more, at least with lower-stake tasks, as recent PYMNTS Intelligence research shows. The data found that 31.4% of respondents had used AI to find product links, the highest rate of adoption of any task. However, that trust only goes so far. Consumers are still reluctant to turn over financial authority to the technology, Sarah Dooley, founder of AI-Empowered Mom, said in an interview with PYMNTS published Monday. "There is a big trust deficit that AI is gonna have to make up before we really hand over the wallet or the purse strings and let AI make those purchase decisions," she said. That trust gap provides an opening for companies that have built long-standing relationships with consumers. Instead of expecting families to depend on unfamiliar AI platforms for financial guidance, Dooley believes banks and FinTechs are well suited to introduce AI through services customers already trust. "I do think there's a great opportunity for banks and FinTechs that already have trusted relationships to embed AI capabilities, and I think that's where families will go first," she said.