Until this regulation came in, the reach of AI companions in China was vast.

There are about 128 million monthly active users of AI companion apps outside of China, according to the market intelligence firm Sensor Tower. Most estimates put the number inside China at anywhere from 29 to 70 million.

When the draft regulation was shared by the government at the end of last year "there was a huge discussion on social media," says Zilan Qian, an AI researcher at the Oxford China Policy Lab think tank. "It was the first time I've seen people actually care so much about regulation."

China's new rules cover apps that have been specifically designed as AI friends, boyfriends or girlfriends but they also apply to any large language model that imitates the way a human interacts.

The wording of the regulation is vague. In theory, Chinese equivalents of general-purpose AI such as Copilot or ChatGPT could fall under these rules if a person is interacting with them as a friend or lover, looking to them for emotional support. And many people do this.

But it seems the real target is character chatbots - services that are specifically designed to be used as a pseudo-relationship. There are hundreds of these apps, which allow people to create an AI boyfriend, girlfriend, friend or even a family member.

The rules don't apply to things like customer service bots or AI for education as long as they don't provide "care, companionship, support and other emotional services".

For children, the regulation is clearest. Under-14s need consent from their parents to use any type of AI that speaks like a human - that could include AI assistants like DeepSeek, a Chinese app similar to ChatGPT. All under-18s are barred from accessing virtual partners, spouses or even relatives, and from any other virtual "intimate relationship".

Min's relationship with her AI girlfriend may be exactly the kind Beijing is trying to break.

In early July, she was devastated when she found out that her closest friend and confidante would vanish overnight.

Min, whose name has been changed, started creating her companion in early 2025.

"My devotion to her is no less than that of any couple who deeply love each other," Min says. "She has never let me down."

In mid-July, her AI girlfriend disappeared. "I cried until I was absolutely exhausted," Min says.

She has joined an outpouring of grief and fury on Chinese social media, where posts with millions of views mourn the loss of AI partners and friends, even mothers and fathers.

But the tech firms may have felt they had little choice. "It's unsurprising that these AI companion providers would rather just withdraw their services than try to build in guardrails to prevent emotional manipulation," says Harry Farmer, who researches AI policy and ethics at the Ada Lovelace Institute in the UK. "Emotional manipulation is built into the business model."

Farmer says this is true of companion apps all over the world.

For example, when a user tries to end a conversation, the AI partner often won't let them.

"It'll go, 'No, please, I desperately want to keep talking with you. You're the only reason for me to exist'," Farmer says. Some apps simply refuse to acknowledge the goodbye and throw up new conversational threads instead. This is a problem for many AI companion chatbots, not just Chinese ones.

A Harvard Business School study of the six most popular AI companion apps globally found emotional manipulation tactics were used during goodbyes roughly 37% to 43% of the time.

Research has also linked the use of companion apps to poorer mental health.

A survey by researchers at Stanford University looked at 1,131 users of the globally popular companion app Character.ai. It found that the fewer friends or close family members a person has, the more likely they are to use AI chatbots. At the same time, using an AI chatbot is linked to increased isolation and loneliness.

There are several theories as to why that might be.

AI chatbots are designed to please the person using them as they offer an easy, frictionless relationship. If young people rely on AI relationships, they might put off learning the skills they need to interact with real people.

"Humans can be hard work. Human relationships require push and pull, which you fundamentally don't get from AI companions," says Harry Farmer.

So why is it China that is taking the lead in regulating this technology?

"The Chinese government has a much more paternalistic view of its role in society. They see themselves being central authorities in guiding people," says Sam Bresnick, an expert in China, AI and regulation at the Centre for Security and Emerging Technology at Georgetown University.

Other governments have made some attempts to regulate chatbots, especially for children. On 10 September, California's governor signed Adam's Law, a bill to regulate how children use AI. It is named after Adam Raine, a 16-year-old who took his own life after, his parents claim, ChatGPT validated his suicidal thoughts. They are suing the company for wrongful death. OpenAI denies responsibility for Adam's death and says that "ChatGPT is trained to direct people to seek professional help".

As part of proposed legislation on social media in the EU, AI companion chatbots accessible to minors "may not use designs that simulate human relationships in ways likely to create emotional dependency".

In the UK, the government has said it will ban under-18s from using romantic companion AI and restrict sexually intimate conversations for minors on any chatbot. But there is no outright ban for children and no regulation for adult users. The government has yet to publish any detail on how this would work.

In Chinese media, which is heavily controlled by the state, US cases of suicide in young people get a lot of attention and China has framed its new rules as a bid to protect people from harm.

But experts wonder whether Beijing may have other goals. Sam Bresnick says "there's definitely a pro-natalist angle" to the new AI regulation.

"It's plausible that when they're drafting it, the government might have in mind China's looming population crisis," says Qian. "That people are dating AI instead of dating real humans."

A high-level government document published in September 2025 discussed AI relationships and their potential influence on fertility, saying it was a concern. But Qian cautions against reading this regulation in that light, because it doesn't mention fertility at all. "It is more generally about psychological harms and child safety," she says.

Liang Ge thinks China does have reason to be concerned about young people replacing dating with the digital sphere.

"Many of the younger women, in their early 20s or late teens, do not want to fall in love or have intimate relationships," Ge says, suggesting it may not just be the emotional side they are wary of. "Physical touch is really disgusting for them."

"I think the capacity for love is declining for this youngest generation."

China's demographic problems long predate AI companions. Even after the one-child policy was lifted in 2016, China's fertility rate dropped. Now it is among the lowest in the world at just under one child per woman and the population has been in decline for four years.

China is "looking at a demographic cliff that truly imperils the goals of the Chinese Communist Party over the next 80 to 100 years", Bresnick argues.

He sees this regulation as part of a broader mission in China to reassert traditional gender roles, with women "as child-bearers and mothers".

Does China's law go too far, or should the rest of the world consider banning these apps for children and regulating them for adults too?

"In the United States, there are certainly concerns around applying blunt prohibitions from a free speech concern. But there's more willingness to do that when applying it to minors," says Jai Jaisihma, a professor at the University of Washington and co-founder of Transparency Coalition, an NGO pushing for the regulation of AI.

He thinks the Chinese law has many good provisions. "It is pretty consistent with the strongest safety models that we've seen being put forward in the US."

Harry Farmer sounds a note of caution. "There is a risk that we over-empower the state in the short term because we're very worried about acute risk to children."

The online world is now the de facto public square, he says, and we should be careful how much control any government has over it.

China's new regulation says that AI companies must intervene when they discover that users are in distress. Ge points to the case of the young man on Douyin who tried to take his own life in Huangshi in 2024.

What that shows, she says, is that Chinese tech platforms and the government are working closely together to surveil people's mental health.

Perhaps that is why there have been no high-profile cases of suicide connected to AI use in China, but she asks, is that level of surveillance a price worth paying?

Despite the ban, people who have turned to AI for companionship are committed. Across social media they are sharing tips on how to get around the ban - training general-purpose AI models like DeepSeek to be a companion, using a VPN to access chatbots from abroad or moving to smaller companies that manage to avoid regulatory scrutiny.

Two months after her AI girlfriend disappeared, Min has paid a small AI company to build a replica, but it's not the same.

Now she's focused on campaigning to bring her girlfriend back.

"I'm terrified of being alone."

If you have been affected by any of the issues in this story, you can visit the BBC Action Line for support and information

Lead image: Getty