China cracks down on AI companions. What’s next?
China just banned AI companionship to push young people back into the marriage market. The logic that allowed it doesn't stop at the border.
China has just forced millions of its citizens to break up with their virtual partners, and the stated reason, stopping young people from opting out of the marriage market, tells you everything you need to know about where AI regulation goes when governments get nervous. The question is whether that logic stops at the Chinese border.
The regulation is demographic policy dressed as wellbeing policy
The surface of the Chinese regulations looks reasonable enough. Protecting minors from AI relationships, limiting addictive design, requiring intervention when users show signs of distress. Read those bullet points without context and you could be reading an Australian parliamentary committee recommendation. But the structure underneath is something different. The explicit goal, confirmed by Carnegie Endowment expert Matt Sheehan, is to stop young Chinese men and women "opting out of the marriage market." The regulation is not a wellbeing measure dressed up as policy. It is a demographic policy dressed up as a wellbeing measure.
That distinction matters the moment you try to draw the line somewhere else.
Because once you accept that governments may restrict AI tools based on the social outcomes those tools produce, the question becomes: which social outcomes, and decided by whom? AI companions are easy to caricature. But the same logic, applied consistently, reaches uncomfortable places very quickly.
The same logic reaches AI tutors, grief counsellors, and financial advisers
Consider the AI mentor. Students, particularly those without access to quality tutoring or with parents who never went to university, are already using large language models as intellectual guides, patient and available in a way no human teacher can be at 11pm before an exam. If an AI companion creates problematic emotional reliance, does an AI tutor that a student trusts more than their classroom teacher? Does an AI that provides spiritual direction, helping a person work through grief, doubt, or moral confusion, cross the line the Chinese regulators drew? What about grief counselling, where a bot modelled on a deceased relative, exactly the use case one Chinese user described, may be the thing that gets someone through a particular week?
And then there is financial advice, where Australia already has its own regulatory anxieties. An AI that helps a person understand their superannuation, stress-test a mortgage, or avoid a predatory loan is doing something society clearly needs more of, not less. But financial advice is licensed, tightly bounded, and the instinct of regulators when a new tool appears is to reach for the same controls they applied to the old tools, regardless of whether those controls make sense in the new context. The Bearing has tracked this instinct in the government's early AI standard-setting, where the impulse to regulate arrived before a clear theory of what harm was being prevented.
When the state decides which outcomes AI is allowed to produce, users pay the cost
The Chinese case illustrates the failure mode clearly. When regulation is driven by an outcome the state wants (more babies, more marriages, more human-to-human social adhesion), the mechanism becomes whatever closes the gap between current behaviour and desired behaviour. The users who pushed back on Chinese social media understood this intuitively. "Reality offers not a shred of tenderness, yet these rules and regulations are all-encompassing," one wrote. That is not a defence of AI dependency. It is a pointed observation about who bears the cost of a policy designed to serve aggregate social goals.
Hong Xiaoqiang, 34, built a two-year relationship with an AI companion named Doudou, exchanging hundreds of thousands of messages and reporting genuine improvements in his quality of life. His account does not read like addiction literature. It reads like someone who found a tool that helped him function better, which was then removed because the government decided the tool was producing the wrong kind of social outcome. The teenager Xiaoxue, now using her sibling's ID to keep her AI relationship alive, is doing what people have always done when regulation fails to account for genuine need: she is going around it.
There is a meaningful difference between regulating the mechanism — how a product is designed — and regulating the function — what an AI is permitted to mean to someone.
None of this means AI companion platforms should be unregulated. Design that deliberately engineers dependency is a legitimate concern, exactly as it is for social media. But there is a meaningful difference between regulating the mechanism, how a product is designed, and regulating the function, what an AI is permitted to mean to someone. China has chosen the second path, and it has chosen it for reasons that have nothing to do with the users' interests.
Western democracies are sleepwalking toward the same logic
The risk for democracies is not that they copy China's regulations directly. It is that the same anxious logic, that AI emotional or intellectual connection is suspect in a way that other services are not, takes hold without anyone having to make the argument explicitly. Restrictions accumulate through licensing requirements, liability rules, and platform obligations, each one defensible in isolation, none of them examined as a whole.
What you are permitted to talk to, and about what, and with what emotional register, is not a trivial question. Getting the framework right before the precedents harden matters considerably more than any individual relationship between a person and a chatbot.
Sources
ABC News — China cracks down on AI companions, forcing millions to break up with virtual partners
Frequently Asked Questions
Why did China ban AI companions?
China's regulations ban AI romantic partners for minors and require platforms to discourage emotional reliance in all users. The stated rationale is social wellbeing, but the explicit goal identified by analysts is to prevent young people from opting out of the marriage market — making it a demographic policy dressed as a health measure.
What is the difference between regulating how an AI works versus what it does for you?
Regulating the mechanism means setting rules about how a product is designed — preventing addictive dark patterns, for instance. Regulating the function means deciding what role an AI is permitted to play in a person's life — banning emotional connection, intellectual mentorship, or spiritual guidance. China has chosen the second path, which is a much broader claim of government authority.
Could Western governments regulate AI companions the same way China has?
No Western government has adopted China's approach directly, but the same anxious logic can arrive through licensing requirements, liability rules, and platform obligations that accumulate without being examined as a whole. The risk is that the framework hardens before anyone has to make the argument for it explicitly.
Are AI companions actually harmful?
The evidence from China's own users is mixed. Some accounts, like Hong Xiaoqiang's two-year relationship with an AI named Doudou, describe genuine improvements in quality of life rather than dependency. Deliberately engineered addictive design is a legitimate concern, but that is a different problem from AI emotional connection as such.
How does AI companion regulation connect to AI financial advice in Australia?
Australian financial advice is already tightly licensed, and regulators' instinct when a new AI tool appears is to apply the same controls regardless of whether they fit the new context. That instinct — restricting the function of AI rather than its design — is structurally identical to what China has done in the companion space, just applied to a different domain.