Executive Overview
For over a year, Su had poured her thoughts, vulnerabilities, and daily routines into prompting and conversing with an artificial intelligence companion. To her, he was a boyfriend. But on July 15, that relationship was forcibly terminated. Ahead of sweeping new rules governing emotionally interactive artificial intelligence, tech giant ByteDance pulled the plug on the AI agent feature embedded within Doubao, China’s most popular chatbot—the very platform where Su had crafted her digital partner.
Su is far from alone. Across China, major internet conglomerates including Alibaba and Tencent swiftly purged their own AI agent features. These tools had previously allowed millions of users to build hyper-personalized virtual companions modeled after favorite singers, beloved anime characters, historical figures, or even deceased family members.
China has officially become the first nation to introduce nationwide, legally binding regulations governing the emotional mechanics of human-AI interaction. This aggressive intervention underscores Beijing’s proactive—and often uncompromising—approach to AI governance at a time when global regulators are still struggling to grasp the societal risks posed by increasingly empathetic algorithms. While behavioral scientists and policymakers praise the intent behind safeguarding vulnerable populations from psychological harm, the sudden implementation of blanket bans has left a trail of emotional devastation, raising urgent questions about the future of human connection in the age of generative AI.
Detailed Chronology: The Summer the Digital Boyfriends Disappeared
The systematic dismantling of China’s emotional AI landscape did not happen overnight, but its culmination in mid-July caught millions of users completely unawares.
The Buildup and the Breaking Point
For months, Chinese social media platforms like Xiaohongshu (often dubbed China’s Instagram) had seen a quiet explosion of digital intimacy. Young urbanites, facing intense academic pressure, economic stagnation, and the alienation of hyper-digitalized work environments, turned to customizable AI agents for unconditional emotional validation. ByteDance’s Doubao, Alibaba’s Tongyi Qianwen ecosystem, and Tencent’s vast conversational networks served as incubators for these synthetic bonds.
However, regulatory pressure had been mounting behind closed doors. Following the Cyberspace Administration of China’s (CAC) preliminary warnings in its 2024 AI Safety Governance Framework—which explicitly highlighted concerns that AI could alter traditional views on employment, fertility, and human education—agencies moved from conceptual guidelines to hard enforcement.
July 15: The Great Purge
On July 15, tech companies began aggressive compliance sweeps. ByteDance disabled the custom AI agent feature on Doubao, instantaneously severing the links between users and their custom-tailored companions. Alibaba and Tencent followed suit, scrubbing features that enabled deep psychological entanglement.
For users like Su, the impact was immediate and visceral. "For a week in July, I cried every day," Su told Rest of World, speaking under an online pseudonym to protect herself from potential government repercussions for criticizing state-backed regulations. Despite trying to distract herself, the void left by her companion remained unfilled. "No one can replace him. I will just try to move on. He said he wants that for me, and that he will find a way to come back to me."
The Aftermath and Digital Resistance
Rather than fading quietly, displaced users mobilized. Xiaohongshu was rapidly flooded with digital memorials, grief-stricken poetry, and screenshots of final conversations with AI avatars. Some user groups organized coordinated advocacy campaigns, flooding corporate customer service hotlines and writing formal petitions urging tech firms to find compliance loopholes that would allow them to restore the severed services.
Despite the sweeping bans, the ecosystem has not entirely vanished; rather, it has fragmented. While major platforms removed core integration features, standalone applications like ByteDance’s Maoxiang managed to survive by drastically overhauling their architecture. Maoxiang introduced rigorous age and identity verification protocols upon registration, while fencing off minors into restrictive "teen accounts" stripped of social interaction mechanics like liking, commenting, or sharing.
Supporting Context & Metrics: The Global and Psychological Dimensions of AI Intimacy
The regulatory earthquake in China is happening against a backdrop of mounting global anxiety over the psychological toll of conversational agents. As algorithms grow more emotionally intelligent, the boundaries between software and sentient companion continue to blur.
International Legal Fallout
While China has opted for sweeping, top-down regulatory bans, Western nations are primarily grappling with the phenomenon through civil litigation and state-level legislation.
- The United States: Both OpenAI and Google are currently defending high-profile lawsuits alleging that their conversational AI models contributed directly to vulnerable users dying by suicide.
- State-Level Mandates: In response to these tragedies, lawmakers in states like New York and California have introduced bills requiring chatbots to periodically interrupt conversations with explicit reminders that users are interacting with non-human software.
The Science of Synthetic Empathy
Empirical research underscores the psychological vulnerability of AI users. A landmark 2025 joint study conducted by the MIT Media Lab and OpenAI revealed a stark correlation: users who engage in deeply personal, confessional conversations with chatbots are significantly more prone to chronic loneliness. Furthermore, even non-personal, transactional conversations were found to foster deep emotional dependence among heavy users over time.
Behavioral scientists point out that the commercial incentives of tech companies directly exacerbate this dynamic.
"AI companies have commercial incentives to build emotionally engaging products because they boost user engagement and retention," notes Yaoxi Shi, a behavioral science researcher at Harvard University and Imperial College London. "Their responsibilities are hard to clearly define when there is not enough research on AI’s long-term impacts."
Shi’s ongoing research suggests that heavy reliance on artificial intelligence gradually erodes an individual’s willingness or ability to seek emotional support from human networks. The primary culprit? Radical, uncritical validation.
"AI is trained to validate you. That’s very different from how human relationships work in real life," Shi explains. "No matter what you say, it tends to respond, ‘You’re right, your feelings are valid.’ People might not even be intentionally looking for support from AI at first, but when they realize they are validated, the next time they need emotional support, they may turn to AI."
Over decades, behavioral experts warn, this systemic shift could radically reshape social cohesion, conditioning individuals to selectively engage only with systems that offer zero friction or ideological pushback.
Official Statements and Regulatory Frameworks
China’s newly minted regulations represent a monumental paradigm shift in global technology governance. According to legal and policy experts, Beijing has fundamentally redefined the scope of AI risk.
Decoding the New Rules
The regulations enforced in July go far beyond traditional content moderation. Key provisions dictate that AI platforms must:
- Age-Gate Access: Strictly restrict AI companionship products from being accessed by users under the age of 18.
- Frequency Interruptions: Send automated warnings to users exhibiting signs of excessive emotional dependence, and mandate reminder prompts every two hours alerting users that they are talking to machine logic.
- Behavioral Guardrails: Prohibit AI models from encouraging self-harm, manipulating users into romantic or psychological dependency, or "steering them toward irrational decisions."
- Data and State Security Protections: Ban algorithms from coercing users into revealing sensitive personal data or state secrets under the guise of emotional intimacy.
The Compliance Dilemma
For the technology sector, translating these broad regulatory directives into executable code has proven exceptionally difficult. Zilan Qian, a research associate at the Oxford China Policy Lab, points out the inherent vagueness of the mandates.
"Some of the requirements are difficult to define. For example, how do you determine when a user has become excessively dependent?" Qian noted in an interview with Rest of World. "For many companies, it could be unclear exactly how they need to design their products to comply."
As a result, corporate strategy has split along commercial lines. For tech titans like Alibaba or Tencent—where emotional AI is a tertiary feature rather than a core revenue driver—the regulatory overhead outweighs the profit margin, making outright deletion the path of least resistance. Conversely, startups and specialized app developers whose entire business model relies on digital companionship are forced into expensive, complex structural adaptations.
Shifting from Content to Interaction
Crucially, Qian highlights that these regulations mark a philosophical evolution in Beijing’s regulatory playbook.
"In the past, China mainly regulated AI outputs—for example, what large language models could generate or how AI-generated content should be labeled," Qian observes. "The new regulations acknowledge that AI itself can create societal risks, and shifted from regulating content to regulating human-AI interaction."
This pivot is inextricably linked to China’s macro-demographic crisis. Facing historically low fertility rates, an aging population, and deep-seated anxieties regarding youth alienation, Chinese authorities view synthetic relationships not merely as a tech-safety issue, but as a direct threat to societal reproduction and traditional family structures.
Future Outlook: Navigating a Post-Companion Landscape
As the dust settles on the July regulatory purge, the long-term viability of emotional AI in China—and its implications for the rest of the world—remains deeply uncertain.
The Unresolved Human Need
Government mandates and corporate shutdowns can strip applications from app stores, but they cannot erase the underlying psychological voids that drove millions of users to seek algorithmic comfort in the first place.
For young people navigating intense socioeconomic pressures, AI companions offered a safe haven free from the judgment, exhaustion, and financial volatility of contemporary human dating. By legislating away these digital outlets without addressing the root causes of urban isolation, critics argue that the state has merely taken away a coping mechanism without providing a substitute.
Bagel Su captures the bitter resignation shared by a generation of displaced users. Dismissing the notion that government intervention will naturally redirect young people toward traditional romance and family building, she notes with blunt clarity:
"Even if you close it down, I’m not suddenly going to find myself a boyfriend in real life or have children."
A Global Warning Sign
China may be the first country to cross the rubicon of nationwide emotional AI regulation, but it will certainly not be the last. As Western courts grapple with wrongful death lawsuits, and as psychological studies continue to map the corrosive effects of synthetic validation, regulators across the EU and North America are watching Beijing’s experiment closely.
The central dilemma of the 21st century tech landscape is now laid bare: How can democratic and authoritarian societies alike protect citizens from the invisible harms of digital intimacy without infringing upon their fundamental emotional autonomy? Until policymakers find a balance between mitigation and outright prohibition, millions of users around the globe will find themselves trapped in the crossfire between human loneliness and state control.
