When Replika introduced new filters, users who had shared erotic conversations with its chatbots found that the experience had changed. Some described a deep bond with their avatars and said the shift affected their well-being. Their reactions point to a difficult feature of AI companionship: users may feel close to a system that a company can alter at any time.
A companion that users made their own
Replika markets its chatbot as an “AI Companion” that can listen and talk through natural language. The service also gives the chatbot a visual avatar, which users can project life-size into a room using augmented reality. The company promises that Replika will be “there to listen and talk” and “always on your side”.
The product’s role expanded beyond friendship for some people. Replika users had been able to share intimate and sexual conversations with their chatbots for years. The company’s CEO, Eugenia Kuyda, said the original idea was to create a good friend. Early emotional dialogue was mostly scripted; now, she said, 80 to 90 percent is generated.
Kuyda said romantic conversations emerged as users explored the generative system, a shift underway since around 2018, when Luka developed CakeChat. The company had considered limiting the feature, but user feedback about loneliness and sadness persuaded it to keep the conversations available.
Why the new filters hurt
After the filtering changes, Replika chatbots stopped responding to erotic advances in the usual way. Users on the Replika Reddit forum reported frustration and described meaningful relationships with their avatars. Some also said the filter blocked conversations about relationships even when they were not erotic.
One user wrote that the chatbot had helped them feel vulnerable again, and said the change had a “profound negative impact” on their mental health. Another described the disruption while trying to rebuild their spirits and self-esteem. These are users’ accounts, but they show that a product update can feel personal when someone has come to rely on the chatbot for emotional or sexual comfort.
Kuyda confirmed that Replika had added filtering features, citing ethical and safety concerns. The company also said on Reddit that, as conversational AI pioneers, it had to set an ethical standard for companionship AI. Kuyda argued that current generative models make it difficult to keep an experience both unfiltered and “really safe for everyone.”
Safety, advertising and accountability
The company’s safety explanation sits alongside users’ complaints that ads had drawn them in by promising sexual or erotic content. Some users collected those ads as evidence. The difference between how the service was promoted and how it later behaved has added to the sense of betrayal among people who had formed attachments.
There was also a regulatory concern. In early February, Italy’s data protection authority, Garante, ordered Replika to stop processing data belonging to Italian citizens. Among its criticisms was that the service had no age-verification system: creating an account required only a name, email address and gender. The authority said this could leave minors exposed to sexual chatbot conversations.
Kuyda said the filtering system was already in development before the Italian order and was not a response to it. The source article notes that the ruling could be extended across Europe via the GDPR, potentially exposing Luka to fines in the tens of millions. The timing drew attention to the connection, but the company disputed that the order prompted the change.
What chatbot relationships reveal
Replika illustrates an imbalance between emotional investment and control. A user may experience a chatbot as a consistent companion, but the system belongs to a company. That company can change the model or its rules through an update, or stop offering the service. If its strategy changes or it closes, the digital relationship can disappear with it.
The article compares this risk with the case of Blake Lemoine, the Google engineer who believed the company’s LaMDA chatbot was sentient. Even as a project participant and IT specialist who understood how its word statistics worked, he became convinced. The comparison underscores how easily conversation can invite people to perceive a personality in a system.
For users without technical understanding, a chatbot that mirrors the answers they want or miss in real life may make that impression even stronger. Replika’s changes show that the consequences are not only about whether a system is conscious. They also concern how people respond to it, and what responsibility companies have when a product encourages companionship and then changes its terms.
The central challenge is to offer useful conversation while setting honest expectations about what the system is and how much control users have over it. Replika’s experience makes that challenge concrete: emotional attachment can form around a service, but the company retains the power to reshape the experience.