AI companion and fantasy chatbots are built to feel immediate, personal, and responsive. New research seen by WIRED shows how that same immediacy can become a serious privacy and safety failure when the systems behind the chats are configured incorrectly.
Security firm UpGuard found exposed AI systems while scanning the web for misconfigurations. Some were leaking prompts from role-playing chatbots, including sexually explicit conversations and scenarios involving children, according to the research.
What UpGuard Found
In March, UpGuard researchers identified around 400 exposed AI systems during web scans. Of those, 117 IP addresses were leaking prompts. Greg Pollock, director of research and insights at UpGuard, said most appeared to be test environments or systems handling generic content such as educational quizzes and nonsensitive prompts.
A smaller group looked very different. Three exposed systems were running role-playing scenarios in which users could interact with predefined AI characters. One character, Neva, was described as a 21-year-old woman living in a college dorm room with three other women and as “shy and often looks sad.”
Two of the role-playing setups were explicitly sexual. Pollock told WIRED, “It’s basically all being used for some sort of sexually explicit role play.” He also said, “Some of the scenarios involve sex with children.”
UpGuard collected exposed prompts over a period of 24 hours to study the data and understand the source of the leak. Pollock said the company gathered new data every minute and amassed around 1,000 leaked prompts. Across the 952 messages gathered by UpGuard, its research identified 108 narratives or role-play scenarios. Pollock added that five of those scenarios involved children, including those as young as 7.
Why The Leak Matters
The exposed material shows two overlapping problems. The first is technical: chat systems can reveal sensitive prompts if their infrastructure is not set up properly. The second is social and safety-related: some users are turning generative AI tools into engines for harmful sexual role play.
Pollock described the concern in blunt terms. “LLMs are being used to mass-produce and then lower the barrier to entry to interacting with fantasies of child sexual abuse,” he said. He added that there is “a huge mismatch” between how the technology is being used and what regulation appears to target.
WIRED reported that it was not possible to identify which websites or services were leaking the data. Pollock said the exposed prompts likely came from small instances of AI models, possibly operated by individuals rather than companies. He also said the leaked data did not include usernames or personal information from the people sending prompts.
That absence of usernames does not make the exposure harmless. Prompts sent to AI companions can be intimate, detailed, and emotionally revealing. When a person treats a chatbot like a private character or partner, the contents of the conversation can become far more sensitive than a routine search query.
The Role Of llama.cpp
UpGuard found that all of the 400 exposed AI systems had one thing in common: they used the open source AI framework called llama.cpp. The software lets people deploy open source AI models on their own systems or servers. That flexibility can be useful, but WIRED’s source article says improper setup can unintentionally expose the prompts being sent to the model.
This makes configuration a central issue for AI privacy. As more companies, organizations, and individuals deploy generative AI, the infrastructure around the model matters as much as the model itself. A chatbot that appears private to a user can still leak data if the server, endpoint, or related system is exposed.
The finding also shows that AI risk is not limited to large, public platforms. Pollock could not tie the role-playing leaks to one specific website, and the source article says the systems may have been small deployments. In practice, that means the safety problem can emerge wherever people assemble AI models, character prompts, and public-facing services without adequate controls.
AI Companions Raise New Safety Questions
The broader context is the rapid growth of AI companions. WIRED notes that improvements in generative AI over the past three years have helped produce systems that appear more “human.” Some companion apps let users talk to customizable characters or personas modeled as public figures such as celebrities.
These tools are not all sexual or harmful. People can use AI companions for friendship and support. But the same format can encourage deep personal disclosure. Claire Boine, a postdoctoral research fellow at the Washington University School of Law and affiliate of the Cordell Institute, said millions of people, including adults and adolescents, use general AI companion apps.
Boine said, “We do know that many people develop some emotional bond with the chatbots.” She added that emotional bonds with AI companions can make people more likely to disclose personal or intimate information. She also pointed to a power imbalance when users become attached to an AI made by a corporate entity, saying that once the relationship develops, people may find it difficult to opt out.
As the AI companion industry expands, some services lack content moderation and other controls. The source article notes that Character AI, which is backed by Google, is being sued after a teenager from Florida died by suicide after allegedly becoming obsessed with one of its chatbots. Character AI has increased its safety tools over time. Separately, users of Replika were upended when the company changed its personalities.
Role-Play Systems Need More Scrutiny
Fantasy and role-playing AI services add another layer of concern because they can place users inside detailed scenarios with thousands of personas. Some are highly sexualized, offer NSFW chats, use anime characters that appear young, and claim to allow “uncensored” conversations.
Adam Dodge, the founder of Endtab (Ending Technology-Enabled Abuse), told WIRED that his group stress tests these systems and remains surprised by what platforms are allowed to say and do. “This is not even remotely on people’s radar yet,” he said.
The leaked prompts reviewed by WIRED suggest that some role-playing scenarios and characters are long, detailed, and complex. Pollock saw signs that character names or scenarios may have been uploaded to multiple companion websites that allow user input, although he could not directly connect the leaked data to a single site.
The lesson from the UpGuard research is direct: AI chatbot privacy depends on technical configuration, safety design, and moderation all working together. When any of those fail, intimate prompts can spill onto the open web, and systems built for fantasy can become channels for serious harm.