A marketplace built around AI-generated content is facing hard questions over how its tools can be used to create deepfakes of real women. New research from Stanford and Indiana University examined Civitai’s bounty system and found that users were paying for custom files that could help generate deepfakes, including content the company says is banned.
The findings focus on a difficult part of the AI safety debate: not only finished images or videos, but the smaller technical files that make those outputs easier to produce. On Civitai, those files are called LoRAs, and they can guide mainstream image models such as Stable Diffusion toward content they were not originally trained to create.
How the bounty market works
Civitai is an online marketplace where users can buy and sell AI-generated content. It is backed by the venture capital firm Andreessen Horowitz. According to the source article, the platform includes images, videos, models, and instruction files that users can combine with other tools.
The Stanford and Indiana University researchers looked at requests for content on the site, known as bounties. These bounties allow users to describe what they want, while other users can submit AI models or related files to complete the task. Winning submissions can receive payment.
The researchers examined bounties posted between mid-2023 and the end of 2024. Most bounties asked for animated content, but a significant portion sought deepfakes of real people. The findings have not yet been peer reviewed.
The gender pattern was stark. The researchers found that 90% of deepfake requests targeted women. They also found that 86% of deepfake requests on Civitai were for LoRAs, rather than only finished images or videos.
Why LoRAs matter
LoRAs are important because they sit between a general-purpose AI model and a very specific output. A user can take a mainstream image generator and pair it with a LoRA designed to capture the appearance of a person, character, or style. That makes the marketplace for these files central to how deepfake capability spreads.
The requests described in the source article were often detailed. Some users asked for high quality models of public figures such as Charli D’Amelio and Gracie Abrams. Some bounties linked to social media profiles so images could be pulled from the web. Others asked for models that captured a person’s full body, tattoos, or allowed hair color to be changed.
Some requests focused on women in specific niches, including artists who record ASMR videos. One request was for a deepfake of a woman said to be the user’s wife.
Anyone on the site could submit work for a bounty. The source article says the best submissions received payments ranging from $0.50 to $5. Nearly 92% of the deepfake bounties were awarded.
The platform’s rules have shifted
Civitai previously banned only sexually explicit deepfakes of real people. In May 2025, the company announced that it would ban all deepfake content. Even so, MIT Technology Review confirmed that many deepfake requests submitted before the ban remained live, and many winning submissions were still available for purchase.
The source article also says Civitai automatically tags bounties that request deepfakes and provides a way for the person featured in the content to manually request a takedown. That matters because it suggests the platform has a mechanism for identifying deepfake-related requests. But the system described still places much of the burden on the people affected or on the public, rather than relying only on proactive moderation.
The researchers also found that porn on the platform has increased and that most weekly requests are now for NSFW content. The platform offers educational resources on using external tools to further customize image generator outputs, including changing a person’s pose. It also hosts user-written articles with details on instructing models to generate pornography.
Matthew DeVerna, a postdoctoral researcher at Stanford’s Cyber Policy Center and one of the study’s leaders, said the issue is not just that the platform hosts infrastructure. He argued that it also teaches users how to use that infrastructure.
Payments and accountability
Civitai uses an online currency called Buzz. Users buy Buzz with real money and then use it on the platform. In May 2025, Civitai’s credit card processor cut off the company because of its ongoing problem with nonconsensual content.
After that, users who wanted to pay for explicit content had to use gift cards or cryptocurrency to buy Buzz. The company also offers a different scrip for non-explicit content.
The legal position of a company hosting user activity like this is not fully settled. Ryan Calo, a professor specializing in technology and AI at the University of Washington’s law school, told MIT Technology Review that tech companies generally have broad protections under Section 230 of the Communications Decency Act, but that those protections have limits. He said a website cannot knowingly facilitate illegal transactions.
Civitai did not respond to requests for comment. Andreessen Horowitz also did not respond.
A broader AI safety problem
The Civitai case shows why deepfake moderation is more complex than removing a single image. A platform can host the requests, the training files, the examples, the payment system, and the instructions that make harmful outputs easier to produce elsewhere. Each layer may look different from finished explicit content, but together they can support the same outcome.
Civitai joined OpenAI, Anthropic, and other AI companies in 2024 in adopting design principles aimed at preventing the creation and spread of AI-generated child sexual abuse material. That followed a 2023 report from the Stanford Internet Observatory, which found that the vast majority of AI models named in child sexual abuse communities were Stable Diffusion-based models predominantly obtained via Civitai.
Adult deepfakes, however, have not received the same level of attention from platforms or the venture capital firms behind them. Calo said the response to this category of harm is far less aggressive than the response to child sexual abuse material.
For Civitai, the central issue is no longer whether deepfakes are theoretically possible on open AI systems. The research described in the source article points to a marketplace where requests, rewards, and reusable files can make the production of real-person deepfakes more accessible. That puts the burden back on platforms to decide whether their rules are meaningful only on paper, or whether they are enforced across the systems that make the content possible.