AI image generators are often expected to produce new images rather than repeat examples from their training material. A study of diffusion models examined how often that expectation holds, finding that some generated images can closely resemble items in the training data.
Training data size affects the chance of copying
Researchers at New York University and the University of Maryland studied diffusion models trained on datasets including Oxford Flowers, Celeb-A, ImageNet, and LAION. Their paper, “Diffusion Art or Digital Forgery?”, examined how training data volume and training relate to image replication.
The broad pattern was that models trained on smaller datasets were more likely to produce images copied from, or very similar to, their training examples. As the training set grew, the amount of replication fell. That relationship offers useful context for discussions about whether AI-generated images are distinct from the material used to train a model.
The issue has been part of debates about AI art since tools such as Stable Diffusion, DALL-E 2, and Midjourney appeared. Copyright concerns are especially prominent. One flashpoint described in the article is the prompt “trending on ArtStation,” which became the focus of protests because it was based on imitating artworks popular on ArtStation.
What the Stable Diffusion tests found
To study Stable Diffusion, the researchers used “12M LAION Aesthetics v2 6+,” a twelve-million-image dataset, to examine only a small portion of the model’s two-billion-image training dataset. In some cases, they found that models such as Stable Diffusion “blatantly copy” images from their training data.
In random tests, about two out of every 100 generated images were very similar to images in the dataset, using a similarity score greater than 0.5. That result does not mean that every output is a copy, or that most outputs resemble a training image. It does show that close matches appeared often enough in the tested sample to merit attention.
The study also suggests that copying is not an unavoidable feature of generative models. A latent diffusion model trained with ImageNet showed no evidence of significant data replication. The paper notes that earlier studies of generative models such as GANs also found that near-exact reproduction is not inevitable.
Why model and dataset design matter
The researchers suspect Stable Diffusion’s replication behavior reflects several factors working together. These include its use of text conditioning rather than class conditioning, as well as a skewed distribution of repeated images in its training dataset.
That explanation matters because the study does not point to a single cause. The rate of replication may depend on how a model is trained and on the makeup of its data. A larger dataset may reduce the likelihood of copies, but the findings do not establish that size alone eliminates them.
The measured rate may be too low
The researchers stress that their Stable Diffusion test covered only 0.6 percent of its training data. They say the larger dataset contains many examples that could not be found in the portion they examined. They also caution that the methods used may fail to detect some forms of replication.
For those reasons, the paper says, “the results here systematically underestimate the amount of replication in Stable Diffusion and other models.” The estimate of about two similar images per 100 should therefore be read as a result from a limited test, not a complete count of copied or closely matching outputs.
The study’s central finding is that diffusion models can reproduce high-fidelity content from their training data. Typical images from large-scale models did not appear to contain copied content detectable by the researchers’ feature extractors, but the researchers conclude that copies occur often enough that their presence cannot safely be ignored.