Meta’s CM3leon is designed to work in both directions: it can create images from text instructions and generate text based on images. Meta presents it as a single model for several visual tasks, with training that uses licensed image and text data.
One model for images and text
Pronounced “chameleon,” CM3leon is a foundation model that can take in and generate both text and images. Its architecture uses a decoder-only, tokenizer-based transformer network, drawing on an approach associated with text models.
The model is intended to handle more than text-to-image generation. Meta says it can edit images in response to instructions, write captions, answer questions about visual content, and make edits guided by structure. Those abilities put image creation and image understanding within one system.
How Meta says it was trained
CM3leon builds on previous work called RA-CM3 and uses retrieval augmentation during training. In this approach, an external database helps surface relevant and varied data for the model to learn from, rather than relying only on the raw training material already supplied.
Meta describes a two-stage process. The model first goes through large-scale retrieval-augmented pre-training, followed by supervised fine-tuning with instructions. Instruction tuning teaches a model to respond to requests expressed in text, helping it apply its capabilities across different tasks.
Meta also says its training recipe adapts ideas developed for text-only models to image generation based on tokens. The company argues that these scaling methods could support stronger results as models grow, train longer, and use more data. That is a claim about the approach’s potential, rather than a guarantee about future performance.
Reported results and practical capabilities
On the zero-shot MS-COCO image generation benchmark, Meta reports a Fréchet Inception Distance score of 4.88, which it describes as a state-of-the-art result. The source says this beats Google’s Parti image model.
Meta says CM3leon can follow complex instructions while preserving overall shapes and smaller details. It can also generate text or numbers requested in a prompt, and perform guided image editing that previously called for specialized models.
Its ability to describe images may also be useful beyond captioning. Meta says those detailed captions can be used as starting points for further image creation or editing, or to build synthetic training datasets. The model’s text abilities are another part of the company’s case: Meta says CM3leon matches or beats Flamingo and OpenFlamingo on text tasks, despite training on less text—3 billion text tokens.
Licensed training data and Meta’s next steps
Meta says CM3leon was trained on a large Shutterstock dataset containing only licensed image and text data. The company argues this helps avoid concerns about image ownership and attribution without giving up performance. The source does not describe the dataset’s size or provide further details about the licensing arrangement.
The model brings together image generation, editing, captioning, and visual question answering under one system. Meta frames it as a step toward higher-fidelity image generation and understanding, and says models like CM3leon could support creativity and applications in the metaverse.