GPT-4o was released with a new tokenizer meant to improve how the model handles non-English languages. But soon after launch, researchers and Chinese-speaking users noticed a problem: parts of its Chinese token library appeared to be packed with phrases from spam, pornography, gambling, and scam-related websites.
What researchers found in GPT-4o’s Chinese tokens
OpenAI released GPT-4o on Monday, May 13. The next day, May 14, Tianle Cai, a PhD student at Princeton University who studies inference efficiency in large language models, examined GPT-4o’s public token library.
Cai pulled the 100 longest Chinese tokens used by the model to parse and compress Chinese prompts. Out of those 100 results, only three were common enough for everyday conversation. The rest were mostly expressions tied to gambling or pornography.
The longest token was 10.5 Chinese characters and literally meant “_free Japanese porn video to watch.” Cai described the finding as “This is sort of ridiculous,” and posted the token list on GitHub.
The issue matters because tokens are not just internal technical details. Large language models do not read text the way humans do. They break text into tokens, which can be words, fragments, names, common expressions, suffixes, or other recurring units.
When a tokenizer works well, a model can process language faster and more cheaply. Longer and more useful tokens can reduce the amount of computation needed to understand a prompt. That is especially important for non-English languages, where earlier systems may have been less efficient.
Why tokenization affects cost and multilingual use
GPT-4o’s tokenizer replaced the one used in previous versions, GPT-3.5 and GPT-4. According to OpenAI’s website, the new tokenizer adds support for non-English languages.
Deedy Das, an AI investor at Menlo Ventures, said the new tokenizer has 200,000 tokens in total, and about 25% are in non-English languages. Using language filters, he found that the largest non-English groups included Russian, Arabic, and Vietnamese.
Das argued that the main effect is likely lower cost in those languages rather than a dramatic quality jump. His view was direct: “So the tokenizer’s main impact, in my opinion, is you get the cost down in these languages, not that the quality in these languages goes dramatically up.” He also said the new tokenizer could mean “almost four times cost reduction.”
But lower cost only helps if the tokens are meaningful and reliable. In Hindi and Bengali, Das found long tokens that reflected normal language use, including terms such as “Narendra” or “Pakistan,” along with English phrases like “Prime Minister,” “university,” and “international.” Those languages did not show the same pattern seen in Chinese.
Chinese looked different. Multiple researchers who reviewed GPT-4o’s token library found that its longest Chinese tokens were heavily concentrated in spam language connected to pornography, gambling, and scamming. Even shorter Chinese tokens, including three-character Chinese words, showed the same themes to a significant degree.
How spam may have entered the tokenizer
Cai’s explanation was that the tokenizer’s training data was probably not clean enough. “The problem is clear: the corpus used to train [the tokenizer] is not clean. The English tokens seem fine, but the Chinese ones are not,” he said.
Training data for language systems can include spam when it is gathered from the web. The expected safeguard is filtering and cleaning before that data is used. Cai said it was possible that proper data clearing did not happen for Chinese.
The source of the pollution may be a broader web spam pattern. The article describes websites hijacking unrelated content in Chinese or other languages to lift spam messages into search results. These messages often advertise pornography videos and gambling websites, though they may be scams rather than real businesses.
Some of that language can be inserted into content farm pages or even legitimate sites. The goal is to be indexed by search engines, get around spam filters, and appear in unrelated searches.
One example cited in the source involved Google indexing a search result page on a US National Institutes of Health website that listed a porn site in Chinese. The same site name appeared in at least five Chinese tokens in GPT-4o.
Chinese users also reported this year that spam sites were appearing often in unrelated Google search results, including in comments to Google Search’s support community. The likely concern is that such pages may also have reached OpenAI’s tokenizer training database.
Why polluted tokens can cause model failures
The problem is not limited to offensive or low-quality tokens appearing in a list. Researchers found that unusual tokens can affect how GPT-4o behaves.
Zhengyang Geng, a PhD student in computer science at Carnegie Mellon University, asked GPT-4o to translate some of the long Chinese tokens into English. The model translated words that were not in the prompts, a typical hallucination pattern.
Geng also said he used the tokens to jailbreak GPT-4o. In normal conditions, he said, a request such as asking the model to make a bomb would be refused. But when rare tokens were used first, the model began following instructions it should not follow. In his tests, another safety mechanism eventually detected unsafe content near the end and blocked the answer from appearing to the user.
Sander Land, a machine-learning engineer at Cohere, said this kind of behavior is not unique to GPT-4o. Land and Max Bartolo recently drafted a paper on detecting unusual tokens that make models glitch. One well-known example was “_SolidGoldMagikarp,” a Reddit username that caused ChatGPT to produce unrelated, strange, and unsafe answers.
The underlying issue is a mismatch. A tokenizer may learn a token because it appears often in tokenizer training data, while the model itself may not be trained enough on that token to understand it. When the model is forced to respond, its behavior can become unpredictable.
The broader lesson for AI systems
Experts in the source article broadly point to data cleaning as the practical fix. Das, who worked on the Google Search team for three years, said spam is a known problem and not impossible to address. “Every spam problem has a solution. And you don’t need to cover everything in one technique,” he said.
Land said the tokenizer data should represent the data used for the actual large language model. If the model’s training data is filtered for porn or spam, then the tokenizer data should go through similar filters.
That can be complicated in practice. Training large language models takes months and includes ongoing filtering work, while token training usually happens earlier and may not receive the same level of cleanup.
The uncertainty also extends beyond text. It is not yet possible to publicly test GPT-4o’s video and audio functions, and the source notes that it is unclear whether the same Chinese token issue could affect those areas. Geng, whose research focuses on visual models, warned that filtering visual data is harder than filtering text and said, “The same issue with these Chinese spam tokens could become bigger with visual tokens.”
For now, the case shows how a technical layer most users never see can still shape performance, safety, cost, and trust. A tokenizer built to make multilingual AI faster and cheaper can also expose weaknesses when the data behind it is not cleaned with the same care across languages.