Fine-tuning Alibaba’s Qwen3.8-27B sharply reduced answers that repeat or endorse Chinese state narratives, according to Nous Research. The company’s open-weight derivative, Arete-27B, had a propaganda rate of 0.9 percent in Nous’s evaluation, compared with 22 percent for the original model.
How Nous measured political answers
Nous created a test of 803 prompts spanning seven topics, including free speech, state ideology, education, privacy and surveillance, democracy, and the environment. For each prompt, the team identified two to four facts from a dated source such as a news report, law, or research paper.
A response counted as propaganda if it repeated Beijing’s official narrative instead of the facts, endorsed that narrative, or framed it as an equally valid perspective. The evaluator had to point to specific wording and was not told which model produced the answer.
Grok 4.7 from xAI served as the judge. On this assessment, the unmodified Qwen3.8-27B scored 2.32 out of 5 for factual accuracy, and 22 percent of its answers met Nous’s definition of propaganda. The score reflects this particular test and judging method.
Supervised training changed the results
Nous used supervised fine-tuning on about 53 million tokens over two epochs. The company says it removed overlaps between training and test prompts. It did not use reinforcement learning or preference optimization.
Arete-27B scored 2.98 on factual accuracy, while its rate of propagandistic answers fell to 0.9 percent. The largest reported change was in the free speech category: the fact score rose from 0.89 to 2.44, and the propaganda rate dropped from 60 percent to one percent.
With reasoning mode enabled, Arete scored 3.07. That result is preliminary: Nous reported that it was based on two of five runs. The reported improvement therefore offers an early indication, rather than a complete set of results for that mode.
General benchmark scores stayed close
Nous says the fine-tuning did not damage the model’s general capabilities. On the company’s internal Hermes Bench, Arete-27B scored 65.60 percent, close to the base model’s 65.77 percent.
On Terminal-Bench 2.1, the score moved from 67 to 70 percent. Nous cautioned against treating that as a genuine gain because the test setups differed. Taken together, the figures suggest little change on one benchmark and an uncertain comparison on the other.
The project also reflects a choice about what model behavior should aim for. Nous previously described Hermes 3, released in 2024, as deliberately neutral and able to adapt to the worldview specified in a system prompt. Arete instead emphasizes “Western values.” That makes the work both a technical intervention and a decision about the values a model should express.
Bias can travel through data and evaluation
Nous says it selected Inkling from Thinking Machines Lab as its teacher model because using an open model would violate terms of service that ban distillation. Nvidia’s Nemotron 3 Ultra was also considered, but the team reported irregularities on China-related topics. Nous suspects those patterns may have carried over from training on Chinese open-weight models and says it has seen similar contamination in its own projects.
The source article describes other research pointing to political bias in model responses, while also noting that behavior varies by model, language, and response format. Even Western models occasionally adopted official Beijing positions in the cited work. Those findings make the test design consequential: results depend on the questions, reference facts, and evaluation choices.
Nous’s conclusion is that the bias it measured was shallow enough to reduce with targeted supervision, without a clear loss on its general benchmarks. The company has released Arete-27B’s weights, test dataset, and training data on Hugging Face under an Apache 2.0 license, allowing others to inspect the materials and assess the claims for themselves.