None of the ten foundational language models assessed in a Stanford study reached the score the researchers associated with full compliance under the EU AI Act. The results point to a common challenge: providers may need to disclose more about how their models are built and evaluated.
How the researchers assessed the models
The study scored models across twelve categories, with up to four points available in each. A total of 48 points represented full compliance in the researchers’ assessment. Two lead authors gave the initial scores using a set methodology, then the full author group discussed and voted on them.
The categories covered issues such as transparency about data sources, treatment of proprietary data, risk mitigation, and computational and energy requirements. Together, they look beyond what a model can do and ask how clearly its provider explains the processes and risks behind it.
Open models led the scores
Bloom, from Big Science, received 36 points, the highest score in the assessment. GPT-NeoX, from EleutherAI, followed with 29 points. Both are open-source models, which the article says are more transparently documented than models from commercial providers that take competition into account.
That lead does not mean open models have no shortcomings. The article notes that open-source models are improving but still lack performance, and cites OpenAI CEO Sam Altman as saying they are likely to continue to do so. The study’s scores are about compliance-related criteria, while model performance is a separate concern.
Disclosure remains a widespread weakness
The researchers identified several areas where most providers performed poorly: the use of copyrighted material in training data, unclear reporting on computational and energy requirements, and limited information about risk mitigation. They also pointed to a lack of standards for evaluating model performance, particularly its adverse effects.
These gaps make it difficult to assess a model’s full footprint from public information. For example, when providers do not explain their data sources or risk controls clearly, outside observers have less information to use in judging those choices. The assessment therefore highlights transparency as a practical issue for both compliance and public scrutiny.
Compliance may be achievable, but transparency is slipping
Despite the low scores, the study’s authors say many providers could reach a score between 30 and 40 through “meaningful, but plausible changes.” They suggest incentives such as fines for non-compliance may be enough to encourage progress without much regulatory pressure. The authors also argue that sufficient transparency about data, computing and related factors should be commercially feasible if providers act collectively through industry standards or regulation.
The researchers see the EU AI law as influential beyond Europe because of the Brussels effect: lawmakers elsewhere may use it as a reference, while multinational companies may seek consistent processes for AI development. In their view, those choices can shape the digital supply chain and AI’s societal impact.
The study describes implementing the Act’s 12 requirements as capable of bringing “significant positive change in the foundation model ecosystem,” while also warning that the current trend is toward less transparency. Its central message is that providers and policymakers both have a role: shared industry standards could improve disclosure, and policy can help ensure transparency underlies this general-purpose technology.