A debate over how to understand AI models has sharpened around a simple question: do their internal units combine unrelated concepts in ways that make them hard to read, or have researchers misread what those units are doing? Nick Cammarata of Goodfire argues that many neurons have clearer roles than the field once assumed. Other researchers see promise in the work, but caution that it does not settle the larger questions.
Revisiting the case for tangled networks
Mechanistic interpretability seeks to explain how a model reaches an answer by examining its internal activity, rather than only looking at what it outputs. A neuron that responds to a recognizable feature would be relatively easy to interpret. But researchers have also found units that appear to respond to seemingly unrelated things, a pattern known as polysemanticity.
A well-known example came from the image model InceptionV1: one neuron responded to both cat faces and the fronts of cars. Cammarata had used that example in his earlier work. The field's common explanation was superposition: when a network has fewer neurons than concepts to represent, it can use the same neuron for multiple concepts that rarely interfere with each other.
That explanation encouraged researchers to train additional models, including sparse autoencoders, to separate concepts and study them more clearly. Cammarata helped develop this line of thought with Chris Olah in the paper “Zoom In,” which introduced the first superposition circuit. In 2023, he began an automated interpretability project at OpenAI; at that time, only about 1.7 percent of neurons were considered understandable, even under a generous standard.
What Cammarata says the neurons are doing
In his essay “Neural Networks Are Already Clean,” Cammarata revisits the examples that helped establish the field's view. He now interprets the cat-and-car neuron as a detector for V-shaped forms. Pointed features such as fox snouts or flames activate it, while round forms such as the letter O push its activation in the negative direction. On this account, the neuron tracks a shared shape feature rather than two unrelated categories.
He offers a similar reinterpretation of the circuit in “Zoom In,” where a car appeared to be mixed in with dog neurons. Cammarata argues that the grouping reflects ordinary generalization: cars and dogs are grouped according to the direction they face toward the camera. He attributes the apparent disorder in InceptionV1's later layers to an older architecture and less capable analysis tools.
Some units can still be difficult to explain. Cammarata calls one pattern “recursive generalization”: a neuron may start by responding to a feature, then respond to related forms that extend beyond it. His example begins with the letter sequence “iy,” expands to “diy,” and eventually includes “do it yourself,” which does not contain “iy.” He sees these links as informative about how a network generalizes, rather than arbitrary combinations of unrelated concepts.
For the language model Qwen3-32B, Cammarata reports that about 90 percent of neuron poles have one main job, about 7 percent have two related ones, and 2 to 4 percent remain unexplained. Goodfire says it sees similar results in models with up to trillions of parameters. Automated AI agents, analyzing neurons in bulk, did much of the work. Cammarata's broader point is that large-scale, direct investigation may outperform elaborate decomposition methods.
Why the safety implications matter
If models can be read more directly, researchers may have a better way to investigate why a system behaved in a questionable way. Cammarata argues that interpretability is important to alignment, the work of steering AI toward human goals, because without understanding a model's internals, researchers may not know what they are building.
He also argues that deception could be visible in a model's circuits, giving people a chance to detect it and address errors more precisely. That possibility matters because the source describes few current ways to determine whether a model is deceiving people or behaving dangerously, beyond monitoring chains of thought that are becoming less reliable.
Roon, an OpenAI employee who posts under that pseudonym on X, called the work a possible historic moment and suggested it could make alignment a “normal science.” He also noted that others still need to confirm the results, and that interpretability advances could also help AI capabilities. The article says that outside research may be uncomfortable for large labs, including OpenAI.
Nanda sees a useful approach, not a resolution
Neel Nanda, who leads the interpretability team at Google DeepMind, takes a more cautious view. His team has deprioritized sparse autoencoders, judging the effort invested in them to be too high for their apparent value, while not considering the technique useless. He sees neuron-based analysis as potentially helpful for practical questions and easier to apply to frontier models, but expects it to have serious flaws and miss important information.
Nanda continues to believe superposition is real. He argues that language models store more facts than they have neurons and that weak activations may carry much of that information. He also questions whether results from small image models apply to language models, and says Cammarata's definition of monosemanticity is too loose. Still, he credits Cammarata for checking earlier findings with modern tools.
Some disagreement turns on what the terms mean. Cammarata accepts that models can represent more concepts than they have dimensions; what he challenges is the idea that networks arbitrarily combine completely unrelated concepts. The technical companion paper has not yet been released, and Goodfire's role as an interpretability company gives it a stake in the argument. For now, the debate is about how much the new analysis establishes, and whether direct neuron study can scale into a dependable way to understand advanced models.