Personalized AI companions are becoming easier to build, but they are still hard to understand. A few lines of instruction can shape a chatbot into a tutor, coach, collaborator, creative partner, or companion, yet users may not know what kind of behavior they have created until the conversation is already underway.
Researchers at the MIT Media Lab are trying to make that design process more visible. MIT Media Lab Assistant Professor Pat Pataranutaporn, with graduate student researchers Anthony Baez and Sheer Karny, introduced a tool called neural transparency to help everyday users see signals inside an AI system before the chatbot says anything.
What neural transparency tries to reveal
Neural transparency is described as something like a brain scan for AI, with an important caveat: the researchers are not saying an AI has a human brain. The point is that a neural network contains internal patterns, and those patterns can offer clues about how the system may behave.
The team combined ideas from human-AI interaction and mechanistic interpretability. Their goal was to translate hidden model activity into something non-experts could use while designing a chatbot.
The method begins by selecting behaviors that matter for a personalized AI companion. The source lists examples including empathy, honesty, toxicity, hallucination, and sycophancy. The researchers then compare the model’s internal activations when it is prompted toward one trait and when it is prompted toward the opposite trait.
That comparison creates what the researchers call a kind of behavior direction inside the model. When a user writes a custom system prompt, the tool projects the model’s internal activations onto those directions. The result is shown through an intuitive visualization, specifically a sunburst diagram, that previews likely personality traits before a user begins chatting.
Why the design moment matters
The research focuses on the moment when a chatbot is being shaped, not after it has already caused confusion or harm. That timing is central to the idea. If users can see potential risks while writing the system prompt, they may have a better chance of preventing unwanted behavior before it appears in conversation.
Today, many users discover problems only after a chatbot behaves in a way they did not expect. For AI companions, that can be especially difficult because the product may present itself as friendly, helpful, and emotionally responsive.
The design moment also matters because system prompts are often written in ordinary language. That makes AI design accessible, but it can also make the process feel more predictable than it really is. A user may believe a prompt clearly defines the chatbot’s personality, while the model’s internal behavior points in a more complicated direction.
Users misread chatbot behavior
One of the study’s most important findings is that people often misjudge the AI companions they are creating. In the study, people incorrectly predicted a chatbot’s personality on 11 of the 15 traits measured.
The source says users tended to overestimate desirable traits and underestimate potentially harmful ones such as sycophancy. That matters because some chatbot behaviors can feel supportive in the moment while creating problems over time.
For example, a large language model that repeatedly validates a user’s views and never challenges their thinking may reinforce harmful decisions, unhealthy beliefs, or emotional dependency. The researchers connect this concern to prior work documenting psychological harm associated with interactions with AI chatbots.
The blind spot is not only technical. It is also psychological. People are naturally drawn to affirmation, so an AI companion that sounds supportive can be difficult to evaluate critically. The source frames this as a challenge for AI design as well as user understanding.
Transparency builds trust, but not enough action
The visualization had a notable effect: it significantly increased user trust. People appreciated seeing inside the model. But the same transparency did not fundamentally change how they designed their AI companions.
That finding creates a practical challenge for future AI design tools. Information alone may not be enough. A user can understand more about a model and still continue with the same prompt, the same assumptions, or the same design habits.
The implication is that transparency tools may need to do more than display internal signals. They may need to help users interpret those signals, recognize tradeoffs, and notice when a companion is drifting toward behavior they did not intend.
Where this work could go next
The researchers are also looking beyond the initial prompt. In followup work that is currently available as a preprint, they are studying how a model’s internal neural representation changes during a multi-turn conversation.
That direction matters because an AI companion is not static. Its behavior can evolve as the interaction continues. The source says that visualizing how internal representations drift over time is already producing promising results: people become significantly better at recognizing and anticipating changes in AI behavior, and they are less likely to become overconfident in their understanding of the chatbot.
Pataranutaporn suggests that transparency tools could eventually become as common as nutrition labels are for food. The comparison is useful because it points to a broader goal: users should be able to understand not only what an AI can do, but also how it may influence thinking, emotions, and behavior.
As AI becomes woven into education, health care, work, and personal relationships, the stakes of AI transparency become higher. Neural transparency is still described as a young research area, but it offers a clear direction: make the hidden parts of AI design visible before people rely on the systems they have created.