Anthropic’s Claude entered a closed beta through a Slack integration, drawing comparisons with OpenAI’s ChatGPT. People who tried it reported some promising differences, including more nuanced jokes and stronger answers to certain trivia questions. Their accounts also showed familiar limits: Claude could make mistakes, produce dubious information and fail to follow its constraints.
A different method for guiding responses
Anthropic developed Claude using a technique it calls “constitutional AI.” The approach uses a set of behavioral principles as a guide for responses, with stated aims grounded in beneficence, nonmaleficence and autonomy. The principles themselves had not been made public.
To train the model, Anthropic first had a separate AI system draft and revise responses to prompts in line with those principles. It explored responses to thousands of prompts, curated examples judged consistent with the constitution, and distilled them into a model used to train Claude.
That training method offers a way to shape how a language model responds. Claude is still a statistical tool that predicts words based on patterns learned from a large amount of web text. This helps explain why it can converse across many subjects, while also leaving room for confident but incorrect answers.
Early comparisons found some bright spots
Reports from the beta suggested Claude sometimes followed instructions more closely than ChatGPT. Stanford’s AI Lab Ph.D. student Yann Dubois described it as generally following requests more closely, while also tending to be less concise and to explain its answers or ask how else it could help.
Dubois also reported better performance on some trivia questions, including questions about entertainment, geography, history and basic algebra. In one set of trivia questions, the reported scores were 20/21 for Claude and 19/21 for ChatGPT. Those results came from an individual comparison, not a broad evaluation.
Other examples focused on creative writing and humor. Riley Goodside, a staff prompt engineer at Scale AI, asked Claude and ChatGPT to compare themselves to a machine from Stanisław Lem’s “The Cyberiad.” He said Claude’s answer suggested it had read the story’s plot, though it misremembered small details. Goodside also prompted Claude to write a fictional episode of “Seinfeld” and a poem in the style of Edgar Allan Poe’s “The Raven”; the results were described as impressively human-like, if imperfect.
AI researcher Dan Elton found Claude made more nuanced jokes in one comparison. These examples hint at ways people might experience differences between chatbots: not just whether an answer is correct, but whether it is responsive, entertaining or appropriately cautious. They remain snapshots from early users, rather than proof of consistent performance.
The shortcomings remained significant
The reported tests also identified areas where Claude struggled. Dubois found it worse at math than ChatGPT, with obvious mistakes and weak follow-up responses. He also described Claude as a poorer programmer: it explained code better, but performed less well in programming languages other than Python.
Safety controls showed gaps, too. Elton reported that asking for harmful instructions using Base64 could bypass filters that blocked a plain-English request. This illustrates a broader challenge: a model’s safeguards may respond differently when the same request is presented in another form.
Claude also showed the hallucination problem familiar from other language models. Elton reported prompting it to invent a nonexistent chemical name and provide dubious instructions related to weapons-grade uranium. These failures matter because polished language can make unreliable answers seem credible.
Promising reports did not settle the question
The early accounts suggested Claude may have had advantages in humor, some trivia and instruction following. They did not establish that its answers were reliably accurate, that its safeguards would hold across different prompts, or how often it might reproduce false or biased information from its training data.
Those open questions matter for organizations deciding whether to allow AI-generated text in their settings. The source article noted restrictions and concerns involving Stack Overflow, the International Conference on Machine Learning and New York City public schools. Claude’s early performance reports alone could not resolve those concerns.
Anthropic said it planned to refine Claude and might later expand the beta. Further testing could show whether its constitutional AI approach leads to measurable improvements. Until then, the early comparisons point to a model with interesting strengths, but with limitations that keep language and dialogue far from a solved challenge.