AI legal assistants are moving into everyday legal work, but a Stanford University study suggests the tools still carry a serious reliability problem. Researchers found that systems designed for legal research gave incorrect information in one in six queries, even when they used retrieval-augmented generation to connect language models with legal knowledge bases.
What the Stanford study tested
The study came from researchers at the Stanford RegLab and the Stanford Institute for Human-Centered Artificial Intelligence (HAI). They examined tools from two major providers: LexisNexis and Thomson Reuters, the parent company of Westlaw.
The context matters because legal professionals are already looking closely at AI. According to a recent survey cited in the source article, up to three quarters of lawyers plan to use AI in day-to-day work. The intended uses include drafting contracts and assisting with legal opinions.
The Stanford researchers found that specialized legal AI tools performed better than general AI models such as GPT-4. But the improvement did not remove the core risk. The tools still delivered incorrect information in more than 17 percent of cases, which the study describes as one in six queries.
That finding is important because legal research is not a low-stakes search task. A wrong answer can shape how a lawyer understands a dispute, a contract, or a legal position. If the user does not catch the mistake, the error can move from a screen into professional judgment.
Why retrieval is not enough
Both tested tools used retrieval-augmented generation, often called RAG. In plain terms, RAG gives a language model access to a knowledge base so it can draw on external documents rather than relying only on its training.
The study found that this approach is not a complete fix for hallucinations. It can reduce some errors, but it does not guarantee that a legal AI assistant will find the right source, understand the legal context, or connect the source to the answer correctly.
The researchers identified two main forms of hallucination in these systems. One is straightforward: the AI gives a wrong answer. The other is more subtle: the AI may describe the law correctly but cite a source that does not support what it says.
That second failure mode can be especially difficult for users to detect. A cited source may exist, but it may be irrelevant or even contradictory. The presence of a citation can make an answer look more trustworthy than it really is.
Users "may place undue trust in the tool's output, potentially leading to erroneous legal judgments and conclusions," the researchers warn.
For lawyers, this means a source list is not the same as verification. Each cited authority still has to be checked against the claim the AI is making.
Legal research creates special AI problems
The study points out that legal work creates challenges that are different from ordinary factual lookup. Law is not just a collection of simple, verifiable facts. It depends on relevance, jurisdiction, time period, context, and the relationship between sources.
The researchers highlight several specific risks:
- Finding the right source is difficult. Legal relevance can be hard to establish because the law is not merely a database of isolated statements.
- Retrieved documents can mislead. Differences between legal systems and time periods can make a document inaccurate for the question being asked.
- AI can reinforce bad assumptions. A system may agree with a user's incorrect premise, making the user feel confirmed in a mistaken belief.
These risks explain why legal AI cannot be judged only by whether it produces fluent answers. A polished response can still rest on the wrong document, the wrong legal context, or a flawed assumption from the user.
The source article also notes that at least two cases have been reported where lawyers used incorrect information from ChatGPT, faced further investigation, and were subsequently convicted. That background shows why hallucinations in legal tools are not an abstract concern.
At the same time, AI is expected to affect the legal system. US Chief Justice John Roberts predicted earlier this year that human judges will still be needed, while AI will have a significant impact on judicial work, especially in trials.
The transparency problem
The Stanford researchers criticize what they describe as an alarmingly intransparent use of generative AI in law. According to the study, the tools examined do not provide systematic access, publish few details about their models, and report no evaluation results.
That lack of visibility creates a practical problem for lawyers and legal organizations. If buyers cannot see how a product performs, what its limits are, or how it has been evaluated, they cannot make fully responsible decisions about using it.
The study states that this lack of access and transparency makes it harder for lawyers to meet ethical and professional obligations. The concern is not only whether the tools make mistakes, but whether users have enough information to understand and manage those mistakes.
The researchers also argue that the current error rate undercuts the efficiency case for legal AI. If lawyers must verify every statement and every source produced by an AI assistant, much of the promised time savings may disappear.
What the legal field needs next
The researchers say they are not singling out LexisNexis and Thomson Reuters. Their point is broader: those products are not the only AI tools for lawyers that lack transparency. The source article says many startups offer similar products that are even less accessible and more difficult to evaluate.
The proposed path forward is public benchmarks and rigorous evaluations. In the researchers' view, the legal community needs transparent standards that make it possible to compare tools, understand failure rates, and decide where AI can be used responsibly.
The central lesson is clear. AI legal assistants may become useful parts of legal work, but hallucinations remain unresolved. Until vendors provide stronger evidence and clearer evaluation results, legal professionals should treat AI output as a starting point for review, not as a reliable final answer.