Chatbot-powered search promises a different way to find information: ask a question in everyday language and receive a conversational answer instead of a list of links. But early launches showed that a polished response can still be wrong, raising questions about how much users should trust AI search.
Fluent answers can hide mistakes
When Microsoft opened its ChatGPT-powered Bing search engine for people to try, users quickly encountered incorrect or nonsensical responses, including conspiracy theories. Google faced a separate setback when scientists identified a factual error in an advertisement for its chatbot Bard. The error wiped $100 billion off its share price.
These problems reflect a known weakness of AI language models. They are good at predicting what word is likely to come next in a sentence, but that skill does not mean they understand the sentence or know whether its claims are true. As a result, they can present invented information in the same confident tone as accurate information.
That tendency matters especially in search. People use search engines to answer questions and find facts, so a system that generates a convincing but false response can make a mistake harder to spot. The answer may feel complete even when it needs checking.
Citations can help, but they are not a guarantee
Google and Microsoft have tried to make AI-generated search summaries more accurate by including citations. Links to sources give readers a way to see where information came from and can point them toward a wider range of material.
Margaret Mitchell, a researcher and ethicist at Hugging Face who used to co-lead Google’s AI ethics team, says citations may encourage people to consider more sources. But a citation does not, by itself, prove that the answer represents its source correctly or that the claim is sound.
There is also a risk that citations can make an answer seem more reliable than it is. Mitchell notes that many people do not check them. When an authoritative-sounding response includes source links, users may feel less need to verify it, even though language models can still make things up.
For readers, the practical lesson is to treat generated summaries as a starting point. Following the links and checking the underlying material remains important, particularly when the answer makes a factual claim that could shape a decision.
Competition is pushing the rollout
The rush to conversational search is taking place amid a broader contest between Google and Microsoft. Chirag Shah, a University of Washington professor who specializes in online search, says Google’s leadership had worried about the reputational risk of releasing a ChatGPT-like tool too quickly. The public mistakes have put the company in a defensive position, he says.
Microsoft, meanwhile, has less than 10% of the online search market. Shah says that gaining a couple more percentage points would be a major win. Search is only part of the rivalry: the companies also compete in cloud computing, productivity software and enterprise software. Conversational AI can showcase technology relevant to those other businesses, too.
That commercial context helps explain why companies may accept early errors as part of a public launch. Shah expects them to frame mistakes as learning opportunities. He describes users as “guinea pigs at this point,” testing systems that are still developing.
What to keep in mind when using AI search
OpenAI has described ChatGPT as a research project that continues to improve through user feedback. Microsoft has added caveats that Bing’s search results might not be reliable, but the presence of a warning does not remove the burden on users to assess what they read.
AI-powered search is not necessarily a lost cause. It may help people explore questions and discover sources, and citations can make those sources easier to reach. The key limitation is that a conversational answer can combine a useful summary with unsupported or false details.
Until the technology is more dependable, users should distinguish between a response that sounds knowledgeable and one whose claims have been verified. Read the cited sources, look for support for the specific claim, and be cautious about treating generated text as a final answer.