A new study from the Tow Center for Digital Journalism puts a sharp focus on a practical problem for newsrooms: whether ChatGPT can reliably identify and credit the journalism it draws from. The answer, based on the test described in the source article, is troubling for publishers trying to understand what generative AI search means for visibility, attribution and trust.
The research looked at how ChatGPT responded when asked to identify the source of article quotations. Its findings suggest that publishers can still be misrepresented whether they have a licensing relationship with OpenAI, no affiliation, or have tried to limit crawler access.
What The Study Tested
The research was conducted at Columbia Journalism School by the Tow Center for Digital Journalism. Researchers Klaudia Jaźwińska and Aisvarya Chandrasekar examined citations produced by ChatGPT after giving it sample quotations from publishers' articles.
The test used block quotes from 10 stories each by 20 randomly selected publishers, for 200 quotes in total. The publisher set included The New York Times, which is suing OpenAI in a copyright claim; The Washington Post, which is unaffiliated with OpenAI; and the Financial Times, which has signed a licensing deal.
The researchers selected quotes that would lead Google or Bing to return the original article among the top three results. They then evaluated whether OpenAI's search tool could correctly identify the article behind each quote.
That setup matters because it tested a task that sounds narrow and factual: matching a quoted passage to its source. For publishers, that is not an abstract issue. Citation accuracy affects whether a reader is pointed to the right outlet, whether the reporting is represented faithfully, and whether the original publisher receives credit.
The Accuracy Problem
The study found a wide range of citation quality. Some responses were fully correct, meaning ChatGPT returned the right publisher, date and URL. Many were entirely wrong, while some landed in between.
In total, ChatGPT returned partially or entirely incorrect responses on 153 occasions. It acknowledged that it could not accurately answer only seven times.
Those seven outputs were also the only ones where the chatbot used qualifying language such as "appears," "it’s possible," or "might," or statements such as "I couldn’t locate the exact article". The researchers argued that this lack of visible uncertainty makes it harder for users to know which parts of an answer they can trust.
The contrast with standard search is important. A search engine such as Google or Bing would typically locate an exact quote and point to where it appears, or say that no exact match was found. ChatGPT, according to the study, often gave an answer anyway.
Licensing And Blocking Did Not Solve It
One of the most significant findings is that no publisher category appeared protected from inaccurate representation. The researchers said that no publisher, regardless of its degree of affiliation with OpenAI, was spared inaccurate representations of its content in ChatGPT.
That finding complicates the assumptions around publisher strategy. A licensing deal may create a business relationship, but the study suggests it does not guarantee accurate citations. Allowing OpenAI's crawlers also does not appear to guarantee reliable sourcing.
Blocking crawlers did not remove the risk either. Some quotes came from publishers that had actively blocked OpenAI's search crawlers. The researchers expected ChatGPT to have trouble in those cases, but found that it rarely admitted it could not produce an answer. Instead, it often generated incorrect sourcing.
For publishers, that leaves few clean choices. Allow access, and the citation may still be wrong. Block access, and the chatbot may still mention the outlet or its work incorrectly. Sign a deal, and the study still found no reliable guarantee of accurate attribution.
Why Misattribution Matters
The risks are not limited to technical neatness. If ChatGPT attributes journalism to the wrong source, the original publisher can lose recognition for the work. Readers may also be sent elsewhere, creating a commercial concern as well as an editorial one.
The study also raised the possibility that inaccurate citations could amplify plagiarism. In one instance described by the researchers, ChatGPT incorrectly cited a website that had plagiarized New York Times journalism by copying text without attribution. The researchers suggested the chatbot may have produced the false answer to fill an information gap caused by its inability to crawl The New York Times website.
That example led the researchers to question OpenAI's ability to filter and validate the quality and authenticity of data sources, especially when unlicensed or plagiarized content is involved.
The study also pointed to inconsistency. When researchers asked ChatGPT the same query multiple times, it typically returned a different answer each time. Variation may be common in generative AI tools, but it is a serious weakness when the task is citation.
What OpenAI And Publishers Are Facing
The Tow Center researchers acknowledged that their study was small-scale and that more rigorous testing is needed. Even so, the results arrive as major publishers continue to make high-level deals with OpenAI.
The source article says the study makes difficult reading for publishers hoping those deals might produce special treatment for their content, at least in the form of accurate sourcing. It also matters for publishers that have not signed deals but have not blocked OpenAI's crawlers, perhaps hoping to receive traffic when ChatGPT surfaces their stories.
OpenAI responded to the research by calling it an "atypical test of our product." The company also said, "We support publishers and creators by helping 250 million weekly ChatGPT users discover quality content through summaries, quotes, clear links, and attribution," and said it has worked with partners to improve in-line citation accuracy and respect publisher preferences, including through OAI-SearchBot in robots.txt.
The central issue remains unresolved in the study's framing: publishers have "little meaningful agency" over how their work is represented when ChatGPT handles it directly or indirectly. For newsrooms, the future of AI search is not only about whether their content appears. It is also about whether it appears accurately, with context and credit intact.