AI scraping bots are no longer just a background technical problem for website operators. The source article describes a web that may soon be shaped by autonomous AI bots as much as by human visitors, with publishers and AI firms already locked in a contest over access, control and value.
OpenClaw, formerly known as Moltbot and before that Clawdbot, is presented as a sign of this broader shift. The issue is not only whether AI systems can train on online material, but also how AI tools fetch fresh information from the live web to improve what they deliver to users.
AI Bots Are Becoming A Regular Web Visitor
A new report from TollBit, along with related data shared with WIRED by Akamai, shows that AI bots already make up a meaningful share of web traffic. TollBit tracks web-scraping activity, and its findings point to a faster-moving Internet where automated visitors are becoming harder to treat as an edge case.
Toshit Pangrahi, cofounder and CEO of TollBit, framed the change in direct terms: “The majority of the Internet is going to be bot traffic in the future,” he says. “It’s not just a copyright problem, there is a new visitor emerging on the Internet.”
That distinction matters. Many major websites have already tried to limit what bots can collect for AI training purposes. WIRED’s parent company, Condé Nast, and other publishers are suing several AI companies over alleged copyright infringement related to AI training.
But the source article makes clear that another category of scraping is rising too. Chatbots and other AI tools can now retrieve real-time information from the web and use it to improve their outputs. The examples given include up-to-the-minute product prices, movie theater schedules and summaries of the latest news.
The Traffic Shift Is Measurable
According to Akamai data cited in the source, training-related bot traffic has been rising steadily since last July. Global activity from bots that fetch web content for AI agents is also increasing.
TollBit’s figures show how quickly the pattern changed across its customers’ websites. In the fourth quarter of 2025, the company estimated that an average of one out of every 31 visits was from an AI scraping bot. In the first quarter, that figure was only one out of every 200.
The report also highlights a breakdown in the informal rules many websites use to signal what bots should avoid. In the fourth quarter, more than 13 percent of bot requests were bypassing robots.txt. TollBit says the share of AI bots disregarding robots.txt increased 400 percent from the second quarter to the fourth quarter of last year.
At the same time, site owners are becoming more aggressive. TollBit reported a 336 percent increase in the number of websites making attempts to block AI bots over the past year. That combination creates the arms race described in the source: more bot traffic, more blocking, and more tactics designed to get around the blocking.
Defenses Are Getting Harder To Enforce
Robert Blumofe, Akamai’s chief technology officer, told WIRED that the consequences go beyond technical filtering. “AI is changing the web as we know it,” he says. “The ensuing arms race will determine the future look, feel, and functionality of the web, as well as the basics of doing business.”
The challenge for publishers is that scraping behavior is becoming less obvious. Pangrahi says scraping techniques are getting more sophisticated as websites try to control how bots access their content. Some bots make their traffic appear as if it is coming from a normal web browser, while others send requests that resemble ordinary human interaction with websites.
TollBit’s study notes that some AI agents now behave in ways that are almost indistinguishable from human web traffic. That makes simple separation between people and machines much harder. It also raises the stakes for any business that depends on human web traffic.
TollBit markets tools that let website owners charge AI scrapers for accessing content. Other firms, including Cloudflare, offer similar tools. Pangrahi argues that “Anyone who relies on human web traffic—starting with publishers, but basically everyone—is going to be impacted,” and says there needs to be a faster “machine-to-machine, programmatic exchange of value.”
Scraping Firms Defend Public Web Access
WIRED attempted to contact 15 AI scraping companies named in the TollBit report. Most did not respond or could not be reached. Several said their AI systems are designed to respect technical boundaries that websites put in place, while also saying those guardrails can be complex and difficult to follow.
Or Lenchner, CEO of Bright Data, said his company’s bots do not collect nonpublic information. Bright Data was previously sued by Meta and X over alleged improper scraping from their platforms. Meta later dropped its suit, and a federal judge in California dismissed the case brought by X.
Karolis Stasiulevičiu, a spokesperson for ScrapingBee, defended the idea that public pages can be read by both people and machines. “ScrapingBee operates on one of the Internet’s core principles: that the open web is meant to be accessible. Public web pages are, by design, readable by both humans and machines.”
Oxylabs said in an unsigned statement that its bots do not have “access to content behind logins, paywalls, or authentication. We require customers to use our services only for accessing publicly available information, and we enforce compliance standards throughout our platform.”
Oxylabs also said there are legitimate reasons to scrape web content, including cybersecurity and investigative journalism. The company argued that some anti-bot systems fail to distinguish between harmful traffic and legitimate automated access.
A New Market Around AI Visibility
The fight over scraping is also creating new commercial opportunities. TollBit’s report found more than 40 companies marketing bots that can collect web content for AI training or other purposes. The rise of AI-powered search engines and tools like OpenClaw are described as likely drivers of demand for those services.
Not every company is focused on blocking bots. Some are helping companies make their content easier for AI agents to find, a strategy called generative engine optimization, or GEO. Uri Gafni, chief business officer of Brandlight, described it as “the rise of a new marketing channel.”
Gafni expects that channel to keep expanding. “This will only intensify in 2026, and we’re going to see this rollout kind of as a full-on marketing channel, with search, ads, media, and commerce converging,” he says.
The picture that emerges is a web being reorganized around automated access. Publishers want control, AI tools want fresh information, scraping firms defend public web access, and new intermediaries are building businesses between them. The outcome will shape not just who reads the web, but who pays for access to it.