ChatGPT appears to decide who it will recommend before it searches. In one practitioner's network trace, brands that appeared in ChatGPT's own self-written search query were cited 68.9% of the time, against 2.1% for brands that merely turned up in the results.

OpenAI documents four agents: OAI-SearchBot (surfaces sites in ChatGPT search), GPTBot (training), OAI-AdsBot (ad landing page safety), and ChatGPT-User (user-initiated actions in ChatGPT and custom GPTs). On ChatGPT-User the page says, in OpenAI's words, "Because these actions are initiated by a user, robots.txt rules may not apply," and separately that "ChatGPT-User is not used for crawling the web in an automatic fashion." OpenAI points site owners at OAI-SearchBot as the agent to manage through robots.txt. Perplexity takes the same position on Perplexity-User. Anthropic does not: it states all three of its bots respect robots.txt, and each token needs its own directive because blocking ClaudeBot does not block Claude-SearchBot or Claude-User.

Across European sites in the report, roughly 15% of identified AI page fetchers reached URLs those sites had marked disallowed. ChatGPT-User, Bytespider and Youbot each reached disallowed pages on close to half of the European sites naming them, with ChatGPT-User covering the most sites. Blocking rates differ sharply by region: 9% of European sites disallow Claude-User against 26% in North America, and 13% disallow Perplexity-User against 26%. TollBit counted any request to a disallowed URL as a bypass regardless of what the operator's own documentation claims.

Judge it honestly. TollBit sells bot paywalls, content controls and paid licensing to publishers, so it has a direct commercial interest in "your protections are being bypassed". We could not find the H1 2026 report on TollBit's own blog, which lists Q1 and Q2 2025 as its most recent, so this is reported through two secondary sources rather than read at source. Sample size is not stated in the coverage. TollBit itself flags the trap that a low bypass count can mean an agent is too obscure to have been named in anybody's robots.txt.

Meta-WebIndexer went from about 2.2% of tracked AI crawler requests in mid-July to 37.8% on 9 August 2026, a surge starting around 17 July. Meta documents the token: "The Meta-WebIndexer crawler navigates the web to improve Meta AI search result quality for users," and says allowing it supports Meta AI citing and linking to content. It is a distinct token from Meta-ExternalAgent (training) and Meta-ExternalFetcher (agentic tasks, may bypass robots.txt). Caveat: the share figure is Promptwatch's monitored subset of crawler logs, not the web, and the sample is not disclosed.

Network-traffic analysis of ChatGPT conversations, reading the JSON that carries the model's self-written search query before any results come back, then comparing that query against what the user actually typed. Across 27 conversations tested for the first query, 21 contained brand names the user never typed. Splitting citations by origin: brands that appeared in ChatGPT's own query were cited 68.9% of the time, against 2.1% for brands that only showed up in retrieved pages. Overall citation rate was about 3.1% of roughly 600 retrieved pages, from a wider set of 57 conversations and 3,554 pages. Categories included accounting, hosting, therapy, language learning and robotics.

Judge it honestly. One account, no peer review, and the author says plainly that every percentage is "a direction rather than a measurement". The sample skews to software and AI tools, which are the categories most saturated with training data, and personalisation affects results. A coach in a small town is the hardest possible case for this mechanism, and nobody has tested it.

163 respondents over three weeks in July 2026, self-selected from the author's network, LinkedIn and X, roughly a 7-point margin of error, 123 giving open-text answers. Practitioners rated the underlying data 4.20 out of 5 but the platforms selling it 3.19. Only 44% thought buying a tool was worthwhile, and just 7% named cost as the objection while 57% raised credibility: "I don't believe the number" or "I can't connect this to money". The leading specific complaints were methodology opacity (24%), ROI attribution failure (20%), non-determinism and personalisation (15%), and synthetic prompts standing in for real demand (11%).

Judge it honestly. Self-selected sample, one person's network, and SEO practitioners are not coaches. It measures what buyers of tools say, not what they do.