What Is GPTBot? The AI Crawlers That Visit Your Website, Explained

Your web person sends a screenshot of the visitor log with one name circled: GPTBot. Or your website builder adds a new switch labeled "block AI crawlers" and you have no idea whether to flip it.

So what is GPTBot? It is a program OpenAI runs to read public web pages. It is not ChatGPT, it is not a person, and its visit is neither a compliment nor a threat.

Below is a plain definition, the three other OpenAI visitors it gets confused with, how to tell a real visit from a fake one, the other AI crawlers you will see, what a visit does and does not mean, and the one decision worth making.

What is GPTBot?

GPTBot is OpenAI's web crawler for training. It visits public web pages and collects their content, and OpenAI says that content may be used to train its AI models. It does not write answers and it does not choose which businesses ChatGPT names.

A crawler is a program that visits web pages on its own, one after another, and saves what it finds. Search engines have used them for decades. Google's guide to how Search works describes its own as "automated programs called crawlers", and says there is no central registry of web pages, so a crawler has to keep looking for new ones.

OpenAI's overview of its crawlers gives GPTBot one job. It "is used to crawl content that may be used in training our generative AI foundation models". Note the word "may". A visit means a page was collected. It is not a promise that the page ends up in a model.

GPTBot is one of four OpenAI visitors

OpenAI lists four named visitors, and GPTBot is only the training one. The other three handle ChatGPT search, pages a person asks ChatGPT to open, and checks on ads. Most confusion about GPTBot comes from giving it a job that belongs to one of the others.

• GPTBot collects pages that may be used for training.
• OAI-SearchBot is the search one. OpenAI says it "is used to surface websites in search results in ChatGPT's search features".
• ChatGPT-User opens a page when a person asks ChatGPT a question that needs it. OpenAI says it "is not used for crawling the web in an automatic fashion".
• OAI-AdsBot checks pages that have been submitted as ads on ChatGPT. OpenAI says it only visits pages submitted as ads.

The line that matters for a business owner is on the same page: "Each setting is independent of the others". You can let the search crawler in and keep the training crawler out, or the reverse. Whether each of these follows your instructions is its own subject, covered in do AI crawlers obey robots.txt.

One detail is easy to miss. OpenAI says that if your site allows both GPTBot and OAI-SearchBot, it "may use the results from just one crawl for both use cases". So seeing only one of the two names in your log does not mean the other job was skipped.

How to tell if GPTBot has visited your site

Search your website's visitor log for the word "GPTBot". Every match is one request for one page. Then check where the visit came from, because the name alone proves nothing.

A visitor log is a plain list your web host keeps, one line per request. Each line records the address the request came from, the time, the page asked for, your site's reply, and the name the visitor gave for itself. The figure takes one made-up line apart.

ILLUSTRATIVE: ONE MADE-UP LINE FROM A VISITOR LOGONE VISIT, PART BY PART

203.0.113.24 [07/Oct/2026:06:14:09 -0600] "GET /services HTTP/1.1" 200 "Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko); compatible; GPTBot/1.4; +https://openai.com/gptbot"

1. The address it came from

   203.0.113.24

   Which computer made the visit. This is the part to compare with the list OpenAI publishes.

   Hard to fake

2. When

   [07/Oct/2026:06:14:09 -0600]

   The date, the time and the time zone of your server.

   Written by your server

3. The page it asked for

   "GET /services HTTP/1.1"

   Here, your services page. A crawler asks for one page per line.

   Written by your server

4. What your site replied

   200

   200 means the page was sent. 403 means your site refused. 404 means no such page.

   Written by your server

5. The name it gives

   "Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko); compatible; GPTBot/1.4; +https://openai.com/gptbot"

   Look for "GPTBot" inside it. The rest is boilerplate, and the version number changes.

   Anyone can type this

Two parts decide it. The name says GPTBot, and the address is on OpenAI's published list: a real GPTBot visit. The name says GPTBot and the address is not on the list: something else borrowing the name.

AnswerHalo answerhalo.com

CLICK TO ENLARGE

That line is made up, and its address is from a range set aside for examples, so it belongs to nobody. The name part is real: OpenAI gives that text as an example of what GPTBot sends and warns that the version number may change.

The address is the check. OpenAI publishes the addresses GPTBot comes from at openai.com/gptbot.json. When we read it on October 7, 2026, the file listed 18 address ranges and was dated September 22, 2026, so it changes. A range covers a block of neighboring addresses. Your web host or web person can tell you in a minute whether an address falls inside one.

Now the limit. Many website builders do not show you a visitor log at all. If yours does not, you cannot see crawler visits directly, and that is fine. What you can always check is whether your site is telling crawlers to stay out, which is one of the steps in how to audit your website for AI readability in 30 minutes.

The other AI crawlers you will see

GPTBot has company. Anthropic and Perplexity run named crawlers of their own and publish what each is for, and Google controls AI training with a setting that never appears in a log. The table lists what to look for and how to confirm each one.

Anthropic's help page on how site owners can block its crawlers makes the same point about addresses. It says that if a crawler's address is on its published list, "it indicates that the crawler is coming from Anthropic". It also runs two more visitors, Claude-SearchBot and Claude-User, for search and for pages a person asks Claude to open.

You will see other names too, most of them nothing to do with AI. Search engines, uptime monitors and security scanners all leave lines. A name you do not recognize is not a problem in itself.

Does a GPTBot visit mean ChatGPT will recommend you?

No. A GPTBot visit means one of your pages was collected and may be used in training. It does not mean ChatGPT will mention your business, and no visit at all does not mean it never will.

The two things run on different tracks. When a buyer asks ChatGPT for a coach and it searches the web, the pages it can show come through OAI-SearchBot, not GPTBot. And training is slow and indirect: a page collected this month might shape what a future model knows, mixed in with a very large amount of other text.

So a busy log is not a score. We have seen people take a burst of GPTBot visits as a sign of progress. It is a sign a crawler found your pages, which is where being read starts and not where being named ends up.

The reverse holds too. If you cannot find GPTBot in your log, check the dull explanations first: the log covers only a few days, or your builder does not show one, or OpenAI used a single crawl under the other name.

What to do about GPTBot

For most coaches and consultants, nothing. Leave GPTBot alone unless you have a reason to keep your writing out of training, and spend two minutes making sure the search crawler is not blocked by accident. That is the whole job.

The reason to block is ownership. If your site holds paid course material or writing you license, keeping it out of training is a fair choice. OpenAI says disallowing GPTBot "indicates a site's content should not be used in training". Blocking it does not take you out of ChatGPT search, because that is a separate setting.

The accident to avoid is the broad switch. A setting labeled "block AI bots" in a website builder or security service may block the search crawlers along with the training ones. If you turned one on, read what it covers. OpenAI says a change to these rules can take about 24 hours to show up in search results.

And skip anything sold as optimizing for GPTBot. A crawler collects what is on the page. The work that changes answers is on the page itself: clear statements of what you do, for whom, where and at what price. A related file comes up in the same sales pitch, and we cover it in what llms.txt is and whether it helps.

An honest limit on our side: we cannot see your visitor log, and the free check does not look at crawlers at all. It measures the other end. It asks ChatGPT 25 real buyer questions from your niche at $0 and emails a plain-English readout with your score, who got named in your place, and the one fix to start with. It runs as soon as you ask for it, and the report lands within the hour between 7am and 5pm Central, or by 10am the next morning outside those hours.