Key Takeaways
- Assume AI crawlers read only the first response. A Vercel study published in December 2024 found that none of the major AI crawlers it tracked render JavaScript, including OpenAI's, Anthropic's and Perplexity's.
- Check your page source, not your screen. What you see in a browser is the page after JavaScript ran. View the page source and search for a sentence from your services section to see what a crawler receives.
- Put the words that sell you in the HTML. Your name, what you do, who you serve, where and your prices should arrive in the server's first response, not appear after a script loads them.
- Do not judge by Google alone. Googlebot renders JavaScript with a headless Chromium browser, so a page can rank on Google and still arrive nearly empty to ChatGPT's crawlers.
- Fix the page, not the robots file. Allowing a crawler in robots.txt lets it fetch the page. It still reads only what the server sends, so the remedy is server-side rendering or a static page.
Your website looks finished. The services are listed, the testimonials scroll past, the booking button works. Then someone asks ChatGPT about your kind of business in your city, and the answer reads as if your site said almost nothing.
Can AI crawlers read JavaScript? Mostly no. A study of AI crawler traffic published by Vercel in December 2024 found that none of the major AI crawlers it tracked run JavaScript, so a page that draws its words with a script can arrive nearly empty.
Below is which crawlers run JavaScript and which do not, why a page that looks full to you can be empty to them, and a five-minute way to see exactly what they receive from your own site.
Can AI crawlers read JavaScript?
Most AI crawlers cannot read content that JavaScript adds to a page. They download the HTML your server sends in its first response and read that. If your services, prices or reviews are added later by a script, those words are not in what they read.
The clearest evidence is a study by Vercel, the web hosting company, called The rise of the AI crawler. It watched crawler traffic on nextjs.org and across Vercel's network for several months and concluded that "none of the major AI crawlers currently render JavaScript." It found OpenAI's and Anthropic's crawlers do fetch JavaScript files, but they do not run them.
Two limits travel with that finding. The study is from December 2024, and crawlers change, so treat it as the best public measurement rather than a permanent rule. And it left out Microsoft Copilot, because Copilot has no user agent of its own to track.
Which crawlers run JavaScript and which do not
Google's crawler and Apple's crawler run JavaScript; OpenAI's, Anthropic's and Perplexity's AI crawlers do not, according to the Vercel study. Gemini runs it too, because it uses Googlebot's infrastructure. The table sums up what each source says.
| Crawler | Used by | Runs JavaScript? | Source |
|---|---|---|---|
| GPTBot, OAI-SearchBot, ChatGPT-User | OpenAI, including ChatGPT search | No. Fetches JavaScript files, does not run them | Vercel study, December 2024 |
| ClaudeBot | Anthropic | No. Fetches JavaScript files, does not run them | Vercel study, December 2024 |
| PerplexityBot | Perplexity | No | Vercel study, December 2024 |
| Googlebot | Google Search, including AI Overviews | Yes. A headless Chromium renders the page | Google Search Central |
| Gemini | Yes, through Googlebot's infrastructure | Vercel study, December 2024 | |
| AppleBot | Apple | Yes, through a browser-based crawler | Vercel study, December 2024 |
| Microsoft Copilot | Microsoft | Not measured; no user agent of its own to track | Vercel study, December 2024 |
Google describes its side in its guide to JavaScript SEO basics. Googlebot crawls a page, puts it in a queue, and "a headless Chromium renders the page and executes the JavaScript," then uses the rendered HTML for indexing. A page can wait in that queue for seconds or longer.
The crawler names matter because you will see them in your robots.txt file and your hosting logs. OpenAI lists GPTBot, OAI-SearchBot and ChatGPT-User on its crawler documentation, with OAI-SearchBot the one used to surface websites in ChatGPT search.
One page, two layers, and which crawlers read each
-
GPTBot, OAI-SearchBot, ChatGPT-UserOpenAI
Layer 1Read Layer 2Not read -
ClaudeBotAnthropic
Layer 1Read Layer 2Not read -
PerplexityBotPerplexity
Layer 1Read Layer 2Not read -
GooglebotGoogle
Layer 1Read Layer 2Read -
GeminiGoogle, on Googlebot's infrastructure
Layer 1Read Layer 2Read -
AppleBotApple
Layer 1Read Layer 2Read
Every crawler reads the first layer. Only Googlebot, Gemini and AppleBot go on to read what JavaScript adds.
Read the figure from the top down. Anything in the first layer reaches every reader. Anything that exists only in the second layer reaches Google and Apple, and is missing for ChatGPT, Claude and Perplexity when they read your page with their own crawlers.
Why some pages arrive empty
Some pages arrive empty because the server sends a near-blank shell and a script fills in the words inside your browser. You never see the shell, because your browser runs the script in a fraction of a second. A crawler that does not run scripts only ever sees the shell.
The case most at risk is a site built as a single-page app, where the first response is little more than a title, a few script tags and an empty container. Parts of an otherwise normal site can do it too: a reviews widget, a pricing table pulled in from another service, or a services list that loads when you scroll.
The result looks like a content problem but is a delivery problem. A beautiful page can be close to invisible to an assistant for the same reason, which is covered in why a beautiful website can be invisible to AI.
How to check what a crawler receives
To check what a crawler receives, look at the page source rather than the page. The source is the server's first response, which is what an AI crawler reads. It takes about five minutes for your three most important pages.
- Open the source. On your services page, right-click and choose View Page Source, or add
view-source:in front of the address in Chrome. - Search for a real sentence. Press Ctrl+F (Cmd+F on a Mac) and search for a phrase from your services section, your price, or a client's name from a testimonial.
- Read the result. If the phrase is there, crawlers get it. If the source is mostly script tags and the phrase is missing, those words arrive by JavaScript.
- Repeat for home and about. Those three pages carry most of what an assistant needs to describe you.
- Ask for the fix by name. If words are missing, ask your developer or builder's support for server-side rendering or static pages for those sections.
Google recommends the same remedy for its own reasons: server-side rendering or pre-rendering makes a site faster, and its guide notes that "not all bots can run JavaScript." Server-side rendering simply means the server builds the full page text before sending it. A fuller walkthrough of the other things to look at is in how to audit your website for AI readability in 30 minutes.
What fixing JavaScript will not fix
Fixing JavaScript makes your words readable; it does not make them persuasive. Once a crawler can read the page, the assistant still has to decide your page answers the buyer's question better than the others it read.
It also does not touch answers written without a search. When ChatGPT answers from what it learned in training, no crawler visits your site that day, and a fix made today shows up only after the model is next updated.
The damaging admission: for most coaches and consultants on a mainstream website builder, the source check comes back fine, and JavaScript is not the reason they are missing. If that is you, the reason is more often what the page says, or that it is missing from the indexes assistants search. Both are covered in why ChatGPT can't find my business.
To see which buyer questions name you today, start with the free check. It asks ChatGPT 25 real buyer questions from your niche at $0 and emails a plain-English readout: your score, who got named in your place, and the one fix to start with. It runs as soon as you ask for it, and the report lands within the hour between 7am and 5pm Central, or by 10am the next morning outside those hours.
Questions we hear the most
Can AI crawlers read JavaScript?
Mostly no. A Vercel study published in December 2024 found that none of the major AI crawlers it tracked render JavaScript, including GPTBot, OAI-SearchBot, ChatGPT-User, ClaudeBot and PerplexityBot. They read the HTML your server sends and skip anything a script adds afterward.
Does ChatGPT see content loaded by JavaScript?
Not when it reads your page with its own crawlers, according to the Vercel study, which found OpenAI's crawlers fetch JavaScript files without running them. Text that only appears after a script runs is missing from what they receive.
Does Google render JavaScript for AI Overviews?
Google says Googlebot queues pages for rendering and a headless Chromium runs the JavaScript, then the rendered HTML is used for indexing. AI Overviews draw on Google Search, so Google is the main assistant that sees JavaScript content.
How do I know if my website depends on JavaScript?
Open a page, choose View Page Source in your browser, and search for a sentence from your services section. If the sentence is in the source, a crawler gets it. If the source is mostly script tags, the words arrive by JavaScript.
Will allowing AI crawlers in robots.txt fix a JavaScript problem?
No. Robots.txt decides whether a crawler may fetch the page. It does not change what the page contains when fetched, so a crawler you allow in still receives the near-empty first response.
What is server-side rendering?
Server-side rendering means your server builds the full page text before sending it, so the first response already contains your words. Google recommends it, partly because not all bots can run JavaScript.
Does the AnswerHalo free check test whether AI crawlers can read my pages?
No. The free check asks ChatGPT 25 real buyer questions from your niche at $0 and emails your score, who got named in your place, and the one fix to start with. Checking your page source is a separate five-minute job you can do yourself.
What does AnswerHalo cost after the free check?
The full audit is $500 once and runs 125 checks across ChatGPT, Claude, Gemini, Perplexity and Google AI Overviews, landing by this time tomorrow. Ongoing watching is $399 a month, or $300 a month added with the audit.
Your buyers are already asking. Find out what AI tells them.
The free check asks ChatGPT 25 real buyer questions about your niche and sends you the report within the hour, at $0. If it shows you are already getting named everywhere, we will say so in plain words.
Get my free check →