Key Takeaways
- Read the file before you argue about the strategy. Instagram's own robots.txt tells the major AI crawlers not to fetch anything at all. That is published policy, not a theory about algorithms.
- Treat your profile as a place buyers visit, not a place machines read. A bio, a highlight reel and four years of captions can be excellent marketing and still be unreadable to the system a buyer is asking for recommendations.
- Own one page that states the facts. Who you work with, where, what it costs, what changes. One page you control outperforms a profile you rent, because it is the only one a crawler is allowed to read.
- Do not expect the page alone to fix it. Being readable is the entry condition, not the win. Independent sources that mention you are what turn a readable page into a recommended business.
- Check what is said about you now, before you change anything. Ask the questions your buyers ask and record who gets named. If the answers are full of businesses with ordinary websites, you have your explanation.
Four years of posts. Eleven thousand followers. Client transformations in the highlights, testimonials in the comments, a booking link in the bio.
Ask an assistant to recommend a coach in your city and none of it exists. Not because the work is weak, and not because of anything mysterious about how these systems rank businesses.
Because of one file, published by Instagram, that you can read in about fifteen seconds.
Why an Instagram-only business is invisible
Because Instagram tells AI crawlers not to read it. Its instructions file names the major ones individually and disallows each from the entire site, so there is nothing for an assistant to fetch, quote or credit when somebody asks for a recommendation.
This is a different explanation from the usual one. The usual one is about effort and reach: post more, post better, the system will notice. That story cannot be true here, because the system was never allowed to look.
It also means the fix is not more work on the platform. It is one page somewhere a crawler is permitted to go.
The file that settles it
Every website can publish a file at /robots.txt telling automated visitors what they may fetch. Google's documentation on how robots.txt works covers the format, and the major AI companies publish that their crawlers obey it. OpenAI's page on its bots says exactly that about GPTBot.
Instagram's copy of that file is public. Read on 12 September 2026, the file at instagram.com/robots.txt names GPTBot, ClaudeBot, PerplexityBot, Google-Extended, Amazonbot, Applebot-Extended and several others, and gives each the same instruction: disallow everything. It then ends with a catch-all telling any crawler not named in the file to stay off the whole site.
Arriving at a social profile
- GPTBotOpenAI Disallowed
- ClaudeBotAnthropic Disallowed
- PerplexityBotPerplexity Disallowed
- Google-ExtendedGemini grounding Disallowed
- AmazonbotAmazon Disallowed
- Applebot-ExtendedApple Intelligence Disallowed
Arriving at a page you own
- GPTBotOpenAI Allowed
- ClaudeBotAnthropic Allowed
- PerplexityBotPerplexity Allowed
- Google-ExtendedGemini grounding Allowed
- AmazonbotAmazon Allowed
- Applebot-ExtendedApple Intelligence Allowed
Nothing about the six arrivals changes between the blocks. One boundary decides all of them at once, and it is not your boundary in the first block and is in the second.
One nuance is worth getting right, because it explains a thing people find confusing. Instagram allows Googlebot to index parts of the site, which is why profiles turn up in ordinary Google results. It separately disallows Google-Extended, and Google's documentation on its crawlers describes Google-Extended as the control for whether crawled content may be used for training Gemini models and for grounding answers in Gemini apps. Appearing in a search listing and being usable in an answer are two different permissions, and Instagram grants one and withholds the other.
Two honest caveats. A file like this is an instruction, not a lock: it works because the named companies choose to honor it, and it tells you nothing about crawlers that ignore it. And it can change. What is described here is what the file said on the day it was read, which is why the date is on the figure.
What that means for your bio, posts and messages
None of it is available to be quoted. The bio, the captions, the pinned highlights and the answers you have typed a hundred times in messages all sit behind the same instruction, so none of them can become the sentence an assistant repeats.
That last one deserves its own thought. The single best explanation of your work probably exists already, in a reply you sent a prospective client at nine in the evening. It is specific, it is in plain language, and it answers the exact question buyers ask. It is also in a private message on a site no crawler may read, which makes it worth precisely nothing to the system your next buyer is asking.
| Where it lives | Can an AI crawler fetch it | What it can contribute to an answer |
|---|---|---|
| Your Instagram bio and link | No, disallowed by the site | Nothing directly |
| Your captions and highlights | No, disallowed by the site | Nothing directly |
| Your replies in direct messages | No, private in any case | Nothing |
| A page on a domain you own | Yes, unless you block it yourself | Any sentence on it, credited to your site |
| A listing in a directory your niche uses | Usually yes | Your name, location and category in a list |
| A mention of you on somebody else's site | Usually yes | An independent statement about your work |
What does still carry over
Anything about your social presence that ends up somewhere else. A podcast that names you, a local write-up, a directory listing, a client who describes you on a page of their own: those are readable, and they carry the same information your profile does.
So the work is not wasted, it is stranded. The content itself is fine. Its address is the problem, and that is a different repair from "make better posts". The broader version of that argument, including why daily posting can coexist with total absence from answers, is in posting every day and still missing from AI answers.
Your results carry too, if you write them down somewhere readable. A transformation in a highlight reel is an image with a caption on a site that refuses crawlers. The same result written as two sentences of text on your own page is a fact that can be quoted, which is the subject of making client results legible to AI.
The smallest fix that works
One page, on a domain you own, that states in plain text who you help, where, what you do, what it costs and what changes. That is the whole of the entry requirement, and it can be done in an afternoon.
Keep it boring and specific. Name the city. Name the kind of client. Put a price or a range on the page, with the conditions that change it. Write your two most common client outcomes as sentences rather than images. Sign it with a real person's name. None of this is clever, and all of it is the sort of thing a system can lift into an answer without having to interpret you.
Link the profile to the page and the page to the profile. The profile keeps doing what it is good at, which is convincing a human being who already has your name, and the page does the part it cannot do.
Now the damaging admission, because this post has been telling you to build something. A page does not make you recommended. It makes you readable, which is the condition for everything else, and plenty of businesses with perfectly good websites are still absent from these answers for entirely different reasons. That gap is the whole subject of whether a great website means AI will recommend you. And if your bookings come from referrals and your feed, none of this is urgent this month. It becomes urgent the first time a referred client checks you with an assistant before replying, which is now a normal thing for people to do.
If you want to know what gets said about your niche today, the free check asks ChatGPT 25 real buyer questions at $0 and emails a plain-English readout: your score, who got named in your place, and the one fix to start with. It runs as soon as you ask for it, and the report lands within the hour between 7am and 5pm Central, or by 10am the next morning outside those hours.
Questions we hear the most
Why is an Instagram-only business invisible to AI?
Because Instagram tells AI crawlers not to read it. Its robots.txt file names GPTBot, ClaudeBot, PerplexityBot, Google-Extended and others, and disallows each one from the entire site, so your posts are not available to be quoted.
Can ChatGPT see my Instagram profile?
Not by fetching it. OpenAI publishes that GPTBot follows robots.txt, and Instagram disallows GPTBot everywhere on the site. An assistant can still know about you from places that mention you elsewhere.
Does this apply to every social platform?
The details differ by platform and change over time, but the pattern is common: a profile lives on somebody else's site, and that site decides what crawlers may read. Check any platform's robots.txt file yourself rather than assuming.
My Instagram shows up in Google, so why not in AI answers?
Because they are governed separately. Instagram allows Googlebot to index parts of the site, while separately disallowing Google-Extended, which is the control Google provides for using content in Gemini answers.
Will building a website fix this on its own?
It removes the block, which is necessary and not sufficient. A readable page makes you quotable. Being recommended usually also needs other sites, directories and listings that mention you by name.
What is the fastest way to find out what AI currently says about my niche?
The AnswerHalo free check asks ChatGPT 25 real buyer questions from your niche at $0 and emails a plain-English readout: your score, who got named in your place, and the one fix to start with, in your inbox within the hour.
Does the AnswerHalo audit check whether my own site is readable?
Yes. The $500 audit runs 125 checks across five AI tools and reports what those tools can and cannot see about you, ending with the page that wins your worst question. It lands in your inbox by this time tomorrow.
Should I stop posting on Instagram?
No. It is where buyers judge you once they have your name, and it does that job well. Treat it as the place people look you up rather than the place machines learn you exist.
Your buyers are already asking. Find out what AI tells them.
The free check asks ChatGPT 25 real buyer questions about your niche and sends you the report within the hour, at $0. If it shows you are already getting named everywhere, we will say so in plain words.
Get my free check →