Key Takeaways
- Treat one ask as an anecdote, not a measurement. A single answer tells you what happened once. The same question asked again can name a different set of businesses, including yours or not including you.
- Repeat every question at least five times. Five to ten asks per question is enough to tell a genuine pattern from a coin flip, and it is the difference between a number you can act on and a feeling.
- Log out and start clean every time. Your own account has history, memory, and a location attached. Testing while signed in measures how the tool treats you, not how it treats a stranger who has never heard of you.
- Count mentions, do not grade the vibe. Write down whether your name appeared and who else was listed. A tally of yes and no across repeated asks is data. An impression of whether the answer felt positive is not.
- Change one thing at a time, then re-run the same list. The only way to know a fix worked is to compare the same questions asked the same way before and after, with enough repeats on both sides to see past the noise.
Almost everyone does the same thing first. They open ChatGPT, type their own business name or the question they think buyers ask, read the answer once, and draw a conclusion from it.
If they were mentioned, the conclusion is that things are fine. If they were not, the conclusion is that they are invisible. Both conclusions are drawn from a sample of one, and a sample of one here is close to worthless.
Here is why a single ask moves around so much, why testing while signed in makes it worse, and what a test that actually tells you something costs in time.
Why does one ask prove nothing?
Because the answer is not fixed. Ask the same question ten times and you will often get several different sets of businesses named, so whichever one you happened to see first tells you very little about what the next person will see.
That makes a single ask an anecdote. It is real, in the sense that it genuinely happened, and it is not evidence of a pattern, in the same way that one coin landing heads is not evidence about the coin.
The practical damage runs in both directions. A coach who happened to be named once relaxes about a problem they still have. A coach who happened to be missed once rebuilds a website that was working. Both decisions were made on noise.
What makes the same question give different answers
Two separate causes. The model chooses between plausible wordings as it writes, so the list it produces varies by chance, and it may read different live pages on different runs, so the raw material varies too.
Neither is a malfunction. OpenAI's own description of how ChatGPT works is a system that generates a response rather than retrieving a stored one, and Perplexity's help center documents answers assembled from a live search whose results move. Variation is what these tools do.
An illustrative run, not a measured one. Look at the top row: three mentions out of ten. A single ask lands on a filled dot about a third of the time and reports a clean yes, and lands on a hollow one the rest of the time and reports a clean no. Same business, same day, opposite conclusions.
Those five rows are illustrative rather than measured, and they show the three shapes you will actually meet: a question you almost always win, a question you never win, and the mixed middle where a single ask is a coin flip. The mixed rows are where nearly all the useful work is.
Why testing on your own account is worse than useless
Because your account knows who you are. It carries your past conversations, any saved memory about you and your work, and an approximate location, and every one of those makes the assistant more likely to bring you up.
So the test returns a flattering answer that no buyer will ever receive. It is not merely inaccurate. It is biased in the one direction guaranteed to make you stop looking at the problem.
Use a logged out session, a private window, and no prior messages in the thread. Start a fresh chat for every ask rather than typing the next question underneath the last one, because the earlier turns steer what follows.
| Element | What most people do | What makes it trustworthy |
|---|---|---|
| Number of asks | One per question | Five to ten per question |
| Session | Signed in, existing chat | Logged out, fresh chat each time |
| Wording | Industry terms, or the business name | The buyer's own sentence, unchanged between runs |
| What gets recorded | An impression of the answer | Named or not named, plus every competitor listed |
| Comparison | Against how it felt last time | Against the same questions run the same way before |
What a test that actually proves something looks like
Five questions, ten asks each, logged out, tallied in a spreadsheet. That is fifty prompts and roughly an hour, and it is enough to tell you which questions you win, which you lose, and who keeps taking the slot.
Choose the five questions by revenue rather than by volume. The question that a buyer asks immediately before hiring someone is worth more than a broad one asked ten times as often by people who are only reading. If you want more coverage than five questions, widen the list before you widen the repeats.
Record three columns per ask: whether you were named, which competitors were named, and whether the answer cited any sources. The competitor column is the one that pays for the exercise, because it tells you which pages are winning the slot you want. The whole procedure, at a slower pace, is the 15-minute test for coaches and consultants, and running it across more than one assistant is covered in testing your business across five AI assistants.
How to read the result once you have it
Read the rows, not the total. A single overall percentage hides the thing you need, which is that some questions are already yours and others are completely closed to you, and those two situations call for opposite responses.
Rows at zero out of ten are not usually about your website. They mean the assistant is not treating you as a candidate at all for that phrasing, usually because nothing published anywhere connects you to that question. Rows in the middle are the ones where a better page moves the number.
Then re-run the identical list after you change something, and only change one thing between runs. Variation between individual asks is normal and has its own causes, which why the same question asked twice gives different answers goes into. Enough repeats on both sides is what lets you see a real move through that noise.
If an hour of prompting is not how you want to spend the afternoon, the free check runs 25 real buyer questions from your niche through ChatGPT at $0 and emails a plain-English readout within the hour. It runs as soon as you ask for it.
Questions we hear the most
Why does ChatGPT give a different answer when I ask the same question twice?
Because the model samples from several plausible continuations rather than repeating one fixed answer, and it may also browse different pages on different runs. Both effects change which businesses get named, even with identical wording.
How many times should I ask each question?
At least five, and ten if the answer matters. Below five you cannot tell a real pattern from chance, and above ten you are usually spending time to sharpen a number that is already clear enough to act on.
Do I need to log out to test properly?
Yes. A signed in session carries your history, saved memory, and location, all of which make the assistant more likely to mention you. That is the one result you can be certain your buyers will not get.
Should I use the exact words my buyers use?
Yes, in full sentences. Buyers type situations rather than keywords, and phrasing changes the answer substantially, so a test written in industry language measures a question nobody actually asks.
Is it cheating to ask the assistant about my own business by name?
It is not cheating, but it measures something different. Asking by name tests whether the assistant knows you exist. Asking the buyer question tests whether it will bring you up unprompted, which is the thing worth money.
What counts as a good result?
Being named in a clear majority of repeats on the questions that lead to work. Occasional mentions on a few questions are a starting point rather than a win, because a buyer usually asks once and sees one answer.
How do I check whether AI recommends my business without spending a whole day on it?
Pick your five highest value buyer questions, ask each one five times in a logged out session, and tally the mentions. That is under an hour and gives you a defensible baseline you can re-run later.
What does the free 25-question check include?
It asks ChatGPT 25 real buyer questions from your niche at $0, then emails a plain-English readout within the hour: how often you were named, who was named in your place, and the first thing to fix. It runs as soon as you ask for it.
Your buyers are already asking. Find out what AI tells them.
The free check asks ChatGPT 25 real buyer questions about your niche and sends you the report within the hour, at $0. If it shows you are already getting named everywhere, we will say so in plain words.
Get my free check →