Key Takeaways
- Expect the list to change on almost every run. Ask an assistant the same question twice and you will rarely get the same names in the same order. That is normal, and on its own it means nothing about your business.
- Watch your share of answers, not any single answer. How often you are named across many runs moves slowly and for real reasons. One answer moves for no reason at all.
- Ignore your position in the list. The order of names almost never repeats. Being named third today and first tomorrow is noise, not progress.
- Look for a change in level after a change in sources. The share moves when what the assistant reads about you moves: a new listing, a clearer page, a newer model. Tie every check to something you actually changed.
- Compare like with like. Same questions, same assistant, similar number of runs. A month of daily checks against a single screenshot from last spring tells you nothing.
Ask an AI assistant which coaches it recommends in your city, then ask again tomorrow. The names will probably be different. Ask a third time and they will be different again.
That leaves most business owners with one question: how often do AI answers actually change, and which changes are real? In 2026 there is finally public evidence from repeated runs, and it points to a clear answer with two parts.
Here is what changes on almost every run, what changes slowly and why, five kinds of change side by side, and how to track your own without being fooled.
How often do AI answers change?
AI answers change on almost every run in their wording and in the exact list of names. What changes slowly is how often a given business is named across many runs, and that is the part worth watching.
So there are two speeds. The fast one is the shuffle: different names in a different order each time you ask. The slow one is the level: the share of answers a business shows up in, which holds steady for weeks and moves when something real changes.
Most confusion about AI visibility comes from mixing the two up. A business owner sees the shuffle, reads it as the level, and decides something worked or failed when nothing happened at all.
What changes on almost every run
The list of names changes on almost every run, and so does their order. Repeating the same question is enough to get a different answer, with no change to the question, the businesses or the web.
The largest public test of this is a study by SparkToro and Gumshoe, published in January 2026. Six hundred volunteers ran 12 prompts through ChatGPT, Claude and Google's AI a combined 2,961 times. The researchers found less than a 1 in 100 chance that ChatGPT or Google's AI would give the same list of brands in any two responses, and the same list in the same order came up less than once in 1,000 runs.
The same study found the other half of the picture. The leading names kept turning up: one hospital appeared in 69 of 71 responses. The authors concluded that how often a name appears across many prompts, run many times, is a reasonable thing to measure, and a ranking position is not. They are candid that the study was not peer reviewed and that larger samples are needed.
Google says something similar about its own products in its guidance on AI features in Search: AI Mode and AI Overviews may use different models and techniques, so the responses and links they show will vary.
What changes slowly, and why
How often a business is named changes slowly, because it depends on what the assistant reads and knows, and those things change slowly. The figure shows one question run daily for four weeks, for three illustrative lanes.
The competitor named most, month 1
Week 1: 5 of 7
Week 2: 6 of 7
Week 3: 5 of 7
Week 4: 5 of 7
Named on 21 of 28 days
You, month 1
Week 1: 1 of 7
Week 2: 2 of 7
Week 3: 1 of 7
Week 4: 1 of 7
Named on 5 of 28 days
You, month 3, after two new listings
Week 1: 3 of 7
Week 2: 4 of 7
Week 3: 3 of 7
Week 4: 4 of 7
Named on 14 of 28 days
Day to day, every lane flickers and no two weeks look alike. Week to week, each lane holds its level: the competitor 5 or 6 of 7, you 1 or 2 of 7. Only the lane after a real change sits at a new level, 3 or 4 of 7.
Every number in that figure is illustrative. The pattern is what matters. If you looked only at one day, the competitor's lane would sometimes show them missing and yours would sometimes show you named, and neither would mean anything. Read by the week, the levels are steady, and the one lane that moved is the one where something real happened first.
The slow level moves for three kinds of reason. What the assistant can find about you changes: a new listing, a clearer page, an article that mentions you. The assistant itself changes: a newer model, with a newer picture of the world. Or the competition changes, when someone else becomes easier to recommend. None of these happens daily.
Five things that change, and how often
Different parts of an answer change at very different speeds. The table sets out five of them, from fastest to slowest, with what drives each one and whether it deserves your attention.
| What changes | How often | What drives it | Should you react? |
|---|---|---|---|
| The wording of the answer | Every run | Each answer is written fresh | No |
| The order of the names | Almost every run | Ordinary variation between runs | No |
| Which names appear in one answer | Most runs, within a fairly stable set | Ordinary variation, and how the question is worded | Only across many runs |
| How often a business is named across many runs | Over weeks to months | What the assistant can find about each business | Yes, this is the number to watch |
| What the assistant knows from training | Only when the model is updated | A new model with newer training data | Yes, but you cannot schedule it |
Read the last column and the rule is simple. React to changes in level, measured over many runs. Ignore everything above that row in the table, however dramatic it looks in a screenshot.
How to track change without being fooled
Measure the same questions the same way at regular intervals, and judge each check by how often you were named, not by any one answer. That is the whole method.
Three habits keep it honest. Use enough questions that one odd answer cannot swing the result. Write down what you changed and when, so a move in level can be tied to a cause. And give each change time to be read before judging it; for how long that tends to take, see how long until AI answers reflect a fix. For how often to check at all, see how often to check whether AI recommends your business.
If you want the causes of run-to-run variation in more detail, see why the same question asked twice gives different answers.
The damaging admission: nobody outside these companies can tell you the exact rate at which answers about your niche will change. The public study is a few thousand runs across a dozen prompts, and your market is not in it. What it establishes is the shape, fast shuffle and slow level, and that shape is enough to stop you misreading your own results.
If you want to know whether ChatGPT names you today, start with the free check. It asks ChatGPT 25 real buyer questions from your niche at $0 and emails a plain-English readout: your score, who got named in your place, and the one fix to start with. It runs as soon as you ask for it, and the report lands within the hour between 7am and 5pm Central, or by 10am the next morning outside those hours.
Questions we hear the most
How often do AI answers change?
The wording and the list of names change on almost every run. How often a particular business is named across many runs changes much more slowly, usually only when what the assistant reads about that business changes, or when the assistant itself is updated.
Why did ChatGPT name me yesterday and not today?
Most likely ordinary variation. Assistants build each answer fresh, and the exact list rarely repeats. If you are named on some runs and not others, the useful number is how often you are named across many runs, not whether you were named on one.
Does my position in an AI answer matter?
Very little. In SparkToro’s study of nearly 3,000 runs, the same list in the same order almost never came back twice. Being named at all, and how often, is the meaningful signal. Where you appear in the list moves too much to track.
How long does it take for AI answers to reflect a change to my website?
It depends on how the assistant answers. When it searches the web, a change can show up once the updated page has been read again. When it answers from memory, the change waits for a future model. No assistant publishes a fixed schedule.
How many runs do I need before I can trust a change?
More than a handful. A change from one answer to the next tells you nothing. A change in how often you are named across dozens of questions, measured the same way before and after, is worth taking seriously.
Does the free check account for answers changing?
It rests on 25 answers rather than one. The free check asks ChatGPT 25 real buyer questions from your niche at $0, so a single unusual answer cannot decide your score, and it shows who is named in your place.
How does the Answer Watch handle answers that change every week?
It re-runs your board every week and tells you what moved, so you can tell a real change in level from ordinary variation. It is $399 a month, or $300 a month if added when you buy the $500 audit.
Your buyers are already asking. Find out what AI tells them.
The free check asks ChatGPT 25 real buyer questions about your niche and sends you the report within the hour, at $0. If it shows you are already getting named everywhere, we will say so in plain words.
Get my free check →