How much do AI recommendations change when you ask the same question again?
Fewer than half of the names come back. We put the same ten buying questions to ChatGPT, Gemini, Perplexity and Claude on four dates in October 2026, with live web search on. When the same assistant got the same question again about a week later, on average 43% of the brands it had named appeared again, and the first name in the answer stayed the same 38% of the time. Across many answers the picture was much steadier: the most-named brand appeared in 30% to 38% of all answers on every date.
What we measured
Ten buying questions, such as which tools help a small online shop check its AI visibility. Four assistants, each using its own web search through its official interface. Four dates: 12, 19, 26 and 31 October 2026. That makes 160 answers, all of which we kept. None of the questions names us.
We then compared each answer with the next answer to the same question from the same assistant: 114 comparisons, leaving out six where one of the two answers named no brands.
Same question, same assistant, about a week later
| Assistant | Brands named again | Same first name | Brands in an average answer |
|---|---|---|---|
| All four | 43% | 38% | 6.7 |
| Perplexity | 54% | 43% | 8.1 |
| Gemini | 45% | 44% | 3.1 |
| ChatGPT | 39% | 40% | 8.3 |
| Claude | 34% | 24% | 6.7 |
Over all four dates
Take each brand an assistant named for a question, 599 combinations in all. 59% appeared on just one of the four dates, and 10% appeared on all four.
The assistants do not agree with each other either
For one question on one date, the four assistants named about 22 different brands between them. In 32 of 35 cases, no brand was named by all four, and 85% of the brands were named by only one of them.
What stays steady
| Brand | 12 Oct | 19 Oct | 26 Oct | 31 Oct |
|---|---|---|---|---|
| Otterly.AI | 14 | 12 | 15 | 14 |
| Semrush | 9 | 10 | 11 | 13 |
| Peec AI | 7 | 7 | 9 | 11 |
| Profound | 6 | 7 | 9 | 9 |
Answers naming each brand, out of 40 on each date. Single answers move a lot, but a brand's share across many answers moves far less. That is the useful number.
What this means if you run a brand
- One check proves little. Being named once, or missed once, can be chance. Our own free report asks two questions, so treat it as a first look.
- Measure the same questions repeatedly and read the share of answers over several weeks.
- Check more than one assistant. Doing well on one says little about the others.
The limits of this study
- It covers one market, AI visibility tools, which is the market we sell in. Other categories may behave differently.
- It is small: 160 answers on four dates, and the last gap was five days, not seven.
- Gemini did not always search, and a length limit in our own setup often cut its answers short. We have since fixed both. That is why it names so few brands above, so read its row with care.
- We cannot say why an answer changed. It may be new pages on the web, different search results, or chance.
- An assistant reached through its official interface can answer differently from the app on your phone.
- A helper model read the brand names. We counted longer names that start with a shorter one as the same brand, so "Semrush AI Toolkit" counts as Semrush. That may hide some real change.
- We sell repeated measurement, so we have an interest in this result. The method is here so that you can test it yourself.
Common questions
Why would the same assistant give different answers?
We did not test the cause. Assistants that search the web can find different pages from one day to the next, and these models also vary their wording and choices between runs.
Does this mean AI visibility cannot be measured?
No. It means one answer is not a measurement. In this data, a brand's share across 40 answers moved far less than any single answer did.
How many answers do I need?
We cannot give a firm number from one small study. Forty answers a week was enough to see steady shares for the most-named brands here. Two questions to four assistants is eight answers, which is a first look.
Can I use these numbers?
Yes, with a link to this page.
See where you stand
One report is free. It goes to your inbox in about a minute.
More: AI visibility tools compared · Alternatives to Profound and AIclicks · llms.txt plugin for WordPress and WooCommerce · What an AI visibility report costs · Pricing · Getting started