No, ChatGPT does not give everyone the same answer. We tested this directly at Honeyb (that is us, so read the method before the numbers). On 13 July 2026 we sent 20 buyer-intent prompts, each three times, to four AI engines via API: ChatGPT (gpt-5-mini), Gemini (gemini-2.5-flash), Claude (claude-haiku-4-5) and Perplexity (sonar). That is 240 answers to identical questions, asked minutes apart, with nothing else changed.
ChatGPT swapped its number-one recommended brand between identical consecutive runs 35% of the time. Gemini was the least stable at 44%. If you asked ChatGPT "what is the best CRM" twice in a row, roughly one time in three you would get a different top pick.
How much the same question changes the answer
We tracked two things per engine. Top-pick change rate: how often the first-named brand differed between two identical back-to-back runs. Brand-set overlap: how much of the full recommended brand list survived from one run to the next.
| Engine | Top-pick change rate | Brand-set overlap between runs | Avg brands per answer |
|---|---|---|---|
| Gemini | 44% | 54% | 4.8 |
| Perplexity | 43% | 61% | 5.2 |
| ChatGPT | 35% | 42% | 5.1 |
| Claude | 28% | 67% | 5.1 |
Read that middle column carefully. ChatGPT kept its top pick more often than Gemini, but its full brand list was the least stable of the four: only 42% of the brands it named in one run reappeared in the next identical run. Claude was the most consistent on both measures.
Top-pick change rate
How often the top recommendation changes between identical runs
This matches what others have found at larger scale. SparkToro reports that the same AI query changes roughly 70% of the time. Our numbers are lower because we measured only the top pick and the brand set, not the full answer text, and we ran everything within a single session on fixed models.
The four engines also disagree with each other
Run-to-run variance is only half the problem. The engines rarely agree among themselves either. Across our 20 prompts, the share of prompts where two engines named the same top brand ranged from 20% to 53%.
| Engine pair | Same top brand | Mean brand-list overlap |
|---|---|---|
| Gemini and Claude | 53% | 44% |
| ChatGPT and Gemini | 47% | 29% |
| ChatGPT and Claude | 40% | 27% |
| Gemini and Perplexity | 37% | 47% |
| Claude and Perplexity | 35% | 37% |
| ChatGPT and Perplexity | 20% | 25% |
ChatGPT and Perplexity agreed on the top brand for just 4 of 20 prompts. So "does ChatGPT give everyone the same answer" understates the issue. Even if it did, your buyers are spread across four engines that recommend different brands for the same question. Our guide to what AI visibility actually means covers why each engine behaves like a separate channel.
Want to see this in action?
See how every major AI model talks about your brand. Free to start.
Why the answers keep changing
Three mechanisms drive this, and none of them is a bug.
Sampling. These models generate text probabilistically. Two brands with similar internal weight can swap places from one generation to the next. That alone explains much of Claude's 28% top-pick churn.
Retrieval variation. Every engine in our test ran live web retrieval, and the pages fetched differ between runs. ChatGPT averaged 15.0 sources per answer, drawn from 445 distinct domains across 900 total citations, with its top three domains accounting for just 7.6% of citations. When the evidence pool is that long-tailed, a different fetch produces a different shortlist. Perplexity leaned hardest on community content: Reddit alone was 14% of its citations and YouTube another 8%, consistent with Semrush's finding that Reddit is 40.1% of all AI citations.
Model updates. Providers ship silent model and retrieval changes. A reading taken before an update does not describe the system after it.
One caveat on sources: via API, every engine returned sources on 100% of answers. Consumer apps often show fewer or none, so what your buyer sees cites less than what we measured. Gemini added its own wrinkle by wrapping 429 of its 580 source URLs (74%) in grounding redirects that mask the real source.
What this means if you checked ChatGPT once
If you asked ChatGPT about your category last month and saw your brand, that reading had roughly a one-in-three chance of flipping on the very next run. It says almost nothing about what the engine tells buyers this week, on this model version, in this retrieval mood.
The practical consequence is that AI visibility is a series, not a snapshot. A single check cannot distinguish "we are reliably recommended" from "we got lucky on that generation". We wrote up the statistics of this in more detail in why spot-checking AI answers fails, and if you want to run repeated checks without doing it by hand, see our comparison of AI model tracking dashboards, which covers Honeyb (our product) alongside the alternatives.
A sensible baseline: track a fixed prompt set across all four engines on a schedule, and treat share of runs where you appear as the metric, not any individual answer. Ahrefs' research suggests the lever that moves that share is third-party mentions and video rather than on-page changes, which is a different playbook from classic SEO.
Want to know what the engines say about your brand right now, measured properly across repeated runs rather than one lucky prompt? Start with a free AI visibility check.
