Last updated October 2026.
A tracking dashboard can show your brand at 40% visibility on Monday and 15% on Tuesday, with nothing about your website, your prompts, or the model changing in between. That swing isn’t a bug in the tool. It’s what happens when a grade gets built from one AI answer instead of a real sample.
Every tracker on this lab’s board claims to measure how often an AI engine mentions a brand. Before any of them earns a letter grade here, this lab runs the same prompt through the same engine 20 times, not once, and scores the mention rate across the full set. Here’s the statistical reason that floor exists, and where it sits next to the industry’s most rigorous published method.
A single answer is a coin flip, not a score
Large language models pick each word from a range of possible options, not a fixed script. Ask the same question twice and the model can name a different set of brands, cite a different source, or skip the category altogether. That’s the system working as designed, not a glitch in the engine or the tool reading it.
It also means a single answer only tells you what the model said once. A tracker that queries a prompt once and reports the result as a score is reporting an anecdote with a percent sign attached. The fix isn’t a smarter algorithm behind the tool. It’s more draws from the same distribution before anyone calls the number a result.
The same instability shows up in what gets cited, not just whether a brand gets named. Run one prompt against one engine and you might see three sources behind the answer. Run the identical prompt an hour later and the source list can shift, even when the brand mention itself stays the same. A grading method that only checks one run treats that shifting citation list as fixed, when it’s really just the first frame of a longer clip.
What 20 runs buys you
Picture a brand with a true 50% chance of getting mentioned in any single answer, the hardest case for sampling because the outcome sits exactly on the fence. Basic probability, not any one vendor’s data, shows what a tracker sees at three sample sizes.
| Runs per prompt | Typical outcome you’d see (true rate: 50%) | What that spread tells you |
|---|---|---|
| 1 | 0% or 100%, no middle ground exists | One answer, not a rate |
| 5 | Lands between 20% and 80%, almost every time | A rough signal, wide enough to hide a real shift |
| 20 | Lands between 30% and 70%, almost every time | Narrow enough to separate a real move from noise |
That spread comes from a simple rule: variance shrinks with the square root of the sample size, not in a straight line. Moving from 1 run to 20 buys real ground. Moving from 20 to 40 buys less than the jump suggests, which is exactly why sampling decisions run into diminishing returns fast.
Twenty is a floor, not a ceiling
Twenty runs won’t match a lab with a seven-figure prompt budget, and it isn’t meant to. It’s the point where a mention rate stops being a coin flip and starts being a number worth putting in a report, without turning every grading cycle into a weeks-long enterprise engagement.
The number also has to scale. This lab tests nine tools across five engines, and a grading run repeats that cycle on a fixed schedule. Twenty runs per prompt keeps that math honest without stretching one grading cycle across weeks of API calls, the trade every methodology makes between statistical depth and how often the board actually gets to update.
Evertune sets the high bar in this category. Its own published methodology states it samples each tracked prompt up to 100 times across 11 models, not once and done. In Evertune’s own test, a prompt sampled once carried a margin of error as high as 31 points at the topic level; sampled 100 times, that margin narrowed to about 3 points (Evertune, July 2026). That’s the strongest published sampling depth of any tool on this lab’s benchmark, and it comes at a price built for the buyer who needs it: Evertune’s plans start at $3,000 a month on an annual contract, not a self-serve signup.
What the rest of the field publishes
Coverage of this detail varies a lot across the field, and a gap in the public record isn’t automatically a red flag.
Profound samples a prompt once a day by default. Its own published test compared that default against a 10-times-daily cadence and found the extra runs mattered more for citation-share noise than for overall visibility. Temso takes a different approach entirely: it scales its tracked-prompt count by Growth tier, from 125 up to 1,000, and collects readings from real chat interfaces rather than model APIs. Its public materials don’t publish a separate repeat-sample count per prompt, which puts it in the same position as most of the field on this one detail.
What 20 runs doesn’t replace
Sampling depth answers one question: how much noise sits inside a single number. It doesn’t answer the rest of what makes a tracker worth paying for.
- Engine coverage across the assistants your buyers actually use.
- A refresh cadence that catches a real shift within days, not months.
- A workflow that turns a mention-rate drop into a fix, not just a chart.
A tool can nail the sampling math and still leave you staring at a citation with nothing to do about it. Weigh sampling depth against those three, not on its own.
Check where sampling sits on the full board
Sampling depth is one input on a longer scorecard, weighed against engine coverage, refresh cadence, and what a tool does with a finding once it lands. See how every tracker on this lab’s board documents its own methodology, sampling, and refresh cadence on the full LLM visibility ranking, and read the exact protocol behind every grade on the measurement page.