How to measure AI search visibility without fooling yourself
How to sample AI answers for buyer questions, what to record, and why the results are a snapshot to compare over time and not a ranking.
Why it is harder than rank tracking, and what to record
A ranking has a position. An AI answer does not. It is generated text, and the same question can produce different wording, different sources and different brands from one run to the next. Different tools and different settings give different answers again.
More
So the honest framing is sampling. You ask a defined set of questions, record what comes back, and compare samples over time. You are measuring a tendency, not a position.
| Signal | Meaning | Limit |
|---|---|---|
| Mentioned | The answer names your brand in its text | Can be a passing mention or a negative one |
| Cited | The answer links to or lists one of your pages as a source | A citation does not mean the page drove the answer |
| Rivals named | Other products the answer lists for the same question | Shows who you are compared with, not who is better |
Record all three. A brand can be mentioned without being cited, and cited without being mentioned. Each case suggests a different fix.
Build a question set
Write the questions a buyer would ask when looking for a product like yours. Use their words, not yours, and avoid including your brand name in most of them, since the point is to see whether you appear unprompted.
More
- Problem questions: 'how do I fix X without Y'.
- Category questions: 'what tools help with X'.
- Comparison questions: 'X vs Y' where Y is a rival.
- A small number of brand questions, to check that the facts stated about you are right.
Keep the set small and stable. Changing the questions every time makes comparison impossible. Five to ten well-chosen questions are enough to see a pattern.
Run it and record it
- Choose the AI search tool or model you want to sample, and keep that choice fixed.
- Ask each question in a fresh session, without earlier conversation influencing it.
- Record the date, the question, whether the brand was mentioned, which URLs were cited, and which rivals appeared.
- Repeat the same set at a regular interval, such as monthly.
- Where results vary a lot between runs, run a question more than once and note the spread.
More
What to do with the results
If rivals are named and you are not, check whether you have a clear page answering that question. If you are mentioned with wrong facts, fix the source: your own pages, documentation and any profile pages you control. If competitors are cited and you are not, look at what their cited pages contain that yours do not.
More
None of this has a guaranteed effect. Better, clearer pages are worth having for readers anyway, which is a reasonable reason to make them.
Common errors, and how to report results
- Reading one run as a trend. Answers vary, so a single sample may differ from the next for reasons unrelated to your site.
- Prompting with your brand name and then counting the mention. Only unprompted appearances say something about discovery.
- Changing the question wording between samples, which breaks comparison.
- Mixing different tools into one number. Each tool has its own sources and behaviour, so report them separately.
- Treating a rival's presence as proof that they are better. Models may repeat what is widely written about a category.
More
Report counts with the sample size next to them, such as 'mentioned in 3 of 5 answers on this date'. Name the model or tool used, and list the questions. A reader should be able to repeat the sample.
| Date | Question | Mentioned | Cited URL | Rivals named |
|---|---|---|---|---|
| Date of run | The exact question text | Yes or no | Your page, or none | Names as they appear |
Alongside the log, keep your normal search metrics. AI-answer sampling does not replace Search Console, and the two sources answer different questions.
Making your own pages easier to quote
You cannot control what a model writes, but you can control what it finds on your site. Pages that state a fact plainly, near the top, with the supporting detail below, are easier for any reader or system to use accurately.
More
- Answer the question in the first paragraph, then explain.
- Use tables for comparisons and lists for steps, because they are easy to read and to extract.
- Keep names, product facts and dates consistent across your site and documentation.
- Say what the product does not do. Models and readers both benefit from accurate limits.
These are good writing habits more than tactics, and none of them guarantees a citation. If an answer states something wrong about your product, the useful response is to check what your own pages say, correct any ambiguity there, and note the error in your log so you can see later whether it persists.
Where BlogFactory fits
BlogFactory's AI search visibility check asks a web-search model, through your own OpenRouter key, up to five buyer questions. For each answer it records whether the brand is mentioned, whether the site is cited, and which rivals are named. The last result is stored on the site. It runs only when you click, because it costs tokens plus a search fee per question.
More
- Needs: your OpenRouter key and the buyer questions to ask.
- Cannot do: report a ranking or a share of voice. It is a sample from one model on one day.
- Cannot do: show what other AI products would answer, or promise any citation or mention.
Try it in BlogFactory
- AI search visibility
- llms.txt and FAQ schema
- MCP SEO agents
- Brave, DataForSEO and OpenRouter keys in the documentation
- The four read-only MCP research tools in the documentation
- Next guide: What is llms.txt, and should your site have one?
Questions
Can I rank in AI answers the way I rank in search?
There is no stable position to hold. Answers are generated and vary between runs, so measure tendencies across samples instead.
How many questions should I track?
Five to ten, kept stable over time. More questions add cost and noise without adding much clarity.
What is the difference between a mention and a citation?
A mention is your brand appearing in the answer text. A citation is a link or source reference to one of your pages. They can occur separately.
Does the check cost money?
Yes. With BlogFactory it uses tokens through your OpenRouter key plus a search fee per question, so it only runs on demand.
Will changing my pages change the answers?
Possibly, but there is no guarantee. Make pages clearer for readers, then compare later samples.
Plan it, draft it, approve it.
BlogFactory does the research and drafting steps in this guide. You approve every draft, and nothing publishes itself.