How to Measure AI Visibility Improvements Without Fooling Yourself
AI answers vary from one query to the next. Here is an honest way to measure whether your AI visibility work is paying off.
· 2 min read
On this page
Ask ChatGPT the same question twice and you may get two different lists of businesses. Ask from another city, or phrase it differently, and the answer changes again. That makes AI visibility genuinely hard to measure, and easy to fool yourself about. Here’s a way to measure it honestly.
Separate what you control from what you observe
There are two kinds of evidence, and they should never be blended into one magic number without saying so:
- Inputs you control: whether AI crawlers can access your site, whether your entity data is present, whether you answer key questions, whether your profiles are consistent. These can be measured precisely, right now.
- Outcomes you observe: whether AI assistants actually mention or cite you. These are noisy and change over time.
Improving inputs is the work. Outcomes are the result you hope for, and they lag.
Measure inputs with a before and after
- Run a baseline scan of your site and save it.
- Make the fixes.
- Re-test the live site and compare. Each fix is either confirmed or still missing.
This is exactly how the AI Visibility Fixer works: a fix is marked Verified only when a re-test no longer detects the issue, and before/after tables show only real numbers from real scans.
Observe outcomes with a fixed question set
- Write 10 to 20 real questions customers would ask, including your service, your city and comparison questions (“best … in …”, “… vs …”, “how much does … cost in …”).
- Keep the wording identical every time.
- Check the same engines (for example ChatGPT search, Perplexity, Google AI Overviews) from a clean or logged-out session where possible.
- Record: are you mentioned? linked? which competitors appear? which sources are cited?
- Repeat on a schedule, such as monthly, and look at the trend, not single results.
Be careful with AI-estimated scores
Many tools, including parts of Onxeera, offer scores where a model estimates how visible a brand is likely to be. They’re useful for direction, but they’re estimates, not observations. Label them that way in reports, and never present an estimated “potential” score as a promised result.
What to put in a report
- Fixes completed and verified on re-test
- Changes in measured site signals (crawler access, entity data, FAQ coverage, local signals)
- Results for your fixed question set, with dates and engines
- Next fixes, in priority order
Agencies can export all of this as a white-label AI Visibility Fix Report. See AI visibility for agencies.
Frequently asked questions
Why do I get different AI answers each time?
AI assistants generate answers on the fly and can vary by wording, location, account history and time. A single answer is a sample, not a ranking.
What is a realistic way to report to clients?
Report what you can verify: fixes completed and confirmed on re-test, changes in measured site signals, and a fixed set of test questions checked on a regular schedule.
Can AI visibility be tracked automatically?
Partly. Website signals can be re-measured automatically. Tracking which brands AI engines mention needs repeated queries per engine and location; Onxeera is building this.
The Onxeera team builds tools that help businesses get found, understood and recommended by AI search engines like ChatGPT, Perplexity, Gemini and Google AI Overviews.
Find out why AI isn't recommending you
Scan your website for the issues that keep ChatGPT, Gemini and Perplexity from mentioning you, and get the fixes.