Two questions keep coming up from store owners. One: “we already use Search Console and store analytics, but neither tells us what happens when someone asks ChatGPT or Gemini for a product recommendation.” Two, from a seller who nearly paid an agency $3,000 for an audit plus $400 a month to keep watching: is that worth it?
The best answer to the second came from another seller in the same thread: if the deliverable is a report you can skip it, at £450 or at $3,000 — and paying to watch a number before you have changed anything is backwards. So here is the check itself. We ran it on ourselves across three platforms, and two of the findings changed what we did.
Step 1 — write the questions your buyers actually ask
Not “is my brand visible”. Twenty to forty questions phrased the way a pre-purchase shopper types them: best <product type> for <use case>, <product type> that <constraint they care about>, alternatives to <the obvious mainstream product>. Keep a few brand-name questions too — they behave completely differently, and that difference turns out to matter.
Write the list into a file. The list is the asset, because you will re-run it.
Step 2 — record what you got, not just whether you appeared
Run each question on ChatGPT and Gemini, and for every answer record four things: were you mentioned; were you cited with an actual link in the sources; which competitors appeared and in what order; and which URLs were cited, for anyone.
Mentioned and cited are not the same, and the gap is the actionable part. In our own runs we found answers that reproduced our positioning nearly verbatim while describing us generically — no brand name, no link. That is a source-text problem, not a ranking problem, and no visibility dashboard would have told us which.
Step 3 — the column that changes what you do
The fourth column is where the money is. When we ran non-brand category questions across three platforms, every citation went to GitHub repositories and third-party list articles — not to our own website. Brand-name questions behaved differently and did cite our own pages.
Two consequences for a store. If non-brand answers in your category cite retailer round-ups, marketplace listings and review sites, rewriting your homepage will not move them; the unit of distribution is whatever the model is actually quoting. And test the two question types separately — average them into one visibility score and you get a number that moves for reasons you cannot act on.
Step 4 — fix the source, not the dashboard
Two failure modes sellers report, both fixable at the source.
Stale facts. One seller had a customer arrive quoting a price they had not charged since the shop opened over a year earlier. Another found ChatGPT recommending a product they had stopped selling months ago, then checked and found two other models showing the same discontinued item. If an old price or a dead product still sits in a cached page, an old post, or a third-party listing, that is what gets repeated back to your customers.
Missing comparison content. A model answering “X vs Y” needs something to quote. If nobody in your category has written an honest comparison, the answer gets assembled from whoever did.
So the action list is: correct the stale facts wherever they physically live, including pages you do not own, then create the missing comparison or use-case content on whichever surface the citations actually point at. The mechanics of why one passage gets quoted and another does not are in our write-up on getting cited by ChatGPT.
Step 5 — re-measure, but not next week
Model-visible metadata lags badly. In our own checks the platforms were reporting a star count and a release date that were months out of date. Do not evaluate a fix within a week of making it. Re-run the same question list after a few weeks and compare mention rate, citation rate and competitor ordering against the first run — which is why step 1 said to keep the list in a file.
One more discipline: this is sampling, not measurement. Answers vary between runs, so repeat each question at least three times across different days before you believe a change. A single run tells you almost nothing.
Running it in Orkas
Orkas ships a built-in SeoGeoAgent, and the question list, the per-run records and the fix list live in one project folder, so run two can diff against run one instead of starting over. The broader workflow — tracking which questions you appear in and turning gaps into pages — is the search and AI answer visibility use case.
What this does not give you
- No traffic numbers. Google Search Console's generative-AI report has no API and, structurally, reports only impressions — you cannot compute an AI click-through rate from it.
- No competitor's real share. You are sampling answers, not measuring a population.
- No guarantee a fix propagates. Some cited sources are outside your control entirely.