Are you asking AI the questions your customers actually ask?

A bike workshop can disappear from one AI answer and belong in the next. Before interpreting a visibility score, check what was asked, whose decision it represents and whether the question fits the business at all.

“How do I adjust bicycle gears?”

“Which bike repair shops in Bristol work on electric cargo bikes?”

Imagine a Bristol workshop that offers those repairs. Both questions concern bicycles. The first asks for instructions someone might follow at home. The second asks for a provider. If the workshop is absent from the first answer, that alone says little about whether it appears when someone wants to book a repair.

These are hypothetical questions, not customer records or measured search demand. They illustrate a decision that comes before any visibility score: which situations belong in the test?

A dashboard can calculate an answer precisely while the questions behind it cover the wrong work, the wrong place or customers the business does not serve.

Start with the work someone could buy

Begin with the business's actual services and the decisions customers need help making. An enquiry about collecting a damaged cargo bike points to a different service requirement from an enquiry about repairing a puncture in the workshop.

Useful starting material includes anonymised enquiries, questions raised before a booking and the reasons customers give for choosing a provider. Treat these as evidence about people who reached the business. They cannot reveal every potential customer who went elsewhere.

For each proposed question, write down the service it concerns and why that service fits. A capability mentioned on an old page, at a different branch or by a competitor needs checking before it becomes part of the measurement.

A useful test question should have a plausible customer behind it. That does not mean inventing a detailed persona. “Someone needs a provider for this job in this area” is a better starting point than a string of keywords chosen because they sound valuable.

Keep the question natural

Google's documentation says AI Overviews and AI Mode may break a question into related searches across different subtopics. The two features can also use different techniques and produce different responses and links. A typed prompt is therefore not necessarily the exact search query used to find a page. Google Search Central

For a practical test, keep the details that affect the customer's choice. Leave out details that exist only to steer the answer towards the business.

A cargo-bike question might reasonably include a collection requirement if that is part of the hypothetical customer's situation. Adding an unusual combination of services simply because one particular workshop offers them would make the result harder to interpret.

Brand names need their own category. “Is Acme Cycles a good repair shop?” supplies the assistant with a name. A discovery question asks which providers to consider. Both can be useful, but a result from the first question should not be presented as evidence that the assistant independently found the business.

Check which situations are missing

Lay out the proposed questions beside the services and places the business actually covers. Look for repeated versions of the same easy question and for important work that never appears.

For the hypothetical workshop, a short review could look like this:

There is no measured demand behind these examples. A blank row in a planning sheet shows a coverage gap in the test design, not proof that a large audience is searching there.

Language and market also need to be recorded explicitly. A place named in a question and the country from which a system is queried are separate settings. Keep both visible when comparing results.

A chosen question set is a defined test

Statistics Canada explains that deliberately selected, non-random samples can contain selection bias and need strong assumptions before their results are extended to a wider population. Its guidance concerns statistical sampling generally; it is not a study of AI visibility. Statistics Canada

The practical implication for a manually chosen prompt set is to state its scope. A result can describe how often the business was named in the recorded answers to the selected questions. It cannot, by itself, establish what share of all potential customers would encounter the business.

Rephrasing one question repeatedly does not demonstrate how common that customer need is. Nor does giving every question equal weight prove that each represents equal demand. Those are choices in the test design, and they should be visible to whoever reads the result.

Keep a stable set for follow-up checks

Once the questions fit the business, keep their wording and settings for the next comparison. Record changes to the question set instead of quietly replacing a difficult question with one that produces a better answer.

Keep a separate list for exploration. A new service, a newly served area or a question repeatedly raised by customers may justify a revised set. Preserve the earlier results and mark where the basis changed.

Conversation history needs the same care. Google describes follow-up questions that retain the context of an earlier answer. A fresh question and a follow-up inside an existing conversation should therefore be recorded as different test conditions. Google's announcement

Repeating a test helps examine changing answers, a subject covered in The core and the carousel. Separately record whether the business was named, linked or explicitly recommended; Cited is not the same as named explains that distinction.

Before interpreting the next visibility report, ask to see its question list. Check whether each question fits an actual service, whether the market settings are stated and whether the comparison uses the same basis. Those checks make it possible to decide what the result can help the business do.