How to Build the Query Set That Actually Tests Your AI Visibility

Good ideas travel faster when you share them; forward this to someone who’d find it useful.

Every AI visibility test starts the same way: someone writes a list of queries, runs them through ChatGPT or Perplexity or Google AI Mode, and counts how many times a brand shows up.

Almost nobody stops to ask whether the list itself is any good.

That’s not a minor setup problem. The query set defines what the experiment is capable of measuring — which buyer needs are represented, which competitors can appear, which buying stages matter, and ultimately, what “visibility” even means in your results. Change the query set and you change every conclusion that follows from it.

The query set isn’t the setup for the experiment. It is the experiment.

We’ve built query sets for consumer products, B2B industrial distributors, and services companies over the past several months. Here’s what we learned about doing it properly, and what we had to fix along the way.

---

Stop treating AI queries like SEO keywords

Most people building an AI visibility test start from their SEO keyword list. It feels efficient — the keywords already exist, they’re already tied to the business, why not reuse them?

Because keyword syntax doesn’t reliably capture how buyers express intent in a conversational AI interface — and an AI visibility study shouldn’t assume that it does.

Sure, people sometimes type short fragments into ChatGPT. “Best CRM for small business” works fine as a ChatGPT prompt. But a keyword list built for search engine optimization systematically under-represents the longer, more contextual, more constrained queries that are precisely where AI recommendations diverge most from traditional search results.

Test your AI visibility with keyword strings and you’re not measuring how buyers use AI. You’re measuring how well your brand performs against a syntax that doesn’t represent the full range of how people actually ask.

---

Build from buyers. Validate against the taxonomy.

The instinct once you accept that keyword lists don’t work is to swing the other way: define your query types first — comparison, specification, objection-handling, whatever taxonomy you’re using — then write example queries to fill each slot.

This produces guaranteed coverage. It also produces queries that read like a form was filled out, not like a person asking a question. This was the first thing we had to fix: in one of our early engagements, a first-pass query set of 40 queries, built type-first, was scored against a simple classification criterion — does this query contain a complete conversational thought, use buyer language rather than category terminology, and read as something a real person would type? Twenty-eight of the 40 failed. The set technically covered every category in the taxonomy. It was still a bad experiment.

The fix isn’t to abandon the taxonomy. It’s to change the order of operations.

Build bottom-up. Validate top-down.

1. Start from buyer personas, not query types. Who is actually asking — a first-time buyer, a repeat customer, a procurement lead, a gift-giver? Personas should be grounded in real evidence — customer conversations, support data, sales patterns, market research — not invented solely to populate the query set.

2. Write natural queries per persona. What would this specific person actually type into a chat window? Full sentences, real constraints, real hesitation.

3. Tag each query after it’s written — type, buying stage, branded or not — rather than writing to fill a tag.

4. Run every query through four gates:

5. Then, and only then, check coverage against your taxonomy. If a type is missing, ask one question: is there a real buyer query for this type? If yes, go find it. If no — leave the gap. Don’t manufacture a query to satisfy a category nobody actually asks about.

A taxonomy is a checklist for noticing blind spots, not a template for generating content.

The moment you’re writing queries to fill a taxonomy slot instead of to represent a real buyer, you’ve quietly gone back to the method that doesn’t work.

How many queries do you actually need?

Enough to see meaningful patterns. Not so many that you’re measuring variations of the same question.

Our working range across engagements has settled around 35–42 queries. That’s not a universal statistical threshold — it’s a practical range that has been large enough to represent multiple personas, buying stages, and query types without turning the exercise into an unwieldy census of every possible question.

The important distinction is between coverage and precision. A query set at this size can reveal directional visibility patterns — where you consistently appear, where you’re consistently absent, which query types favor you and which don’t. It should not be presented as a precise estimate of a brand’s “share of AI.” AI responses vary across queries, runs, models, platforms, geography, and time. Confidence in a result depends on more than the number of queries alone.

Use the query set to identify patterns. Don’t use it to manufacture false precision.

If you need a defensible estimate of visibility at a particular confidence level, increase the sample, account for repeated runs and platform variation, and design for that level of rigor from the start — rather than treating a fixed query count as statistically sufficient after the fact.

The query universe changes shape by vertical

The method stays constant. What it produces does not.

For a physical product catalog, natural buyer queries generate plenty of specification questions (dimensions, materials, compatibility), deadline questions (when will this arrive), and budget questions (bulk pricing, minimum order quantities) — because products have a catalog, and catalogs have those attributes.

For a services business — no product catalog, no SKUs, nothing to specify dimensions for — that entire category of query barely exists. Buyers ask trust questions instead (”is this company reliable for fragile shipments”), comparison questions (”how does this compare to the other option”), and objection-handling questions (”what happens if something goes wrong”). Push a products-style taxonomy onto a services company and you’ll spend effort trying to force specification queries that no real buyer asks, while under-covering the objection and trust questions that actually drive their decisions.

A balanced query set does not necessarily mean an evenly distributed query taxonomy.

If services naturally generate more trust and comparison questions, that imbalance is itself information about how the market thinks about the category — don’t flatten it to match a template.

This is why the method has to start from personas and real language every time, even when you’ve built a query set for a similar-looking business before. The taxonomy transfers. The specific queries never do.

Validation should be allowed to change the experiment

Bottom-up construction gets you natural language. It doesn’t automatically get you accurate language — the way you and your team phrase something isn’t always the way the market actually talks.

The second thing we had to fix came from a B2B services engagement. We checked our draft query set against Google Trends and Autocomplete data before finalizing it. To be clear about what this step does and doesn’t do: search behavior validates terminology and language — whether you’re using the words the market actually uses. It does not directly tell you what people ask AI, or how often. It’s a language check, not a volume proxy.

What the language check found: the formal industry term we’d used throughout — the one that shows up in the company’s own marketing — was dramatically outperformed in actual search volume by a more colloquial term real people use instead. Five category-level queries had to be rewritten because the “correct” industry language wasn’t the language buyers were actually using.

There’s one more source worth addressing directly, because it’s the first thing most merchants reach for: your own platform search data. Search Console, Shopify search analytics, Amazon search term reports — and for B2B, the search and intent data sitting inside platforms like LinkedIn Campaign Manager, G2 buyer intent reports, HubSpot search analytics, or even the query patterns in your Salesforce knowledge base and support ticketing system. These give you real queries from real buyers who already interacted with your brand. That makes them useful as a validation layer. If your query set is missing language that real buyers already use to find you, platform data will surface the gap. But this data should validate your query set, not seed it. Platform search data captures buyers who already found you. The queries that matter most for AI visibility are the ones where buyers haven’t found you yet — the category-level, comparison, and discovery queries where someone is starting fresh without a brand in mind. Build from those. Use your platform data to check what you might have missed, not to define the universe.

There are also third-party platforms that estimate search behavior and audience intent across the web — these can supplement your language validation, but treat the data as directional, since most of it is modeled rather than observed.

If you skip external validation entirely, you’re trusting your team’s internal vocabulary over the market’s actual behavior. Those two things diverge more often than you’d expect.

The takeaway

Build from how buyers actually ask. Use the taxonomy to find what you missed — not to decide what buyers should ask.

That’s a small reordering of steps. It’s also the difference between a query set that measures your AI visibility and one that measures how well your brand performs against a list that doesn’t represent anyone real.