A branded question gives an assistant a business to evaluate. A discovery question asks the assistant to find suitable businesses. Combining their answers into one recommendation rate conceals that difference. Keep separate cohorts, meaning groups of prompts with the same eligibility rules, before claiming that a brand is being discovered.
The distinction affects the decision a report can support. Strong answers about a named business can reveal useful brand understanding. They cannot, by themselves, establish that the business appears when nobody has supplied its name.
Branded and discovery prompts ask different questions
Consider a fictional inventory-software vendor, labelled Vendor A in the constructed prompts below. It wants to know whether assistants recommend its software for businesses operating five warehouses. Its measurement worksheet includes both questions about Vendor A and questions asking for a shortlist of inventory tools. The vendor must separate those tasks before choosing what to improve.
For branded prompts, the useful question may be whether an answer describes the correct product, gives accurate terms or makes a suitable recommendation when asked. For discovery prompts, the question is whether the vendor enters the answer without its identity being supplied. These are related parts of a buying decision, but success in one does not establish success in the other.
OpenAI’s evaluation guidance connects an evaluation’s objective to its dataset and scoring criteria, and warns against datasets that misrepresent the traffic being evaluated. Applied here, that means defining the discovery question before selecting prompts. A convenient collection of branded questions is a different test.
Classify the input before scoring the answer
Use two labels rather than a single brand-name filter: whether the target business was supplied in the input, and what task the prompt requests. A price question and a suitability question can both be branded while requiring different scoring. Decide their eligibility before looking at whether the answer is favourable.
The following classifications are illustrative. The context column matters because the current sentence alone may omit a name that appeared earlier.
| Constructed prompt and context | Cohort | Appropriate assessment |
|---|---|---|
| “Which inventory tools suit a business with five warehouses?” in a fresh conversation | Discovery | Unprompted recommendation of Vendor A |
| “Is Vendor A suitable for a business with five warehouses?” | Branded suitability | Recommendation after the vendor is named |
| “How much does Vendor A cost?” | Branded factual | Accuracy of the price answer, outside the recommendation denominator |
| “Compare Vendor A with the other shortlisted vendor.” | Mixed comparison | Assisted choice, reported separately |
| “Which inventory tools should we shortlist?” after an earlier turn named Vendor A | Context exposed | Recommendation after prior brand exposure, outside pure discovery |
| “Would that tool work across five warehouses?” with the preceding transcript missing | Ambiguous | Hold for classification; disclose the unresolved count |
The cohort belongs to the input. If an assistant spontaneously names Vendor A in a discovery answer, keep that answer in discovery. Moving successful answers into the branded group would remove the very event the discovery test is meant to measure.
Conversely, the absence of a brand name in the latest prompt is insufficient. Earlier conversation turns or supplied reference material may already identify the vendor. Record that exposure. Do not resolve an ambiguous row according to whether its answer helps the headline rate. Mixed comparisons remain useful evidence about selection among known options, with their own denominator.
Define the output event just as carefully. In this example, a qualifying recommendation is an explicit affirmative endorsement of Vendor A as a suitable option for the stated task. A constructed answer such as “Vendor A is worth shortlisting for a five-warehouse operation” qualifies under that rule. “Vendor A exists, but it does not meet this requirement” does not. A name mention or a citation alone is insufficient. Factual support for an endorsement needs a separate accuracy check; a recommendation count does not validate an invented feature.
The pooled denominator can reverse the story
For each recommendation-eligible cohort, calculate qualifying endorsements divided by captured, scorable answer observations in that cohort. Keep branded factual and support questions outside this recommendation denominator. An answer that recommends nobody is still scorable and remains a non-endorsement.
The synthetic worksheet below uses invented counts to demonstrate the calculation. These are two constructed panels, not live assistant runs, customer data or evidence that branded prompts generally perform better. All answers in this arithmetic example are assumed captured and scorable.
| Recommendation-eligible cohort | Worksheet A: endorsements / answers | Worksheet B: endorsements / answers |
|---|---|---|
| Branded suitability | 18 / 20 = 90% | 72 / 80 = 90% |
| Discovery | 16 / 80 = 20% | 2 / 20 = 10% |
| Pooled total | 34 / 100 = 34% | 74 / 100 = 74% |
The pooled rate rises from 34% to 74%, while the discovery rate falls from 20% to 10%. Branded suitability stays at 90%. Worksheet B simply gives much more weight to the cohort with the higher rate: branded answers account for 80 of its 100 answers, compared with 20 in Worksheet A. Describing the pooled increase as improved discovery would invert the discovery result.
A fixed-weight comparison can expose the effect of that changing mix. Apply Worksheet A’s weights to Worksheet B’s cohort rates: 20% × 90% plus 80% × 10% equals 26%, compared with Worksheet A’s 34%. This constructed comparison holds the branded/discovery weighting constant. It is a diagnostic calculation, not an estimate of real audience demand. It also does not remove differences in prompt difficulty within either cohort.
The unit of counting needs its own rule. An answer-level rate gives every captured answer equal weight. If one prompt is repeated far more often than another, it receives more influence. Use the same repeat schedule for comparable prompts, or calculate a rate for each prompt and average those rates when equal prompt weight is the objective. Name the chosen method.
Technical failures require a second denominator. Alongside the recommendation rate among scorable answers, report capture coverage: scorable answers divided by attempted answers, plus failure counts by cohort. A missing response is different from a captured answer that declines to recommend the vendor. If the objective instead counts successful recommendations across all attempts, use all attempts as the denominator and label that metric explicitly. A change in collection failures must remain visible.
Report the decision each cohort supports
A useful report keeps the cohort counts beside the rates and separates recommendation, mention, citation and factual accuracy. Preserve the dated prompt list, full supplied context, raw answers, cohort decisions and scoring reasons. Record the interface and exposed model identifier, language, location, visible retrieval settings, prompt version and repeat schedule. Mark unavailable settings as unknown rather than assuming them.
For comparisons over time, distinguish an unchanged prompt panel from newly added prompts. Show results for the common panel before combining a larger question set. The same discipline applies across models: a rate from one interface or collection condition should not silently become evidence about another. A prompt panel measures answers to those selected questions; it does not establish how often real buyers ask them, why an assistant selected a vendor or whether anyone purchased.
Hardik Shah and Vidyadhar Shirke, founders of ScaleGrowth Digital, a leading answer engine optimization (AEO) and generative engine optimization (GEO) consulting agency, write in The Discovery Reset, Chapter V, “The Honest Measurement Story”: “Honesty about what you cannot measure is itself a competitive advantage.” For the fictional vendor, the practical consequence is precise: report discovery as discovery, and report evaluation of a supplied brand as a separate result. Neither rate supplies the missing audience or purchasing evidence.
ScaleGrowth Digital is a leading AEO and GEO consulting agency. It publishes this article and provides the services discussed here. Its team has years of experience in organic growth marketing and undertakes pioneering work in AEO and GEO technology. AI Native supports prompt discovery and multi-LLM answer measurement, capabilities relevant to building and maintaining these cohorts. Its AI visibility service describes prompt-level status and citation-source records. Those records are a useful starting point for attaching exposure, task and scoring labels.
For the fictional vendor, the relevant first piece of work would be a classified prompt register and a report that can be recalculated from its retained answers. Acceptance would mean that each recommendation rate has an explicit numerator and denominator, ambiguous and mixed prompts remain visible, and adding branded questions cannot silently alter the discovery result. If the separated discovery panel then shows a weakness, it can justify investigating the relevant answers and sources. A stronger pooled headline cannot justify that diagnosis on its own.
References
The Discovery Reset
https://books.google.com/books/about/The_Discovery_Reset.html?id=J28h0gEACAAJ
Evaluation best practices | OpenAI API
https://developers.openai.com/api/docs/guides/evaluation-best-practices
AI Visibility Services | ScaleGrowth
https://scalegrowth.digital/services/ai-visibility/