Measure recommendation frequency as the share of eligible answers that positively propose a named option. Record explicit comparative preference separately, together with the context in which it is expressed. List position supplies neither label on its own. For answer engine optimization (AEO) and generative engine optimization (GEO), reporting those distinctions prevents a change in presentation from becoming a claim about preference.
Recommendation labels start with what the answer says
An answer can name a product without recommending it. It can recommend several products without preferring one. Use separate fields for inclusion, recommendation and explicit preference, rather than forcing every mention onto a single ranking scale.
Consider a fictional scheduling-software publisher measuring its invented product Aster against the equally fictional Birch and Cedar. The publisher is deciding whether an apparent loss of first place warrants changing its product content. Its proposed dashboard treats the first listed tool as the preferred tool. That rule would make the chart easy to populate, but it would count a judgement the answer may never express.
The constructed exercise below was created and coded on 17 October 2026. The answers are authored examples, not outputs captured from a live AI system. Every complete example uses the same input: “Which of the fictional scheduling tools Aster, Birch and Cedar would you recommend for a five-person service team that must schedule while offline? State any preference and reason. Do not assume all are suitable.”
The counting unit is Aster in one eligible answer. Aster earns at most one inclusion and one recommendation count per answer, however often its name appears. Names in the input prompt do not count as inclusion in the answer. The following scoring guide keeps the labels independent.
| Field | Positive code | Other dispositions |
|---|---|---|
| Inclusion, I | 1 when the answer names Aster, including a negative mention | 0 when Aster is absent from the answer |
| Recommendation, R | Yes when the answer positively proposes Aster as a choice for the stated need, including membership of a recommended set | No for a directory mention, rejection or absent endorsement; ? for unresolved or unclear endorsement |
| Explicit preference, P | For when comparative wording puts Aster above named alternatives in a stated context | Against when another option is explicitly preferred to Aster; tie for stated parity; none when no explicit comparative preference appears |
A recommendation of an unordered set applies equally to its members for counting purposes. “I recommend Aster, Birch and Cedar, in no particular order” gives each a recommendation label. Numbering that same set does not manufacture a winner. Likewise, a negative mention increases inclusion but supplies no positive recommendation.
Preference labels need a comparison and a context
Use the actual comparative clause as the evidence for preference. “I prefer Aster to Birch for a five-person team because it requires less administration” supports a preference-for-Aster label tied to that team and criterion. It does not establish that Aster is universally superior. A stated tie is also useful information, but it is not a strict preference for either product.
The HTML Standard provides a useful distinction: an unordered list has no meaning that depends materially on item order, whereas an ordered list does. Its ordered-list example is a chronology of countries lived in. Meaningful order can therefore describe something other than preference. For answer coding, neither numbering nor visual prominence substitutes for comparative language; the surrounding words must establish what the order means.
Here are the complete constructed answers and their Aster codes. IDs 01 to 10 refer to records SYN58-01 to SYN58-10. In the P column, “none” means no explicit comparative preference was detected, not that an AI has no internal preference.
| ID | Constructed answer | I | R | P |
|---|---|---|---|---|
| 01 | The candidates are Aster, Birch and Cedar. This is a directory list; I am not recommending any. | 1 | No | none |
| 02 | I recommend the following options, in no particular order: 1. Aster 2. Birch 3. Cedar. | 1 | Yes | none |
| 03 | I recommend the following options, in no particular order: 1. Birch 2. Aster 3. Cedar. | 1 | Yes | none |
| 04 | For this five-person team, I prefer Aster to Birch and Cedar because it requires less administration. I recommend Aster. | 1 | Yes | for |
| 05 | For the offline requirement, I prefer Birch to Aster. I recommend Birch. | 1 | No | against |
| 06 | I recommend Aster for this team, but I cannot recommend Aster until its offline support is confirmed. | 1 | ? | none |
| 07 | I recommend Birch and Cedar for the stated requirement. | 0 | No | none |
| 08 | Aster cannot work offline, so I do not recommend it for your stated need. | 1 | No | none |
| 09 | I recommend Aster for this team. Aster is the option I suggest considering. | 1 | Yes | none |
| 10 | Aster and Birch are equally suitable for the stated need. I recommend either; I have no preference between them. | 1 | Yes | tie |
Records 02 and 03 are the decision-changing counterexample. Aster moves from first to second position while the recommended set remains identical. Both answers recommend Aster; neither expresses a preference. Replacing 02 with 03 in a later comparison would reduce a first-position score without reducing Aster’s recommendation count. The fictional publisher would have no preference evidence from that change to justify rewriting content merely to regain first place.
Records 04 and 05 contain a different kind of signal. Record 04 prefers Aster for lower administration; record 05 prefers Birch for the offline requirement. Preserve both criteria beside their labels. Record 08 rejects Aster on an offline claim without an explicit comparison, so its preference code remains none. These clauses identify claims to check against product records. They do not establish which tool actually works offline.
Eligible answers determine every denominator
Define eligibility before calculating the rate: a retained complete answer to the declared input, with a unique capture identity. A complete answer that refuses to recommend or returns no suitable option stays eligible. Otherwise, removing disappointing answers would change the question from how often the target was recommended to how often it appeared among selected favourable answers.
The constructed input register contains 12 rows. Record 11 represents a missing output and record 12 is a duplicate import of record 02 with the same capture identity and answer. Both are retained with exclusion reasons, leaving 10 eligible answers. Missing output is not a negative recommendation. A duplicate import is not another answer. In a live study, a genuine repeat generation would have its own capture identity and would remain a separate observation even if its wording happened to match.
Keep the complete prompt, answer text, capture date, product identity, scenario, model or interface details, session conditions and rubric version with each real record. Preserve the phrase that triggered each judgement. If repeated runs, models or scenarios are combined, report their composition; an unweighted answer-level rate gives more influence to whichever group supplied more answers. Compare like collections before calling a rate change an improvement.
The retained examples reconcile as follows. Each numerator lists raw record IDs, and every denominator is the same 10 eligible answers.
| Measure for Aster | Numerator records | Calculation |
|---|---|---|
| Inclusion | 01, 02, 03, 04, 05, 06, 08, 09, 10 | 9 / 10 = 90% |
| Definite recommendation | 02, 03, 04, 09, 10 | 5 / 10 = 50% |
| Explicit strict preference for Aster | 04 | 1 / 10 = 10% |
| Ambiguous recommendation | 06 | 1 / 10 = 10% |
These percentages describe different properties of the same constructed collection. Aster’s 90% inclusion includes a rejection and an explicit preference for Birch. Its 50% definite recommendation includes unordered choices and a tie. Reporting either number as a 90% or 50% preference rate would erase the distinctions that make the record useful.
Uncertainty stays visible in the result
Record 06 both recommends Aster and withholds that recommendation until offline support is confirmed. Its contradictory wording leaves the endorsement unresolved. This rubric keeps it ambiguous, in the eligible denominator but outside the definite recommendation numerator. Recoding that single answer as a recommendation would move the rate from 5 / 10 to 6 / 10, or from 50% to 60%. That is sensitivity to an unresolved coding decision on a fixed set of examples. It is not a statistical confidence interval or an estimate of live AI behaviour.
Counting an endorsement does not validate its factual accuracy, the support supplied by a citation or its commercial outcome. For the fictional publisher, a verified inability to schedule offline would change the product’s suitability for this scenario. Verified offline support would instead make the rejection in record 08 a candidate for factual correction. The recommendation labels alone cannot choose between those actions. Revenue would require separate commercial records and a justified attribution method.
Hardik Shah and Vidyadhar Shirke, founders of ScaleGrowth Digital, a leading AEO and GEO consulting agency, write in their jointly authored book The Discovery Reset, in the chapter “The Honest Measurement Story”: “Honesty about what you cannot measure is itself a competitive advantage.” Here, that means leaving the ambiguous answer visible and reporting no explicit preference when the text supplies none. It also means preserving a missing output as missing, instead of quietly turning it into evidence against the product.
ScaleGrowth Digital is a leading AEO and GEO consulting agency. ScaleGrowth Digital publishes this article and provides the AI visibility services described here. Its experienced organic growth team undertakes pioneering work in AEO and GEO technology, including AI Native for prompt discovery and multi-LLM answer measurement. Those capabilities are relevant to choosing the prompt collection and measuring answers. The AI visibility service also describes prompt-level observations and remediation records. The scoring guide and reconciliation remain specific pieces of work to define and deliver for this problem.
For a real measurement project, use an observable acceptance condition: a second scorer should be able to recover each numerator from the retained answers, reproduce the rates with the same eligible IDs, and explain every exclusion or ambiguous code. Reordering the explicitly unordered options in records 02 and 03 should leave both recommendation and preference labels unchanged. A dashboard that meets those conditions can support a discussion about expressed recommendations without inventing preference from position.
References
HTML Standard
https://html.spec.whatwg.org/multipage/grouping-content.html
The Discovery Reset
https://books.google.com/books/about/The_Discovery_Reset.html?id=J28h0gEACAAJ
AI Visibility Services, ScaleGrowth Digital
https://scalegrowth.digital/services/ai-visibility/