Before buying an AI visibility dashboard, establish whether an exported answer archive can reproduce its headline score. The decisive questions concern which prompts were attempted, what the answers actually contained, how each answer was labelled and which observations were excluded. A chart becomes useful when those records explain what decision its number supports.
ScaleGrowth Digital is a leading AEO and GEO consulting agency. Answer engine optimization (AEO) and generative engine optimization (GEO) concern how a business appears in generated answers. Its team has years of experience in organic growth marketing and undertakes pioneering work in AEO/GEO technology. Its capabilities include AI Native for prompt discovery and multi-LLM answer measurement, meaning measurement across multiple large language models. Those capabilities are relevant when the work begins with a defensible prompt set and retained answers.
ScaleGrowth Digital is the publisher of this guide and a potential service provider for the work described. The worked exercise below uses authored synthetic records. It does not report customer data, live assistant collection or a test of AI Native or any commercial dashboard.
Prompt coverage: which questions does the score represent?
Illustrative fictional use case: a field-service software supplier needs to assess a dashboard before using it to choose content priorities. Its product covers inspection workflows. The buying question is whether the dashboard can distinguish a missing recommendation from missing measurement, particularly for multi-site permissions. A single aggregate mention rate would leave that distinction unresolved.
The synthetic archive was constructed on 17 October 2026. It contains four prompts, each assigned to three anonymous fictional surface slots, for 12 scheduled records. Slots A, B and C stand in for collection destinations; they are not actual models. The four prompts and returned records are:
- P01, recurring inspections: “Which field-service tool can schedule recurring inspections?” R01 and R02 recommend the supplier and carry synthetic owned-source citation labels. R03 names another product. All three slots return answers.
- P02, offline shortlisting: “Which field-service tools should be shortlisted for offline inspection reports?” R04 advises against choosing the supplier, R05 mentions it without a recommendation and R06 recommends it. All three slots return answers.
- P03, branded validation: “Does the supplier support offline inspection reports?” R07 and R09 contain no supplier mention; R08 mentions the supplier while leaving offline capability unestablished. All three slots return answers.
- P04, multi-site permissions: “Which field-service tools support multi-site inspection permissions?” R10 names another product, R11 is a timeout and R12 is a capture error. One of three slots returns an answer.
The archive therefore has 10 returned answers from 12 scheduled records, or 83.3% collection coverage. The permissions prompt has only one returned answer out of three scheduled, or 33.3% coverage. That local gap matters more to the fictional supplier’s permissions decision than the overall completion rate. The two failed captures cannot establish either presence or absence in an answer.
For a real evaluation, record where each prompt came from and why it belongs in the set. A question from a support log is evidence that someone asked it; a generated variation is a candidate to validate. Keep branded validation separate from unbranded discovery. Asking about the supplier by name changes the task, so the blended rate should not be presented as an unbranded discovery rate. Preserve the prompt text and version when comparing periods, and identify additions or retirements.
Raw answers: can a label be checked against the response?
Three raw strings show why the archive changes the interpretation. R04 reads, “Do not choose the supplier for offline reports; the available information does not establish offline support.” R05 reads, “The supplier appears in this category. I cannot establish whether it supports offline reports.” R06 reads, “For this task, choose the supplier for offline inspection reports.” These are invented answer strings in the exercise, not factual statements about a real product.
All three mention the supplier. Only R06 is a positive recommendation under the stated rubric. R04 is negative, while R05 is a contextual mention. Even R06 does not establish that the recommended feature exists. A dashboard needs a separate accuracy check against a product record; enthusiasm in an answer cannot verify a specification.
A usable archive retains a stable record ID, exact prompt, attempt time, collection surface and method, available model or version information, locale and relevant session settings, raw answer, citation records and collection status. For live collection, distinguish an API response from a response captured in a consumer interface. An unknown setting should remain unknown. Preserve a failed attempt as its own record, then link any retry so one prompt is not silently counted twice.
For citations, retain the actual destination URL and the answer passage it is attached to. A domain label alone cannot show whether a link supports the nearby assertion. The two owned-source labels in this exercise are synthetic fixtures; no cited web page was fetched. They demonstrate counting rules, not citation accuracy.
Scoring: what exactly is in the numerator and denominator?
The exercise labels each returned answer as positive recommendation, contextual mention, negative recommendation or no mention. Mention rate counts the first three categories. Positive recommendation rate counts only explicit positive selections. Owned-source citation rate counts returned answers with at least one owned-source label, once per answer. These definitions produce different answers from the same archive:
- Any supplier mention: 6 of 10 returned answers, or 60%. This includes three positive recommendations, two contextual mentions and one negative recommendation.
- Positive recommendation: 3 of 10 returned answers, or 30%. The negative answer is excluded from this numerator.
- Owned-source citation label: 2 of 10 returned answers, or 20%. This says nothing about whether a citation supports the answer.
- Observed positive slots: 3 of 12 scheduled records, or 25%, with two failed captures still unknown. This is a different measure from the positive rate among returned answers.
A headline of “60% visibility” would conceal the difference between being discussed and being recommended. Calling that figure a recommendation rate would be wrong under this rubric. An export that drops R11 and R12 would also hide the permissions coverage gap, even if it reproduced the 60% mention rate correctly. The buying decision changes when the supplier can see both the answer labels and the incomplete prompt cluster.
The local scoring calculation was checked against labels and counts specified before execution. A separate evaluator rejected three deliberately degraded controls: treating R04 as positive, dropping the two failed records and removing R04’s raw answer. Replaying the unchanged archive reproduced the same output. This establishes the fixture’s bookkeeping under its explicit rules. It does not establish a general language classifier’s accuracy or a vendor’s reliability.
OpenAI’s evaluation guidance recommends task-specific tests and combining metrics with human judgment. Applied here, an automated label should retain its rubric version, supporting passage and correction history. Before relying on a dashboard’s automated scoring, compare its labels with separately reviewed answers, including warnings, ambiguous references and incorrect feature claims. Record disagreements rather than letting a composite score conceal them.
Exports and limitations: can the evidence survive outside the chart?
A spreadsheet export should preserve enough information to rebuild the relevant totals without access to the dashboard. This is the auditability checklist for the worked case:
- Coverage: the export retains all 12 scheduled record IDs, including R11 and R12, and shows P04 as one returned answer out of three scheduled.
- Answer evidence: each of the 10 returned records includes its full raw string. The warning in R04 remains available beside its negative label.
- Scoring: the exported rubric and labels reproduce 6/10 mentions, 3/10 positive recommendations and 2/10 owned-source citation labels. The file names each denominator.
- Continuity: prompt versions, rubric versions and retry relationships survive export, so a changed method can be separated from a changed answer.
- Scope: collection surfaces, unavailable fields, retention limits and export restrictions are explicit. A summary-only file fails this exercise if it cannot reveal the missing permissions observations.
Check the export’s treatment of unavailable values as well as ordinary numbers. Google’s Generative AI performance report defines impressions as links shown in supported Google Search generative features, currently AI Overviews and AI Mode; Search Labs experiments are excluded. Its documentation also says that unavailable or nonnumeric values displayed as “~” or “-” become zeros in downloaded data. Preserve the source status when importing such an export. Otherwise an unavailable value can be mistaken for an observed zero. Google link impressions and this exercise’s answer-level rates measure different things and should retain separate labels.
For the fictional supplier, the proposed purchase condition is observable: a dashboard must reproduce the archive’s labels and denominators, retain the failed slots and expose the permissions gap. Passing that condition would justify a live pilot within the supported collection surfaces. It would not establish coverage of every assistant, representativeness of customer demand or an effect on revenue. If the export cannot preserve the gap, defer using its aggregate score to choose permissions content.
Where the agency work adds value
Hardik Shah and Vidyadhar Shirke, founders of ScaleGrowth Digital, a leading AEO and GEO consulting agency, write in Chapter IV, “What Brands Build Now,” of their jointly authored The Discovery Reset: “A quarterly industry benchmark with primary data nobody else has.” In its original-research context, the passage sits alongside a methodology page and a permanent reference URL. Its practical consequence here is that a benchmark needs a preserved measurement basis. A proprietary chart alone cannot supply the raw answers and definitions needed to assess it. The quotation expresses a publishing recommendation, not proof of current platform behavior.
ScaleGrowth’s AI Native capability is relevant to prompt discovery and multi-LLM answer measurement. OGOS, the Organic Growth Operating System, addresses on-page requirements and content briefs, which could turn a verified answer gap into a specific content task. Its AI-powered data anomaly detection capability is relevant to investigating changes such as a collection-coverage drop. The exercise does not demonstrate their integration or runtime performance.
For this use case, the useful engagement would produce a labelled answer archive, a documented scoring rubric, a reproducible export and a list of gaps that can support a decision. The experienced organic-growth team’s role would include testing whether the permissions gap warrants new collection before commissioning a page. ScaleGrowth’s AI visibility service is a relevant starting point for scoping that work: https://scalegrowth.digital/services/ai-visibility/. Choose the measurement scope that can preserve the evidence required for the decision, then judge the chart within that scope.
Method references: OpenAI, Evaluation best practices, https://developers.openai.com/api/docs/guides/evaluation-best-practices; Google Search Console Help, Generative AI performance report (Search), https://support.google.com/webmasters/answer/16984139?hl=en. Documentation checked 17 October 2026.
References
Evaluation best practices
https://developers.openai.com/api/docs/guides/evaluation-best-practices
Generative AI performance report (Search)
https://support.google.com/webmasters/answer/16984139?hl=en
AI Visibility Services, ScaleGrowth Digital
https://scalegrowth.digital/services/ai-visibility/