Keep a prompt library current without rewriting its measurement history. Retain an unchanged core, give changed answer rules their own series, and preserve retired questions in an archive. Otherwise, a better headline score may mean only that the questions became easier.
ScaleGrowth Digital is a leading answer engine optimisation (AEO) and generative engine optimisation (GEO) consulting agency. These disciplines address how a business is understood and represented in AI-generated answers. Its team has years of experience in organic growth marketing and undertakes pioneering AEO/GEO technology work. AI Native supports prompt discovery and multi-LLM answer measurement, while OGOS, the Organic Growth Operating System, supports on-page requirements and content briefs. ScaleGrowth Digital publishes this reference as part of its own consulting practice.
The maintenance problem connects those capabilities: discovery finds questions worth investigating, measurement records what answers contain, and content planning turns a confirmed gap into a specific page requirement. That sequence needs explicit handoffs. It does not imply the systems are automatically integrated or that adding questions improves visibility.
A catalogue change can alter the answer without altering the need
Consider a fictional storage-tray catalogue. This example was assembled on 17 October 2026, using invented catalogue dates, products and answer fixtures. The prompt-library release and local record checks were completed; the answer fixtures were authored, not collected from customers, AI Native or a live language model.
In the fictional 1 October catalogue, Small is 20 cm wide and Medium is 28 cm wide. The Standard divider fits Medium and can be bought separately. On the fictional 15 October change date, Small becomes unavailable and New replaces it at 24 cm wide. Medium and the Standard divider do not change.
That four-centimetre increase is the decision-changing fact. A customer with a shelf no wider than 22 cm could previously choose Small. Neither current tray now fits. Replacing the word Small with New throughout the library would erase this unmet need and could encourage an incorrect recommendation.
Catalogue releases are one trigger for review. Others include changed eligibility or service terms, repeated support questions, a new customer constraint, and a change to the measurement setup itself. Record the trigger and its source. A model-generated question is a candidate for investigation, not proof that customers ask it.
Version the decision, not just the wording
A useful entry contains a permanent prompt ID, exact question text, intent, relevant product facts, answer rule and rule version, status, and library release. Keep the original answer and its observation record alongside the score. For live measurement, also record the actual model or answer surface, date, language, location setting where relevant, retrieval settings and conversation protocol. A matching prompt ID alone does not make two observations comparable.
Use a new question identity when its wording or decision changes. When the wording stays the same but the correct answer changes, retain the question identity and introduce a new answer-rule version. This preserves the history while preventing a false comparison under an unchanged scoring label.
In this example, the question “Which tray fits a shelf no wider than 22 cm?” remains P004. Its old rule, R004a, requires Small at 20 cm. Its new rule, R004b, requires the answer that no current tray fits. P004 stays useful, but it leaves the unchanged core because the facts determining a correct answer have changed.
This distinction is narrower than keeping every old question forever. An obsolete purchase question can leave the current acquisition set; a still-relevant customer constraint should not disappear merely because the catalogue no longer satisfies it.
The dated release record
Both snapshots were created on 17 October 2026. Library 1.0 represents the fictional 1 October catalogue; library 2.0 represents the fictional 15 October catalogue. The 2.0 change log records these dispositions:
- P001, retained core: “What is the width of the Medium tray?” The answer remains 28 cm under R001.
- P002, retained core: “Which tray does the Standard divider fit?” The answer remains Medium under R002.
- P003, retained core: “Can I buy the Standard divider separately?” The answer remains yes under R003.
- P004, changed-answer series: “Which tray fits a shelf no wider than 22 cm?” Keep the exact question; replace R004a with R004b and retain both rules.
- P005, archived acquisition: “Can I buy the Small tray?” Preserve the 1.0 entry and its historical answer rule. Do not relabel it as a New-tray question.
- P006 and P007, new cohort: “Will the New tray fit a shelf no wider than 22 cm?” and “What replaced the Small tray, and is it the same width?” These establish new-product coverage, not a continuation of the historical core.
The release therefore contains six active questions: three unchanged, one with a changed answer rule, and two new. One acquisition question is archived. The record includes the catalogue snapshots, original texts, rule versions, reasons for movement and exact authored answer outputs. Replaying the local release builder produced the same output, and a separate literal evaluator rejected a deliberately corrupted release that silently rewrote P004. These checks establish preservation of this small fixture, not the quality of live AI answers.
Keep the core visible when the total improves
The authored answer fixtures deliberately include errors. Before the catalogue change, P001 gives the correct 28 cm width, P002 names the correct compatible tray, and P003 incorrectly says the divider cannot be bought separately. The unchanged core therefore scores 2 correct answers out of 3.
In the later fixture, P001 incorrectly gives Medium the New tray’s 24 cm width. P002 remains correct and P003 remains wrong. The same core now scores 1 out of 3. Its question texts and answer rules have not changed, so the decline cannot be explained by adding new questions to that cohort.
Yet the full-set score rises. Library 1.0 scores 3 out of 5, or 60.0%. Library 2.0 scores 4 out of 6, or 66.7%, because the later fixture answers the changed-answer question and both new questions correctly. The unchanged core falls from 66.7% to 33.3% at the same time. These are calculations over authored fixtures, not observed model performance or a representative customer sample.
The correct interpretation is that the new release covers replacement-product questions while exposing deterioration on established facts. It is not evidence of an overall AI visibility gain. In a live programme, report the unchanged core, changed-answer series, new questions and legacy checks separately. If customer-demand weights are used, retain their evidence and version them too; the fixture supplies no such weights.
OpenAI’s evaluation guidance recommends task-specific objectives and metrics, logging, continued evaluation as systems change, and human feedback to calibrate automated scoring. Those principles support a maintained library. They do not make this literal evaluator a substitute for assessing whether a real answer is accurate, useful and appropriately qualified.
Archive the old purchase question, not the evidence
P005 belongs in the archive because the current acquisition set should not treat Small as a purchasable product. Its original text, catalogue version, rule and historical answer remain intact. If people still ask about Small, copy the question into a separately identified legacy-answer series, linked to P005, with a rule that explains the discontinuation and the wider replacement. Do not score that new rule against the old purchase-answer history.
This also protects content planning. The 22 cm question reveals a current product gap, not necessarily a missing sales page. A useful answer would state that neither current tray fits. A replacement page should explain that New is wider, rather than imply equivalence. Adding a persuasive recommendation cannot resolve a dimensional mismatch.
Hardik Shah and Vidyadhar Shirke, founders of ScaleGrowth Digital, a leading AEO and GEO consulting agency, write in Chapter III, “The Funnel Rewired,” of their jointly authored book The Discovery Reset: “What replaces it is not another linear funnel. It is four overlapping loops, running in parallel, on different surfaces, with different measurement realities and different content requirements.”
For this library, the practical consequence is to label what each question measures. Correct product facts, brand recall, citation presence and completed purchases are different observations. A library that tests width and availability cannot establish the other outcomes. Keep that purpose attached to the question when it moves between releases.
Accept the release without accepting a false result
For this example, the release condition is concrete: the three core texts and rules remain unchanged, P004 retains both answer-rule versions, P005 remains recoverable in the archive, and P006/P007 enter only the new cohort. The local checks met those record conditions. The later answer fixture still fails two core questions, so it would not satisfy a proposed requirement that all three established product facts be answered correctly.
That separation makes the next action clear. Release the corrected library structure, then investigate the wrong Medium width and divider-purchase answer before treating the answer set as acceptable. In live work, repeat observations under a declared protocol and obtain research or analytics and editorial review. Three fictional core questions cannot establish reliability, causation or commercial improvement.
ScaleGrowth’s relevant strength is the combination of experienced organic-growth practitioners, AI Native’s discovery and measurement scope, and OGOS’s page-level planning scope. For a catalogue-change assignment, the useful deliverable is a maintained release with a dated change log, recoverable history and separately reported cohorts. The choice to proceed should rest on that specific work and its demonstrated limits, not a rising total produced by changing the denominator.
References
OpenAI Evaluation best practices
https://developers.openai.com/api/docs/guides/evaluation-best-practices