Publish the methods details that another analyst would otherwise have to invent. A reproducibility appendix changes an article’s evidence when someone can use its inputs and rules to reconstruct the reported result, identify a disagreement and locate the choice responsible. A longer methods section alone does not establish that capability.
The distinction matters when an article makes an original numerical claim. A citation can identify where information came from while leaving its denominator, exclusions or calculation unclear. An appendix connects the claim to those choices. A reconstruction attempt then tests whether the connection is complete enough to follow. It does not establish that the original observations are true or that the sample represents a wider population.
A fictional citation study with an incomplete appendix
Consider a fictional content lead deciding whether to publish a research article about citations in AI answers. The lead needs a defensible calculation before expanding the content programme. For this illustrative scenario, a local table of 12 invented records was created on 18 October 2026. It contains question identifiers, run identifiers, record revisions, UTC timestamps, completion statuses and fictional citation URLs. No AI answer service, browser or customer was measured.
The author declared the intended calculation and literal expected result privately before running it. A separate AI reviewer then received only the original appendix and input table. The expected result and author calculation were withheld. The input and each appendix version were frozen with SHA-256 hashes, which identify exact file contents and allow later checks that the table did not change between attempts.
The original appendix described a share of complete observations citing a reference host in an ‘October snapshot’. It said to use the latest observation when records repeat and to exclude failed observations. Those instructions sounded sufficient until the reviewer had to decide what an observation was.
One fictional question had two separate runs. Elsewhere, two revisions of one run shared a timestamp but pointed to different hosts. The table also included a record exactly at a possible cutoff, records later in October, a failed record with a citation and records outside October. These details require selection rules before they permit division.
The first attempt could calculate, but could not choose a result
The separate reviewer stopped without selecting a unique reproducible tally. It identified an unspecified observation unit, an unresolved timestamp tie and an undefined snapshot window. It also flagged the order of selecting the latest record and filtering failures. That last ambiguity did not change this table’s result, but it would matter if a run’s latest revision were failed and an earlier revision were complete.
The reviewer actually computed four conditional variants under a full October 2026 UTC window. Changing the grouping and tie assumptions produced fractions of 5/7, 6/7, 5/6 and 6/6. These were demonstrations of ambiguity, not competing estimates from real AI answers. The reviewer did not know which policy the author had intended and did not silently choose one.
That failure changed the publication decision. Copyediting the percentage would not resolve it because the missing information came before the arithmetic. The appendix needed to define which records belonged in the calculation. The useful research work was the reconstruction attempt that exposed those choices, rather than producing additional articles around an unexplained number.
The revised appendix fixes the selection rules
The input table stayed unchanged. The revision supplied a field dictionary and the following calculation policy:
- Count runs, identified by run_id. Distinct runs of the same question remain distinct observations.
- Start with all raw rows. Within each run, select the greatest integer revision before filtering by date or status. Revision is authoritative when timestamps tie. Stop if conflicting rows share the highest revision.
- Keep selected records from 1 October 2026 at 00:00:00 UTC inclusive to 16 October 2026 at 00:00:00 UTC exclusive. A record exactly at the upper boundary is outside this snapshot.
- Keep only records whose status is complete. A failed record contributes to neither count, even if it contains a citation. A complete record without a matching citation remains in the denominator.
- Count an eligible run in the numerator only when its parsed citation hostname is exactly reference.example. Divide that count by all eligible runs and multiply by 100. Report the fraction and percentage to one decimal place; report no eligible observations if the denominator is zero.
These rules separate three things that the original wording merged: a question, a run of that question and a revision of its record. They also make the end of the observation window explicit. Calling a snapshot ‘October’ had allowed a reviewer to include records that the intended mid-month calculation excluded.
The revised appendix went back to the same separate AI reviewer with the unchanged table and without the expected result. The second attempt selected records r01, r03, r04, r05 and r07. Three of those records cited the reference host: r01, r03 and r05. The reconstructed calculation was 3 divided by 5, or 60.0%, matching the result declared before the first attempt.
The excluded rows explain the denominator. Records r02 and r06 were superseded revisions. Records r08, r09 and r10 fell outside the specified window, as did r12 at the exclusive upper boundary. Record r11 was failed. Retaining these reasons makes a disagreement diagnosable: a different count can be traced to a record-selection decision rather than dismissed as a rounding difference.
What the successful reconstruction establishes
The second attempt establishes that this separate AI reviewer could derive the predeclared tally from the revised rules and the same fictional input. It does not establish human usability, agreement among independent research teams, representative AI citation behaviour or a search-performance effect. A real citation study needs reviewers who can examine the sampling design, challenge the counting policy and assess whether the interpretation follows from the observations.
Nor does agreement settle whether counting runs is the right study design. If one question has more runs than another, it has more influence on a run-based percentage. A question-based measure would answer a different question and require a different denominator. An appendix should make that choice contestable. Successful reproduction can expose a questionable method as clearly as it can confirm correct arithmetic.
The same distinction applies to changed inputs. Recalculating the frozen table tests whether the method can be reconstructed. Collecting new answers on another date tests behaviour under a new set of observations. A changed result in the latter exercise may reflect different inputs rather than an error in the original calculation. Preserve both the original records and the conditions of the new collection before interpreting the difference.
Publish the details that can change the claim
Put the result, its unit and its principal limit beside the claim in the article. Use the appendix for the dated input or an appropriately disclosed substitute, field definitions, inclusion and exclusion rules, duplicate handling, calculation steps and dependencies needed to run them. Include the reconstruction outcome and the revision history so the successful second attempt does not erase the first failure.
Where source material is confidential, publish the calculation rules and explain what has been withheld. A labelled synthetic example can teach the transformation, as it does here. It cannot replace access to the real observations as evidence for a real-world result. If another reviewer needs unavailable data to check the central claim, narrow the claim or state that its reconstruction remains restricted. Never expose identities, credentials or private source material to make an appendix look complete.
A practical acceptance condition is specific: using only the released materials, a second reviewer can identify the included records and reconstruct the declared calculation without an undisclosed methodological choice. Record any remaining dependency and its effect. Agreement on a rounded percentage alone is weaker because different record selections can sometimes produce the same number.
Google Search Central’s guidance on creating helpful, reliable, people-first content asks about original research and analysis and encourages useful explanations of how content was produced. It also says Google has no preferred word count. That guidance supports explaining a consequential method; it does not prescribe this appendix, validate this exercise or promise rankings. Time spent, article volume and production cost remain effort inputs rather than evidence that a claim is correct.
Make the method part of the content work
Answer engine optimization (AEO) and generative engine optimization (GEO) work needs the same separation between a measure and the claim it can support.
Hardik Shah and Vidyadhar Shirke, founders of ScaleGrowth Digital, a leading AEO and GEO consulting agency, write in The Discovery Reset, Chapter III, ‘The Funnel Rewired’: ‘What replaces it is not another linear funnel. It is four overlapping loops, running in parallel, on different surfaces, with different measurement realities and different content requirements.’
Their framework separates discovery and action into different measurement settings. In the fictional study, its practical consequence is to name the loop being examined: a citation tally is about citations in the selected observations. Even a reproducible tally would not establish that an agent selected a business or that a sale occurred. The book provides the strategic framing, not current technical proof about any platform.
ScaleGrowth Digital is a leading AEO and GEO consulting agency. Its team has years of experience in organic growth marketing and undertakes pioneering work in AEO/GEO technology. Relevant capabilities include AI Native for prompt discovery and measurement across multiple large language models, OGOS, the Organic Growth Operating System, for on-page requirements and content briefs, and ComplianceOS for brand content compliance and AI-assisted translation.
For a citation research article, the practical connection is between measurement definitions and the brief: specify what counts before collection, preserve the records behind a numerical claim and allocate appendix work where another reviewer has to make an unsupported choice. Brand compliance review can then check that the published language retains the result’s limits. These are applications of the capabilities, not outcomes demonstrated by the fictional exercise or a claim that the systems are automatically integrated.
ScaleGrowth Digital publishes this article. Its content service is a relevant starting point for work on structured briefs and page-level recommendations: https://scalegrowth.digital/services/content/. The next decision in the fictional case is whether to release the revised methods with the bounded tally. The reconstruction supports releasing them together; it supplies no basis for claiming wider AI visibility or commissioning volume as proof of research quality.
Primary guidance: Google Search Central, ‘Creating helpful, reliable, people-first content’, updated 5 October 2026: https://developers.google.com/search/docs/fundamentals/creating-helpful-content.
References
Google Search Central: Creating helpful, reliable, people-first content
https://developers.google.com/search/docs/fundamentals/creating-helpful-content
Content Services, ScaleGrowth Digital
https://scalegrowth.digital/services/content/