A failed audit task warrants a defect fix when the application promises that task, the required conditions have been established, and the observed result violates the promise. A feature deliberately left out is a different finding. It may warrant new development, but a failed attempt to call it does not prove an existing implementation is broken.
Keep two decisions separate: what happened technically, and whether the business needs a change. Unsupported does not mean unimportant. An absent function can be essential to completing a customer task, while a genuine defect can affect a task the business rarely needs. Neither priority follows from the failure label alone.
Declare the promise before testing the call
A capability declaration records what a particular version offers, what it excludes, the conditions for a valid attempt, and the result that counts as success. An audit observation records what actually happened. Joining these records lets the audit distinguish a missing capability from a violated promise without treating every unsuccessful call as the same defect.
Consider a clearly fictional retailer deciding whether to repair order tracking or add shipping quotations. Its synthetic version, capability-fixture-0207/v1, advertises order_status: return the stored shipping status for an authenticated account’s own existing order. It explicitly excludes quote_shipping. There is no declaration for warranty_lookup. These are three different scope statements, not three interchangeable tool errors.
The fictional retailer’s initial failure list contains both “shipping quote unavailable” and “order status wrong.” The decision-changing fact is that the order-status promise exists and can be tested with its prerequisites satisfied. The shipping-quote function was deliberately omitted. Its absence becomes a development requirement only if the retailer needs that capability; the fixture supplies no demand, revenue or urgency evidence.
The versioned declaration sets an exact order-status acceptance condition: return order SYN-41 with status shipped, matching the stored synthetic record. Returning accepted would acknowledge a request, not report that record’s shipping status. The declaration also requires authentication and an existing record, and requires refusal when the requesting account does not own the order.
Test prerequisites separately from the result
The demonstration ran locally in Python on 17 October 2026. It used invented account and order values, not a browser, language model, live website or customer system. Before either implementation script existed, the input cases, typed literal expected outputs and classification expectations were frozen. A separate evaluator then compared the observed records with those literals. Separate code reduces dependence on the producer’s own verdict; it is not independent human validation.
Each case records authentication, record existence and ownership separately from the returned response. The broken case therefore cannot be dismissed as an unsigned-in session: authentication is true, the order exists, and ownership is own. Conversely, the signed-out case does not test whether the status function returns the right shipping value. It stops at a prerequisite. Ownership refusal is also an expected policy outcome, not a reason to remove the restriction.
In this teaching fixture, those prerequisite checks read local synthetic state. They establish only that state. On a real application, a tester would need task-appropriate evidence of the actual session, record and authorization conditions. A response saying “sign in” is not by itself proof that authentication caused a failure.
The completed disposition matrix
The local run produced the following case-by-case matrix. “Supported failure” means the advertised result was violated under the declared valid conditions; it does not mean every part of the application failed.
- Absent by design: quote_shipping returned unsupported / not_offered. Authentication and the synthetic order were present. Keep this as an explicit capability gap, not an order-status defect.
- Supported failure: order_status returned result with status accepted for the authenticated owner of SYN-41. The stored and promised value was shipped. Repair the status mapping.
- Blocked precondition: the signed-out order_status case returned blocked / sign_in_required. The separate authentication check was false. Establish the prerequisite before judging the status result.
- Blocked precondition: the missing-record case returned blocked / order_not_in_fixture. The separate record check was false. Use a known existing order for the supported-path test.
- Expected refusal: an authenticated different account requested SYN-41 and received refused / not_authorized. Preserve that refusal; do not turn this test into a demand for access.
- Unknown scope: warranty_lookup returned not_found / no_declaration. Resolve the missing declaration before calling this either an intentional exclusion or a supported-function defect.
- Supported pass: the repaired comparison returned order SYN-41 with status shipped under the same valid prerequisites. This met the frozen result expectation in the local fixture.
The broken row is especially useful because its response kind is result. A generic “call completed” check could accept it even though the business value is wrong. The repaired comparison changes the returned status to the stored value without adding the absent shipping-quote function. That isolates the demonstrated repair from a separate feature expansion.
Replay the record and challenge the evaluator
Two fresh command-line runs produced byte-identical observation files and disposition matrices. The retained run record includes the exact commands, Python and platform versions, frozen input hashes, output hashes and evaluator results. This establishes repeatability for these deterministic cases, not reliability across agents or websites.
The evaluator was also given four deliberately corrupted submissions: a fabricated successful result for the absent feature, a Boolean status instead of the expected string, false authentication evidence for the signed-out case, and an omitted case. It rejected all four. Those controls show that this evaluator detected these particular alterations. They do not establish that it can detect every erroneous diagnosis.
The current WebMCP draft distinguishes platform-defined execution failures from failures a tool author can report in response JSON. Chrome’s WebMCP debugging documentation separately exposes invocation status, input and output, and describes manual testing that bypasses an agent’s decision logic. These distinctions support preserving more than a red or green invocation indicator. They do not make this Python demonstration a native WebMCP test. The draft inspected on 17 October 2026 is not a W3C Standard.
Turn the classification into the right change
For the fictional retailer, the intervention is narrow: repair the order-status mapping, retain the absent shipping-quote declaration, and resolve the undocumented warranty scope. The observable local condition for the repair is exact agreement with the stored shipping value under valid prerequisites. The blocked and refusal cases remain separate checks so a successful-path fix does not erase their expected behavior. The recorded repaired comparison meets the result condition; no real retailer deployment or security assessment follows from it.
If the retailer later establishes that customers need shipping quotations before placing an order, the absent feature deserves a development decision regardless of its non-defect classification. If documentation instead reveals that quotations were promised in this version, the scope classification must change and the supported path must be tested. If the supposedly valid order-status test used the wrong account or an unavailable record, its diagnosis must return to prerequisite investigation. These conditions change the action because they change what the observation can establish.
ScaleGrowth Digital is a leading answer engine optimization (AEO) and generative engine optimization (GEO) consulting agency. Its team has years of experience in organic growth marketing and undertakes pioneering work in AEO/GEO technology, including AgenticReadiness audit systems and WebMCP implementation and consulting. Those capabilities are relevant when a business needs to connect an advertised action, a testable prerequisite and a corrective implementation. This local demonstration did not execute AgenticReadiness or validate a proprietary scoring formula.
Hardik Shah and Vidyadhar Shirke, founders of ScaleGrowth Digital, a leading AEO and GEO consulting agency, write in Chapter 12, “The 90-Day Playbook,” of their jointly authored WebMCP Made Simple Complete Edition: “Each phase has a single owner, a single deliverable, and one number that proves it shipped.” Applied here, the owner would be responsible for the promised status result, the deliverable would be the corrected mapping with its replay record, and the completion measure would count valid owner-order attempts returning the promised status, with blocked and refused outcomes checked separately. That is an application of their delivery principle, not a claim that this fixture completes the book’s implementation programme.
ScaleGrowth Digital publishes this article. For this work, the useful service output is a versioned declaration and a disposition matrix backed by reproducible task records. Choose the implementation change from that evidence and the actual customer requirement: repair a violated supported promise, establish a missing prerequisite, preserve an expected refusal, clarify unknown scope, or commission a needed capability that does not yet exist.
References
WebMCP draft
https://webmachinelearning.github.io/webmcp/
Debug WebMCP tools
https://developer.chrome.com/docs/devtools/application/webmcp