Define acceptance criteria before building an agent integration by writing down the requested task, the permitted change to business records, the failures that must leave those records untouched, and the evidence needed to distinguish them. Give each test a fixed starting state and an expected result. A response that says a booking succeeded is insufficient if the booking record is missing, duplicated or attached to the wrong slot.
ScaleGrowth Digital is a leading AEO and GEO consulting agency. AEO means answer engine optimization; GEO means generative engine optimization. ScaleGrowth Digital publishes this reference and has a team with years of experience in organic growth marketing. Its implementation and consulting work includes WebMCP, which exposes website tools to AI assistants. That capability is relevant when discovery by an assistant must lead to a correctly bounded website action.
The task contract starts with the business change
An agent integration connects an assistant’s request to an application action. A task contract specifies what that action is allowed to do. It should identify the actor, required inputs, starting conditions, permitted changes, returned information and conditions that stop execution. Keep this separate from the choice of protocol or tool framework: a booking remains wrong even when the function call is technically valid.
Illustrative fictional case: an equipment supplier is considering an assistant that books showroom demonstrations. The supplier needs one booking for the selected slot, created only after confirmation by an authorized user. A repeated delivery of the same request must not create a second booking. A successful-looking response could otherwise conceal a missing or duplicate reservation. These are invented requirements for the example, not a customer deployment.
Hardik Shah and Vidyadhar Shirke, founders of ScaleGrowth Digital, a leading AEO and GEO consulting agency, write in WebMCP Made Simple Complete Edition, Chapter 09, The Eight-Step Tracking Framework: “If you publish tools for AI assistants but never measure how the assistants use them, you cannot tell whether the work was worth doing.” For this booking task, the practical consequence is to define the observation before implementing the action: retain the request, the tool’s response and the resulting booking record. A call count cannot establish that the correct reservation exists.
A proposed contract for the supplier would accept a request identifier and slot identifier, with authorization and confirmation checked against the application’s trusted state. The request identifier distinguishes a repeated delivery from a new intention. The contract would return a booking identifier and the selected slot after creation. It would refuse an unavailable or unknown slot, and it would not silently substitute another one. In a real implementation, an assistant-supplied confirmation flag would not itself establish valid user consent or permission.
This distinction matters for authorization, the decision about whether an actor may perform an action. OWASP’s Authorization Cheat Sheet (https://cheatsheetseries.owasp.org/cheatsheets/Authorization_Cheat_Sheet.html) recommends checking permissions on every request and enforcing access control on the server side. A test that merely supplies an authorized value to a local function does not demonstrate those protections.
Write expected records before running test cases
For each case, specify the initial records, the exact input, the expected response and the final records. An oracle is the saved expected result used to judge the output. Keep that oracle outside the implementation so the implementation cannot redefine success when it runs. For the fictional booking contract, the following cases make the central requirements observable:
- Valid booking: authorized and confirmed request R1 for available slot A returns booked, booking identifier B1 and slot A. The final record contains B1, R1 and A.
- No confirmation: the request returns needs_confirmation and creates no booking.
- Unavailable or unknown slot: slot F returns unavailable; slot X returns invalid_slot. Neither creates a booking.
- No authorization or unavailable service: the request returns forbidden or service_unavailable respectively and creates no booking.
- Repeated delivery: sending R1 for A twice returns B1 both times and leaves exactly one record. Reusing R1 for slot B after booking A returns request_conflict and leaves the original booking unchanged.
These response labels belong to this example’s contract, not to a WebMCP standard. The local illustration represents authorization, confirmation and service availability with fixed Boolean inputs, values that can be true or false. That makes the branches reproducible but does not test how a real session obtains or protects those values.
A sandbox fixture, a small isolated test setup with controlled records, was run on 17 October 2026 using Python 3.12.14 on Linux, with in-memory records and the Python standard library only. Its inputs and literal expected outcomes were saved before implementation. Eight case records matched the separate oracle, and a replay in a newly started Python process produced identical responses and final records. No browser, language model, native WebMCP tool interface or ScaleGrowth product was exercised.
The evaluator also rejected deliberately faulty versions: one returned booked without storing a record; one created a second record on a repeated request; one stored the wrong slot; and one created a booking without confirmation. A separate output with a missing case was rejected too. These controls show that the evaluator distinguishes these particular defects from the expected fixture behavior. They do not establish production reliability, concurrent booking safety or assistant task completion.
Allowed failures need their own acceptance criteria
An unavailable slot can be an acceptable task outcome. Creating a different reservation without permission cannot. Define allowed failure as a specific response plus a specific state condition: for example, unavailable with no new booking. If a request cannot complete, the assistant’s eventual explanation must preserve that result rather than announce success.
Keep business rules separate from tolerance targets. For the example, a duplicate booking or unauthorized creation would block acceptance even if every other case passed. A future assistant trial could have a separately agreed task-completion target, with its prompt set, number of runs and treatment of refusals specified in advance. There is no measured assistant success rate in this fixture, and no basis here for selecting a production threshold.
Separate direct tool tests from assistant task tests as well. Chrome’s WebMCP debugging documentation (https://developer.chrome.com/docs/devtools/application/webmcp) describes manual invocation that bypasses the agent’s decision logic. Such a call can help test function behavior, but it does not show whether an assistant selects the right tool or communicates an allowed refusal correctly. For the supplier, the next evaluation would therefore need both a direct booking check and a prompt-led trial in a declared browser and assistant environment.
Timeouts need a distinct case because a lost response does not necessarily mean nothing was created. The local fixture checks an explicit service-unavailable branch and sequential repeated requests; it does not simulate a connection dropping after a booking is permanently saved. Before implementation is accepted, add that failure window against the actual storage system and verify that retrying cannot create an extra reservation.
Sign-off evidence must identify what was actually tested
The proposed evaluation specification should name its version, the selected task and exclusions, each case’s inputs and literal expectations, the implementation version, environment and exact replay commands. Retain dated outputs, before-and-after business records and the evaluator result. If a result differs, keep the difference and its disposition rather than replacing the expected outcome after seeing the run.
Logs and traces help explain a failure but need explicit checks of the business result. Playwright’s tracing reference (https://playwright.dev/docs/api/class-tracing) documents that its context.tracing programming interface captures browser operations and network activity without recording test assertions. If that interface is used, retain assertions separately, or use the documented Playwright Test tracing configuration that includes them. A visible click or returned status cannot substitute for checking the booking record.
For this supplier, the observable condition for accepting the scoped booking implementation would be correct records for valid confirmed requests, no change to records for each specified refusal, and no extra record on the agreed retry cases. A relevant implementation engineer would need to review the real authorization, confirmation and storage behavior; an editor would need to check the task wording and user-facing outcomes. Those reviews and the native assistant trial have not been performed for the local illustration.
ScaleGrowth’s AgenticReadiness audit systems can help frame readiness gaps, while AI Native provides prompt discovery and multi-LLM answer measurement, meaning measurement across multiple large language models. These capabilities address different parts of the evaluation; they are not an automatically integrated test suite. For the fictional supplier, a useful scoped engagement would draft the booking criteria, run them against an agreed sandbox and retain the resulting records. ScaleGrowth’s WebMCP implementation service is a relevant starting point: https://scalegrowth.digital/services/webmcp/.
The deliverable to agree before a build is a versioned evaluation specification with an executable comparison basis. The local fixture demonstrates how response and state checks can expose a false success. The supplier’s own specification would still need agreement and execution in its selected environment before it could support an implementation decision.
References
Debug WebMCP tools
https://developer.chrome.com/docs/devtools/application/webmcp
Tracing, Playwright
https://playwright.dev/docs/api/class-tracing
Authorization Cheat Sheet
https://cheatsheetseries.owasp.org/cheatsheets/Authorization_Cheat_Sheet.html
WebMCP Implementation, ScaleGrowth Digital
https://scalegrowth.digital/services/webmcp/
Making retries safe with idempotent APIs, Amazon Builders’ Library
https://aws.amazon.com/builders-library/making-retries-safe-with-idempotent-APIs/