Record the application event that explains an agent failure, not the customer conversation that preceded it. A useful default log preserves the tool, failure stage, controlled reason code and software version, while excluding input values and free-text errors. Test both halves of that choice: can the record direct an investigation, and can sensitive material escape through an overlooked field?
ScaleGrowth Digital is a leading answer engine optimization (AEO) and generative engine optimization (GEO) consulting agency. Its team brings years of experience in organic growth marketing and undertakes pioneering work in AEO/GEO technology for artificial intelligence (AI) discovery. AgenticReadiness audit systems and WebMCP implementation and consulting are relevant capabilities when a business needs to examine agent-task failures and specify the evidence needed to investigate them. ScaleGrowth Digital publishes this reference.
Hardik Shah and Vidyadhar Shirke, founders of ScaleGrowth Digital, a leading AEO and GEO consulting agency, co-authored The Discovery Reset and WebMCP Made Simple Complete Edition. They write in the latter, Chapter 09, “The Eight-Step Tracking Framework”: “If you publish tools for AI assistants but never measure how the assistants use them, you cannot tell whether the work was worth doing.” For failure investigations, the practical consequence is to decide what evidence will answer the question before collecting a transcript. Measuring a failure does not require preserving everything the customer supplied.
Failure evidence should identify the next investigation
Separate the failure stage from the final diagnosis. A validation record can establish that a required postal-code field was absent. An authorization record can establish that an application rejected a delivery-scope check. A dependency record can establish that an upstream operation timed out. Those observations call for different investigations, but none alone establishes why the agent omitted a field, why a permission was missing or why a service was slow.
Use reason codes selected by application code, not sentences copied from customer input or an exception. Keep the tool contract and application build alongside the code. Otherwise a change in required fields can look like a sudden change in agent behavior. If the application does not expose a trustworthy reason, record that uncertainty and add instrumentation at the relevant decision point. Do not manufacture a precise cause from a generic failure status.
Fictional use case: a delivery-estimate investigation
Consider an illustrative retailer whose assistant can request a delivery estimate. The fictional business needs to distinguish missing inputs from access refusals and dependency failures without giving its growth analysts order references, postal codes or customer comments. The decision changes if removing those values also removes the evidence needed to choose the next investigation.
An offline working demonstration on 17 October 2026 used Python and its standard library with synthetic markers only. It did not run an assistant, a browser, a WebMCP tool or a production collector. Four stipulated application observations represented a missing postal code, a denied delivery-scope check, an upstream timeout and a successful control. Each synthetic input also contained a prompt, an authorization-header marker and exception text that echoed input values. The timeout case included an unexpected input-field name containing a synthetic secret marker.
The inputs and literal expected records were fixed before the logging implementation was written. A projection function constructed a new record using approved metadata and known field names. It did not copy the raw input object. A separate evaluator compared that output with the literal expected records and diagnostic routes; the same author wrote both scripts, so this was not independent human validation.
A diagnostic log schema with explicit data boundaries
The demonstration used the following closed record. These are local design names, not claimed WebMCP or OpenTelemetry standard fields. All fields belong to the diagnostic record; raw prompts, arguments, headers and exception messages do not.
- Record identity: schema_version and policy_version are fixed version strings. event_id and task_id are application-issued opaque identifiers, not customer IDs or session tokens. occurred_at is the event timestamp. The fixture used fixed synthetic identifiers and timestamps.
- Execution context: tool, tool_contract and build identify the approved tool name, input-contract version and application build. They come from trusted configuration, not a user-supplied URL, filename or tool label.
- Failure evidence: stage is validation, authorization, dependency or complete. status is failed or ok. reason_code is postal_required, scope_denied, upstream_timeout or none. The fixture accepts only the declared stage/status/reason combinations.
- Input shape: known_input_fields contains only the contract names comment, order_ref and postal_code that were present. unknown_input_fields_count records the number of other keys without retaining their names. postal_code_state is missing or present; it reveals no postal-code value.
- Attempt and state: attempt is the attempt number, fixed at 1 here. mutation is none in this read-only fixture. In a write-action design, use unknown when the resulting business state has not actually been checked; an error return is not proof that nothing changed.
For example, the timeout record contained schema_version=failure-log-v1; policy_version=boundary-v1; event_id=F139-03; task_id=T139-03; occurred_at=2026-10-17T00:00:03Z; tool=delivery_estimate; tool_contract=delivery-v3; build=fixture-build-1; stage=dependency; status=failed; reason_code=upstream_timeout; known_input_fields=[comment, order_ref, postal_code]; unknown_input_fields_count=1; postal_code_state=present; attempt=1; mutation=none. Its instruction was to inspect the delivery dependency, not to retrieve the customer comment.
Field names also need a boundary. Recording every key from an arbitrary input object would have retained the secret-bearing unknown name in this example. Keeping only known contract names and counting the others preserved evidence of an unexpected field without copying it. In a real application, even a known field name or a precise timestamp can reveal sensitive context. Review the permitted metadata for the particular service, not just the values it excludes.
Redaction must preserve useful distinctions
The evaluator matched all four expected records and routed the three failures to their intended investigations. The successful control remained a non-failure. No synthetic raw-input marker appeared in those records, and replaying the same inputs produced byte-identical output. These are observations about this deterministic fixture, not a measured diagnosis rate for customer incidents.
Two deliberately damaged variants tested the boundary. Adding raw arguments to the records caused the sensitive-marker check to reject them. Removing stage and reason_code caused the diagnostic check to reject the records as insufficient. The second result matters as much as the first: a log can omit private values and still be useless if it reduces every problem to failed.
The Open Worldwide Application Security Project (OWASP) Logging Cheat Sheet recommends excluding or specially handling secrets and sensitive data, sanitizing event data against log injection, and verifying log access controls. OpenTelemetry’s sensitive-data guidance adds an important limit: instrumentation cannot determine what is sensitive in a particular context on its own. It also warns that hashing predictable identifiers may not provide the anonymization needed. Replacing a customer ID with an unkeyed hash is therefore not a reason to stop assessing its exposure.
Prefer constructing a small approved record before export to collecting a large payload and masking it in a dashboard. In this fixture, excluded values never entered the output record. A display-only mask would not establish the same boundary for stored or exported data. For deployment, examine other emitters separately, including exception handlers, request tracing and support exports. This demonstration did not exercise them or test arbitrary secret formats, transport failures or malicious log injection.
Investigation access is a separate design decision
A proposed access split gives growth analysts counts by tool, build and reason code, while engineering investigators can read the minimized event records needed to investigate a specific failure. Access to a customer record, when necessary for an identified investigation, should remain a separate authorized workflow. Do not create a raw-input archive merely because the minimized record is inconclusive. First ask whether an additional controlled observation or a synthetic reproduction can resolve the gap.
Give the proposed diagnostic store an explicit owner, purpose, access policy and retention rule. Audit reading and exporting as well as writing. Include copies and backups in the retention design, and choose the duration against the organization’s actual obligations and investigation needs rather than borrowing a universal number. OWASP’s guidance addresses restricted access, monitoring access and retention of copies; it does not approve this particular design or establish legal compliance.
For the fictional retailer, the schema is acceptable for the tested routing question because missing input, scope denial and dependency timeout remain distinguishable without raw values. It is not sufficient when the unresolved question depends on the meaning of a customer’s free-text request. That condition calls for a different, explicitly authorized investigation, not a claim that the log has reconstructed the customer’s intent.
A focused ScaleGrowth engagement can connect AgenticReadiness audit work and WebMCP consulting to this deliverable: a documented logging boundary and a replayable failure fixture. Its AI-powered data anomaly detection capability is relevant to a later question, whether failure counts have changed, but no detector was run here and no automatic integration is implied. The organic-growth team’s useful contribution is to keep the business question specific enough that instrumentation records what an investigation needs, rather than treating every available input as analytics data.
Before applying the design to live traffic, test the deployed collection path, storage permissions, exports and deletion behavior with synthetic markers, then have the responsible engineering and privacy owners assess the actual data categories. The local demonstration did not perform those checks. Preserve the distinction: a value-free event can support a bounded investigation, while production confidentiality still depends on the whole path the event takes.
References
OWASP Logging Cheat Sheet
https://cheatsheetseries.owasp.org/cheatsheets/Logging_Cheat_Sheet.html
About OWASP Foundation
https://owasp.org/about
OpenTelemetry: Handling sensitive data
https://opentelemetry.io/docs/security/handling-sensitive-data/