AI Agent Development: Building Production Agents That Hold Up Under Real Workloads

AI agent development at ScaleGrowth Digital builds production agents that ship into a real business process and stay there. The team works in Python and TypeScript on top of Anthropic Claude, OpenAI GPT, Google Gemini and open-weight models where the data sensitivity demands it. Engagements run six weeks for a prototype, twelve to sixteen weeks for a production agent, and a six-month retainer for ongoing operation. The output is a running agent, the evaluations that gate every release, and the runbook the operations team uses on day one.

Where agent projects go wrong

Most agent projects start as demos and stay there. The first failure mode is missing evaluations. A team ships an agent that handled six test cases beautifully and discover, four weeks in, that the agent silently fabricates payment account numbers on the seventh case. There is no eval suite to catch it because the eval suite was never written. The agent goes back to demo state for another month. On a multi-LLM diagnostic for a healthcare specialty chain, the team caught five fabricated claims from a single LLM call before they reached the deliverable. The lesson is that the supervisor and the eval pass are the work, not the prompt.

The second failure mode is tool sprawl. An agent gets given fifty tools, retrieves the wrong one half the time, and ends up calling search when it should have called the catalogue API. The fix is a tighter tool surface with explicit routing rules and a tested retrieval layer over the tool descriptions. The third failure mode is observability blindness. An agent goes into production with no per-tool latency tracking, no token-cost dashboard and no per-conversation success metric. The first time something goes wrong the team is reading raw logs at midnight.

Our build approach

Six stages, each gated by an evaluation pass.

Stage 1: scope and eval-first design. Week one. The team and the business owner write the eval suite before any prompt is written. The suite covers the must-handle cases, the must-refuse cases, the edge cases and the adversarial cases. Hallucination guardrails are written here, with a hardcoded list of values the agent is forbidden to invent (account numbers, prices, regulatory citations, medication dosages).

Stage 2: tool design. Weeks two to three. Tools are kept small, typed and documented. Each tool ships with worked input and output examples. The tool catalogue is tested for retrieval accuracy before the agent is built on top of it.

Stage 3: agent loop and prompt. Weeks three to five. The agent loop is built (single-step, multi-step, plan-execute, or supervisor + sub-agent depending on the use case). The prompt is iterated against the eval suite. Concurrency caps are set to match downstream rate limits; on the BFSI content build, the cap was twelve concurrent Sonnet calls.

Stage 4: supervisor layer. Weeks five to seven. For YMYL and high-stakes use cases, a supervisor agent reads every output before it ships. The supervisor pattern is what caught five fabricated Gemini claims on the healthcare project. It is also what gates every BFSI content brief that lacks a mandatory disclaimer.

Stage 5: observability and cost. Weeks seven to nine. Structured logs, per-tool latency, token-cost dashboard, per-conversation success metric, and per-user rate limits. The team will not ship an agent to production without a kill switch and a cost ceiling.

Stage 6: handover and runbook. Weeks nine to twelve. The operations team gets the runbook, the eval suite, the deployment artefacts and a two-hour walk-through. The brand owns the agent on day one of production.

Related reading: the WebMCP service (which is the tool layer agents call against), the AI visibility service (which measures and engineers agent citation behaviour) and the team page.

Agents in production

For a major BFSI lender, the content brief engine is a sub-agent pattern: twelve Sonnet sub-agents generate briefs in parallel under a supervisor that runs nine Pydantic validation gates on every output. A brief that omits a heading tree, fabricates a citation or fails a YMYL guardrail fails the pipeline. Two retries with the failure message appended to the prompt, then manual review. Four batches over five weeks produced 794 schema-validated briefs (Batch 1 = 215, Batch 2 = 57, Batch 3 = 356, Batch 3A = 166), with the final two batches at 100 percent Pydantic pass.

For an industrial-materials manufacturer in Australia, two agent layers ran the rank report and content recommendation engine. Eighteen parallel Jina Reader agents handled the Phase 3 sitewide audit on 380 URLs. Twenty parallel agents handled 473 branded-SERP fallback checks in the Phase 4 rank report. Ten parallel Sonnet agents validated 363 keyword-URL pairs. The combined output was 27,818 lines of JSON across ten page-level recommendations, plus a triaged contamination audit of 2,081 issues.

For a healthcare specialty chain, a multi-LLM supervisor pipeline (four WebSearch agents in parallel plus Gemini CLI plus DataForSEO plus a Claude supervisor) manually verified 30 priority Chennai kidney and urology queries against SERPs, GBP, Instagram, Facebook, YouTube and Tamil-language coverage. Seventy-five raw JSONs cached. Five fabricated Gemini claims caught and rejected before they hit the deliverable.

Working cadence

The team is one agent lead, one prompt engineer, one back-end engineer and one evaluations engineer. The brand provides product, security and one subject-matter expert who can answer eval-suite questions. Weekly review on Tuesday, fortnightly demo on Friday, monthly cost-and-success report on the first of every month.

Retainer work covers prompt iteration on production data, evaluation suite extension, new tool additions and cost optimisation. A typical retainer ships a model upgrade (for example Claude Opus 4 to Claude Opus 4.5) inside one sprint, with the eval suite catching the regression cases before the upgrade lands in production.

Pricing

The prototype (six weeks) is priced at fifteen thousand US dollars internationally and ten lakh rupees in India. The production agent (twelve to sixteen weeks) is priced at forty to seventy-five thousand US dollars and twenty-five to fifty lakh rupees. The retainer (six months minimum, one engineering day per week plus on-call rotation) is priced at four thousand to eight thousand US dollars per month and three to five and a half lakh rupees per month. Inference costs are passed through at provider rates.

Frequently asked questions

Which model providers do you build on? Anthropic Claude (Opus, Sonnet, Haiku), OpenAI GPT (4.1, 5, o-series), Google Gemini (Pro, Flash), and open-weight models (Llama, Mistral, DeepSeek) where data sensitivity, latency or cost demands them. The choice is made per use case at stage one.

How do you handle hallucination in YMYL use cases? Three layers. First, an eval suite with explicit must-refuse cases and a hardcoded forbidden-invent list. Second, a supervisor agent that reads every output before it ships. Third, a Pydantic validation gate that fails outputs missing mandatory disclaimers or carrying prohibited absolute claims.

What is the minimum engagement? The six-week prototype. The prototype proves the eval pass rate and the per-conversation cost. Brands that move forward extend into the production build.

Do you fine-tune or only prompt? Both, depending on the use case. Fine-tuning makes sense for narrow, high-volume tasks where the prompt cost dominates total cost. Most production agents ship on prompts plus retrieval first, with fine-tuning added only after the eval suite shows a ceiling that prompts cannot break through.

Can the brand own and host the agent independently? Yes. The default deployment is to the brand’s own infrastructure (AWS, GCP, Azure, Vercel, Cloudflare Workers, or a private Kubernetes cluster). All artefacts (prompts, eval suite, deployment configs, runbook) hand over at the end of the engagement.

By ScaleGrowth Digital Editorial

{
“@context”: “https://schema.org”,
“@graph”: [
{
“@type”: “Service”,
“@id”: “https://scalegrowth.digital/services/ai-agents/#service”,
“name”: “AI Agent Development”,
“serviceType”: “Production AI agent builds with eval-first design, supervisor layer, observability and runbook handover”,
“provider”: {“@type”: “Organization”, “name”: “ScaleGrowth Digital”, “url”: “https://scalegrowth.digital/”},
“areaServed”: “Worldwide”,
“offers”: [
{“@type”: “Offer”, “name”: “Six-week prototype”, “price”: “15000”, “priceCurrency”: “USD”},
{“@type”: “Offer”, “name”: “Production agent (twelve to sixteen weeks)”, “priceSpecification”: {“@type”: “UnitPriceSpecification”, “minPrice”: “40000”, “maxPrice”: “75000”, “priceCurrency”: “USD”}}
]
},
{
“@type”: “FAQPage”,
“@id”: “https://scalegrowth.digital/services/ai-agents/#faq”,
“mainEntity”: [
{“@type”: “Question”, “name”: “Which model providers do you build on?”, “acceptedAnswer”: {“@type”: “Answer”, “text”: “Anthropic Claude, OpenAI GPT, Google Gemini, plus open-weight Llama, Mistral and DeepSeek where data sensitivity, latency or cost demands them. The choice is made per use case at stage one.”}},
{“@type”: “Question”, “name”: “How do you handle hallucination in YMYL use cases?”, “acceptedAnswer”: {“@type”: “Answer”, “text”: “Three layers: an eval suite with explicit must-refuse cases and a forbidden-invent list; a supervisor agent that reads every output before it ships; and a Pydantic validation gate that fails outputs missing mandatory disclaimers.”}},
{“@type”: “Question”, “name”: “What is the minimum engagement?”, “acceptedAnswer”: {“@type”: “Answer”, “text”: “The six-week prototype. It proves the eval pass rate and per-conversation cost. Brands that move forward extend into the production build.”}},
{“@type”: “Question”, “name”: “Do you fine-tune or only prompt?”, “acceptedAnswer”: {“@type”: “Answer”, “text”: “Both. Fine-tuning makes sense for narrow, high-volume tasks where prompt cost dominates. Most production agents start on prompts plus retrieval, with fine-tuning added when the eval suite shows a ceiling prompts cannot break.”}},
{“@type”: “Question”, “name”: “Can the brand own and host the agent?”, “acceptedAnswer”: {“@type”: “Answer”, “text”: “Yes. Default deployment is to the brand’s own infrastructure (AWS, GCP, Azure, Vercel, Cloudflare Workers, or private Kubernetes). All artefacts hand over at the end of the engagement.”}}
]
}
]
}