Tracking LLM Mentions: Perplexity API and the Other Stacks Worth Wiring Up

By ScaleGrowth Digital Editorial

Measuring AI mention rate is no longer a manual exercise. Three APIs now make it practical to run a fixed prompt cohort, parse the response JSON, extract brand mentions and source citations, and write the result to a database for quarter-over-quarter comparison. Perplexity’s Sonar API is the most accessible of the three. The OpenAI Responses API with web search enabled covers the ChatGPT side. The Google Generative Language API plus a parallel SerpApi pull covers AI Overviews and AI Mode. This piece sets out the wiring, the cost shape, and the schema ScaleGrowth Digital uses on a 300-prompt monthly run.

The Three Stacks That Matter

Each stack covers a different surface a buyer might encounter. Perplexity API answers the question of how the brand performs on a research-grade interface used heavily in B2B procurement. OpenAI Responses with browsing covers the answer surface most likely to influence end-consumer queries. The Google pair covers the answer surface integrated directly into classical search. A monitoring programme that runs only one of the three answers a third of the question and over-fits to the one surface measured.

On a recent BFSI engagement, the same 30-priority-prompt set produced sharply different brand mention rates across the three. Perplexity Sonar cited the brand in 42 percent of answers. OpenAI Responses with browsing cited the brand in 23 percent. Google AI Overview cited the brand in 15.6 percent and AI Mode in 19 percent. A monthly programme that read only Perplexity would have over-stated category visibility by a wide margin.

Why Perplexity API Is the Right Starting Stack

Three reasons. The Sonar API returns sources in a structured citations array, which removes the need to parse free-form answer text for source extraction. The cost per query sits low enough that 300 prompts can run for under three dollars at standard tier rates as of the most recent published pricing. The model selection is exposed (sonar, sonar-pro, sonar-reasoning), so a programme can hold the model constant across quarterly runs and avoid the drift problem that plagues ChatGPT measurement when the underlying model silently changes.

The trade-off: Perplexity skews toward research-mode users. A consumer-category brand under-indexes on Perplexity relative to its true share of voice on the broader AI answer surface. ScaleGrowth has not seen a case where Perplexity-only tracking produced a misleading positive read, but the inverse failure (a brand looks weaker on Perplexity than it deserves) does happen on consumer F&B and direct-to-consumer engagements. The 86-store F&B brand command-centre work surfaced this asymmetry directly: Perplexity rarely surfaced QSR brands by name; Google AI Overview did.

The Schema That Survives Quarterly Drift

The persistent-store schema needs five tables to be useful. A prompts table (prompt text, intent label, cohort id, version). A models table (provider, model id, version label, retrieval mode). A runs table (timestamp, prompt id, model id, raw response JSON, parse status). A mentions table (run id, brand mention yes/no, position in answer text, sentiment label). A citations table (run id, source url, citation order, snippet text). All five tables key off a stable cohort id, so re-running the same 100 or 300 prompts every quarter produces a clean delta.

The trap most teams fall into: storing only the parsed mention boolean and discarding the raw JSON. The raw JSON is what makes the data defensible when leadership asks why the number moved. ScaleGrowth’s 300-prompt scans cache every raw response. On a healthcare engagement, 75 raw JSONs were retained for reproducibility. The cache is what allowed a Claude supervisor pass to retroactively reject five fabricated Gemini claims that would otherwise have polluted the mention dataset.

The Wiring Diagram

Cost Shape Of A 300-Prompt Monthly Run

Working numbers as of the most recent vendor pricing pages, for budgeting only. Perplexity Sonar at the standard tier runs roughly one cent per query at typical answer lengths, putting a 300-prompt monthly scan at about three dollars per month. OpenAI Responses with web_search runs higher because of the tool-call overhead, sitting closer to fifty to eighty cents per query at GPT-4-class models, putting the same cohort near two hundred dollars per month. SerpApi google_ai_overviews is roughly a cent per query before SerpApi plan minimums, putting a 300-prompt scan under four dollars per month plus the plan floor. A full three-stack monthly run lands under three hundred dollars in API costs before any analyst time. The cost objection most teams raise dissolves at this level.

The larger cost is the parsing and validation layer, not the API calls. A brand regex that catches the trade name, parent-company name, ticker symbol, and three to five common misspellings, plus URL canonicalisation across http/https and trailing-slash variations, is two to three days of engineering for a clean first build. Maintenance is light after that.

Where Sentiment Classification Belongs

A brand mention is not the same as a positive brand mention. A model that names the brand inside an answer about regulatory penalties or product recalls counts in raw mention rate but harms the brand. The mentions table needs a sentiment column, with a per-mention classification produced either by a small sentiment model or by an LLM judge pass. ScaleGrowth runs a downstream classifier on every cached response, with neutral, positive, negative, and contextual-warning categories. The contextual-warning category catches mentions that read neutrally in isolation but appear inside an answer about category problems (fraud, recalls, regulatory action) where any brand mention is bad.

The 18-slide pitch to the instant-loan fintech with $6M funding flagged this exact issue. The brand surfaced in 8 percent of category answers, but two thirds of those mentions were inside answers explaining how to spot predatory lending. Raw mention rate told a positive story. Sentiment-adjusted mention rate told the opposite story. The pitch led with the sentiment-adjusted figure because it was the figure the CMO needed to see.

Practitioner Takeaway

1. Wire Perplexity Sonar first. Lowest cost, structured citations, fastest path to a working monthly run. Add OpenAI Responses and SerpApi AIO in the next sprint.
2. Cache the raw JSON. Every run, every prompt, every model. Without the cache the data is undefendable. With it, a supervisor pass can retroactively correct fabrications.
3. Freeze the prompt cohort. Same 100 or 300 prompts every quarter. The frozen cohort is the only honest delta.
4. Classify sentiment, not just presence. Negative mentions inflate raw rate and mislead leadership. The sentiment column belongs in the schema from day one.
5. Pair mention tracking with the upstream work. Tracking without remediation produces a dashboard nobody acts on. The ScaleGrowth AI visibility audit ships the upstream content and schema fixes alongside the tracking. See also the related notes on quotable content blocks and Wikipedia and Wikidata infrastructure.

FAQ

Can a single API give a complete picture?

No. Each API covers one answer surface. Buyer behaviour spans at least three of those surfaces (Perplexity, ChatGPT, Google AI Overview). A complete picture requires running the fan-out across all three at minimum.

How often should the scan run?

Monthly is the right cadence for the head 100 prompts. Quarterly for the full 300. Weekly is rarely informative because the AI surfaces refresh on different cycles and within-month noise dominates real signal.

Does the brand regex need maintenance?

Yes. New product names, sub-brands, acquired entities, and common LLM misspellings (the model occasionally writes the brand with a missing letter or as a near-homophone) all need to be added. A monthly five-minute review of flagged misses is sufficient maintenance.

Should the prompt cohort include branded queries?

Split the cohort. Eighty percent non-branded category prompts (where the brand has to earn citation) and twenty percent branded prompts (where the brand should always appear). The branded twenty percent is the early-warning indicator for entity record problems. If a model fails to cite the brand on a direct branded query, the Wikidata and entity work is broken before the category work matters.

What happens when a model deprecates or changes?

The models table captures the version label. When a provider deprecates a model (OpenAI retires gpt-4-turbo, Perplexity replaces sonar with sonar-pro), the next run records under the new model id. The historical data stays comparable within the old model id; the new model id starts its own time series. Mixing the two without flagging the cut is the most common reason published AI mention metrics later turn out to be uninterpretable.

Stand Up The Tracker

If the brand has no programmatic mention tracking in place, the next deliverable is the three-stack wiring with the 300-prompt frozen cohort and the schema above. Start an AI visibility audit.