Multimodal Search: What Happens When Screenshots Rank

By ScaleGrowth Digital Editorial

A buyer pastes a screenshot of a competitor’s pricing page into ChatGPT and asks “what are the alternatives that cost less than this.” A procurement manager photographs a piece of equipment on a shop floor and asks Gemini to identify suppliers. A product researcher drops a chart from a quarterly report into Claude and asks for the underlying market figures. Each of these queries skips the keyword layer entirely. The model parses the image, extracts entities and facts, then retrieves candidate sources. Brand citation now depends on whether the brand’s content surfaces in that retrieval set against image inputs rather than text. The shift is real, the measurement is ragged, and the optimisation work that follows is mostly upstream of where most SEO teams currently look.

Three Modes That Behave Differently

Multimodal search is not one behaviour. Three patterns recur across observed buyer flows. The first is image-as-query: a user uploads or pastes a visual and asks for related text answers. The second is image-as-context: a user uploads a visual that supplements a text question (a chart plus “explain what changed”). The third is text-asks-for-images: a user types a question whose answer is best delivered as a labelled image or screenshot. Each mode produces different retrieval behaviour. Each rewards different upstream content investments.

The third mode is where brands have the most direct control. A user asking “show me what a balance sheet looks like for a Series B SaaS company” gets an answer that often surfaces a labelled screenshot from a publisher that has built and tagged that exact asset. The publishing brand earns an image-anchored citation. Most B2B brands have invested in zero image assets of this kind. The few that have, win the citation slot on dozens of related queries without further effort.

An Original Position on Image-Anchored Citation

The current public guidance from Google on multimodal retrieval emphasises clear, descriptive alt text and original images. That guidance is correct but insufficient. Observed behaviour across Gemini and ChatGPT vision suggests that two additional factors carry weight that alt text alone does not address. The first is the surrounding caption and figure-block text on the publishing page, which provides the model with context the image itself cannot. The second is whether the image has a stable canonical URL the model can re-fetch on demand, rather than a dynamically generated thumbnail behind a CDN signature.

The 17,848 missing alt-tag count on an Angular 17 fintech SPA audit is the obvious starting failure. The non-obvious failure on the same site: even the images that did carry alt text were served behind a dynamic-token URL pattern, which meant the model could not re-fetch them after the initial retrieval expired. Two structurally distinct fixes for two structurally distinct problems. Most audits surface the first and miss the second.

The Screenshot-Ranks Pattern

A specific sub-pattern within multimodal search is producing a meaningful share of citations on B2B queries. Question shapes like “how do I configure X in product Y” return answers where the model surfaces a labelled screenshot of the product UI with annotated steps. The screenshots being lifted are not random. They come from publishers (or the vendor itself, or an integrator partner) who have created an annotated walkthrough page with each step screenshot carrying a descriptive filename, a structured-data ImageObject reference, a contentUrl that resolves cleanly, and a caption that explicitly names the step.

The 95-variant gold-loan landing page work surfaced the inverse pattern. The page used a single template with id-parameter content swaps across six languages. Zero distinct image assets per variant. The variant work moved conversion on paid traffic, but the multimodal-search citation surface for the same gold-loan category went unwon because there was no labelled image asset to retrieve. The lesson runs both directions: a paid-media-tuned page can be content-poor on the image side and still win paid conversions; the same page wins zero organic image-anchored citations.

Investment Map Diagram

Why Stock Imagery Loses

The retrieval behaviour penalty on stock imagery is empirical, not philosophical. Stock images appear on hundreds or thousands of pages. A multimodal retrieval pipeline that hashes images and detects duplicates lowers the citation weight of every instance because none is a unique source. An original photograph or chart of the same subject, even if technically inferior, gets cited at higher rates because it has no duplicate-detection penalty. The steel exporter content engine work made this concrete. The pages winning category citations carried original product photography with gauge tables and finish-code overlays. The pages losing citations carried stock photos of generic warehouses.

The corollary for budget allocation: a category-leading brand investing in one round of original photography for its top 30 commercial pages produces a multimodal-citation moat that takes competitors months to match. The asset cost is bounded. The defensibility is high.

Measurement Is Still Ragged

The honest part of this brief is that measurement on the multimodal side lags measurement on the text side by roughly a year. The Perplexity Sonar API and OpenAI Responses API return citation arrays for text queries; the equivalent for image-input queries is partial and varies by provider. Manual sampling on a 30 to 60 image-query cohort, run quarterly, is the working substitute. Tag each query with the mode (1, 2, or 3 above), record whether the brand appeared in the response, and over time the cohort gives a defensible read on whether the upstream image-asset work is producing returns.

The pairing matters. Image-asset investment without a tracking cohort produces art-budget arguments nobody can resolve. A small tracking cohort, even at 30 prompts a quarter, makes the art budget defensible against P&L.

Practitioner Takeaway

1. Audit alt text and ImageObject schema together. Missing alt text is the obvious gap. Missing ImageObject schema on the alt-tagged images is the larger gap.
2. Stabilise image URLs. Signed CDN tokens that expire after 24 hours are a multimodal-citation killer. Move to stable canonical image URLs or accept that re-retrieval will fail.
3. Commission original imagery on the top 30 commercial pages. One round of original product or process photography per category. Stock loses on duplicate detection.
4. Build annotated screenshot walkthroughs for top use cases. Step-by-step images with descriptive filenames and structured captions. This is the mode-3 mechanic.
5. Stand up a 30-image-query tracking cohort. Quarterly manual sample. Tag by mode. Record citation presence. See the block density work and the entity infrastructure as the upstream complements. The full programme ships inside the AI visibility audit.

FAQ

Does original photography really outperform stock for AI retrieval?

Observed pattern across manufacturing and BFSI engagements: yes. The mechanism is image hashing and duplicate detection inside the retrieval pipeline. A stock image present on hundreds of pages contributes weakly to any single page’s citation weight. An original image acts as a unique anchor.

What schema properties on ImageObject matter most?

Author, license, contentUrl, caption, and representativeOfPage. The contentUrl needs to be stable and crawlable. The caption needs to name the specific entity in the image (product name, person name, location name), not generic descriptors. License signals the model can reuse without copyright risk.

Do AI-generated images carry the same penalty as stock?

Mostly yes, with one exception. Generic AI-generated illustrations face the same duplicate-detection problem because diffusion model outputs share strong stylistic and structural signatures. Diagrams and custom visualisations generated for specific data are the exception, because the data anchor (the actual numbers in the chart) is unique to the page.

Does Google Image Search still drive separate traffic?

Yes, declining. The classical Image Search interface continues to refer traffic, particularly on shopping and visual-research queries. The growth surface is image-anchored citation inside AI answers. Most B2B brands should treat classical Image Search as legacy maintenance and AI image citation as the growth investment.

How does this interact with screen-reader accessibility?

Strongly aligned. The same descriptive alt text that wins multimodal retrieval citations also serves screen-reader users. Treat alt-text discipline as both an accessibility commitment and a retrieval investment, not as a trade-off.

Audit the Image Layer

If the brand has never tested how its images surface in multimodal queries, the next deliverable is the alt-text plus ImageObject schema audit and the 30-image-query baseline cohort. Start an AI visibility audit.