AI Search Visibility Audit: The Complete Guide to Finding and Fixing Your Brand's AI Search Gaps

Most teams run an AI visibility audit backwards. They open five chat interfaces, ask a dozen questions about their category, notice their brand is missing from some of the answers, and then start publishing blog posts.

That isn’t an audit. It’s a mood check.

A useful audit starts with an observation: the assistant said this, on this date, in this locale, for this prompt. Then we work backwards to something you can change. Every step should help you repeat that process. Otherwise, you’re collecting activity without learning what to fix.

We’ll work through five stages: reach → representation → retrieval → attribution → fix. Keep them in that order. A CDN block and weak content can produce the same symptom: your brand doesn’t appear. But they need different fixes. Check access first, or you could spend a quarter improving pages the crawler can’t read.

Two things have changed recently enough that older versions of this advice are wrong. In February 2026 Microsoft shipped a first-party citation dashboard, and by October 2026 it could tell you which queries other sites win and yours doesn’t. And in October 2026 the single most trusted AI referral measurement broke for roughly eight days, which is a useful lesson about how much any of this data can be trusted. Both are covered below with their sources.

---

Why AI Visibility Needs a Different Kind of Audit

Classical search auditing works because there’s a stable, rankable artefact. You have a query, a result page, and a position. The position moves slowly enough that a monthly snapshot is meaningful, and the mechanism - crawl, index, rank - has decades of documented behaviour behind it.

AI answers have none of that stability.

There’s no position. When Perplexity cites four sources, there’s no “you were third”. Microsoft is explicit about this: its Total Citations metric counts citations “without indicating placement or presentation within a specific answer,” and its Average Cited Pages metric “does not indicate ranking, authority, or the role of any page within an individual answer” (Bing Webmaster blog, February 2026).

There’s also no stable input. The same prompt produces different answers across runs, across model versions, and across geographies. OpenAI’s web search API accepts an explicit user_location - country, city, region, timezone - precisely because results are geography-dependent (OpenAI web search docs). If your audit runs your prompts from one office while your buyers sit on another continent, you’re measuring the wrong market and calling it a finding.

And the measurement instruments are brand new and wildly asymmetric. One engine hands you citation-level data with grounding queries attached. Another folds everything into a single undifferentiated “Web” bucket. A third hands you nothing at all.

Google’s position here has moved, and it matters for how you scope an audit. The page most often quoted says sites appearing in AI features are “included in the overall search traffic in Search Console. In particular, they’re reported on in the Performance report, within the ‘Web’ search type” (Google Search Central, page last updated 2025-12-10 UTC). That remains accurate about the standard Performance report.

It’s no longer the whole picture. Google has since published guidance naming a dedicated surface: “To measure how your content is performing in generative AI features on Google Search and Discover, use the Generative AI performance report in Search Console” (Google’s Guide to Optimizing for Generative AI Features, page last updated 2026-07-10 UTC). The practical consequence for an audit: “Google does not break out AI traffic” is now an out-of-date blanket claim. If your baseline methodology was built on that assumption, recheck it before you defend the number in a board meeting.

An audit has to be built for that reality. Which means: fixed prompts, fixed locale, fixed run cadence, dated snapshots, and a classification scheme that survives the fact that the underlying answers are noisy.

---

The Five Questions an Audit Has to Answer

Before any tool or prompt set, write down the five questions your audit exists to settle. Most failed audits skip this and end up collecting data that answers none of them.

#	Question	Fails when
1	Are we reachable? Can the crawlers that feed AI answers fetch our pages at all?	A CDN, WAF or robots.txt rule silently excludes us
2	Are we representable? Is there an unambiguous entity record of who we are?	Name collisions, missing sameAs, absent organisation markup
3	Are we retrievable? For buyer-shaped questions, do we surface in the candidate set?	We rank for keywords but never for the natural-language question
4	Are we attributed? When we do surface, do we carry the claim, or does someone else?	We are the evidence; a directory or Reddit is the voice
5	What do we fix first?	Symptom-driven fixes with no impact/effort ordering

Questions 1 and 2 are usually skipped, and they’re the cheap ones. Both resolve in an afternoon. Skipping them to get to question 3 faster is how teams spend six figures producing content that’s never fetched.

---

Stage 1 - Build the Prompt Set Before You Touch a Single Assistant

The prompt set is the instrument. Everything you measure flows through it, so its construction determines whether your audit means anything.

Mine prompts from data you already own

Don’t invent them. An invented prompt set measures the prompts you thought of, which is a biased sample of your own mental model, not of your market.

Four sources, in descending order of usefulness:

First-party query data. Your Search Console queries are the closest thing you have to a record of how real people phrase this problem. The Search Analytics API will return them grouped by the query dimension, with filters for country and device (Search Console API, last updated 2026-08-11). Two caveats, both from Google’s own documentation and both easy to get wrong: the API “is bounded by internal limitations of Search Console and does not guarantee to return all data rows but rather top ones,” and the default type is web - the combined “All” tab - which is where AI feature traffic now lives. Expect a long tail you can’t see. Treat your prompt set as a sample of known-unknown size, not a census.

Sales and support transcripts. The most valuable and most under-used source. Take the first thirty seconds of ten recorded discovery calls. The vocabulary buyers use for their problem is almost never the vocabulary on your homepage, and it’s exactly the vocabulary that shapes the fan-out queries an assistant issues.

Review sites and community forums. Where buyers describe the problem in the wild, including the words that embarrass your positioning team.

Your own comparison pages. A strange but reliable signal: the objections you wrote paragraphs to defeat are questions buyers actually ask.

Stratify, don’t average

A flat list of fifty prompts produces a number that means nothing, because commercial and informational questions behave completely differently.

Microsoft already published a usable grid. Its June 2026 release classifies grounding queries into intents including Informational, Commercial, Navigational, Learn and Solve, Research, Creation and Local (Bing blog, 2026-06-16). That taxonomy is vendor-specific and you can adapt or replace it, but it’s a better starting grid than inventing your own buckets.

On top of intent, split every prompt into one of three families:

• Branded - contains your product or company name. Measures whether the entity you built matches the entity the model holds.
• Category - describes the problem or job without naming a vendor. This is where discovery happens and where most audits should spend their weight.
• Competitive - names a competitor, or asks for a comparison. Measures displacement.

Reporting a blended number across all three is the single most common reporting mistake. Branded presence is usually high and tells you almost nothing about demand capture.

Pin the variables or the number is fiction

Every row in your prompt register needs: a prompt ID, the exact prompt string, its intent label, its family, and the locale it’s run against. Every run needs a UTC timestamp, the platform, and the model or product surface.

Then hold all of it fixed. Same locale. Same time of day if you can manage it. Same prompt text, character for character - assistants are extremely sensitive to phrasing, and a “minor” rewording between cycles turns your time series into noise.

How many prompts

For a first full audit, my own recommendation is 40 to 60 prompts: roughly 30 category, 15 competitive, 10 branded, with at least a third carrying a real geographic or industry qualifier. The constraint is that each prompt must be run on every platform across at least three repeats, and that a 60-prompt set across four platforms at three repeats is already 720 observations to classify. Larger sets are honest but exhaust the budget that makes the classification step actually rigorous. If you have more, stratify harder rather than sampling randomly.

---

Stage 2 - Record the Answers With a Reproducibility Contract

An answer you didn’t store is an answer you can’t audit. Capture it in full, with enough metadata that a colleague could reproduce the observation.

Per run, record:

Field	Why it matters
Prompt ID + verbatim prompt	The instrument
UTC timestamp	Answers decay; unanchored data rots
Platform and product surface	”ChatGPT” is not one thing
Model / version if exposed	Cannot be controlled; must be recorded
Locale	Answers vary by geography
Full answer text	The only ground truth for mischaracterisation analysis
Cited URLs	Attribution
All sources consulted	Distinct from citations - see below
Brand mentioned? Y/N	Failure-mode input
First-mention position	Ordering signal, even though vendors refuse to score it
Claim accuracy flag	Human judgement, recorded as judgement
Run index (1 of N)	Lets you compute variance instead of guessing at it

Those two bolded rows deserve explanation, because getting the distinction wrong quietly corrupts your diagnosis.

OpenAI’s documentation distinguishes the inline citations shown to the user from the full sources list, and warns that “the number of sources is often greater than the number of citations” (OpenAI web search docs). A page can shape an answer without being linked. If you only record citations, you’ll systematically misattribute authority and conclude, wrongly, that a domain you never see cited had no influence.

Conversely, the same docs state that search_context_size “does not set an exact token count or guarantee a specific number of sources or citations” - so source counts vary between configurations and aren’t a stable denominator.

Repeat every prompt

The same prompt can produce different answers. One run can look like a trend when it isn’t. Use at least three repeats, or five for prompts you plan to publish. If you have to rely on a single answer, mark it as a single run in the row.

The closest thing to a grounding log

Perplexity’s Agent API is the most inspectable of the mainstream options. The worked example embedded in its own documentation returns a search_results object containing the exact queries the search tool issued - three related queries for one user question - plus per-result url, title, snippet, date, last_updated and source (Perplexity Agent API docs). That’s a grounding query log, in the response body, from a first-party source.

For that definitional query, the results are dominated by IBM, GeeksforGeeks, Google Cloud, Databricks, AWS, NVIDIA, Stanford HAI and Wikipedia. These are large reference domains. Before you decide a competitor has better content, check which sources are filling the answer.

Perplexity also publishes a separate Search API returning “raw, ranked web search results” before any answer is generated (Perplexity docs). For an audit this is more useful than it sounds: it lets you separate is our page in the candidate set at all from was our page chosen for the answer. Those are different problems with different fixes, and conflating them is a recurring root-cause error.

The thing nobody will tell you

No major platform publishes retrieval weights. You’ll never be told why a page was selected, and any vendor or consultant who claims otherwise is describing a correlation they observed and calling it a mechanism. Microsoft comes closest by exposing the grounding queries it used, and even there the data is “a sample of overall citation activity” that they “will continue to refine.” Your audit infers causes from evidence. Say so when you report it.

---

Stage 3: Turn Runs Into Three Separate Numbers, Not One

Collapse these three and you lose the ability to diagnose anything. They move independently and they have different causes.

1. Presence rate

The share of prompts in which your brand appears at all, per platform, per prompt family.

presence = runs mentioning brand / total runs

Report it with its denominator and, where you have them, confidence intervals from your repeat counts. A presence rate of “40%” from six runs isn’t a finding.

2. Claim integrity

Presence is worthless if the presence is wrong. When you appear, does the answer attribute the capability you care about, at the right altitude, without contradicting your positioning?

You need a person for this step. Read the answer and mark each claim as accurate, partially accurate, or mischaracterised. Have two people score a sample independently. If they disagree, look at why. Your positioning may be ambiguous, and that’s a finding worth recording.

Claim integrity is the metric most likely to change the business, and the least likely to move when you publish more content. It moves when you fix what the model believes, which is usually a corroboration problem rather than a content volume problem.

3. Citation share

Here the definition matters, and Microsoft has published a good one. Citation Share is “the percentage of citations attributed to your site out of all citations shown across all sites for that same grounding query” (Bing blog, 2026-06-16).

Note what comes next in that same post, because it bounds what you can do with the number: Bing states Citation Share “is designed as an observational metric – not a ranking system or a competitive scoreboard. It does not expose competitor domains, represent traffic share, or assign quality scores to content.”

That’s the honest ceiling of first-party AI visibility data. One engine gives you your own citation share against an anonymous field. Nobody gives you the field’s contents.

Cross-brand share of voice: you have to compute it yourself

Which means the denominator is your decision, and it’s the most consequential decision in the audit. Three ways to do it, three different numbers:

• Prompt-set share of voice. Among brands mentioned across your prompt set, your share of mentions. Easy, reproducible, entirely dependent on whether your prompt set is representative.
• Category share of voice. Restrict to category-family prompts only, excluding branded and competitive families. Narrower and more honest about discovery.
• Opportunity share of voice. Restricted to prompts where at least one competitor appears. Most actionable, most biased toward whatever your competitor set happens to be.

Fix the definition at the start, write it into the report template, and never recompute it mid-audit. A share of voice that changed because you changed the denominator isn’t a trend.

A worked illustration, since the arithmetic is easy to get wrong: across 30 category prompts on four platforms at three repeats - 360 runs - your brand is mentioned in 126 (35.0%). Of those 126, 71 attribute the capability you sell (56.3% claim integrity). Your domain is cited in 88 runs; total citations across all domains in those runs number 352, giving a citation share of 25.0%. Three competitors are present in 210, 143 and 61 runs respectively. That’s 540 brand mentions across the set, so mention share of the four is 23.3% for you and 38.9%, 26.5% and 11.3% for them.

These are example numbers. Look at the pattern: presence is decent, but claim integrity is weak. The leader has a much larger share of mentions, while a quarter of the citations are yours. That points us toward attribution. Use the routing table in the next section to choose a fix before producing more content.

---

Stage 4: Classify the Failure Mode

Classify every observed run. No exceptions, including the runs that went well - a mode you can’t assign is a prompt you haven’t read closely enough.

Mode	Name	Detection rule	Illustrative symptom
FM-1	Absent	Brand does not appear; no first-party domain cited	”What are the best tools for X?” returns three vendors, none of them yours
FM-2	Named, unclaimed	Brand appears, but with no capability, price or category claim you own	Listed as an example of a category rather than as a tool for a job
FM-3	Named, mischaracterised	Brand appears with a claim that is wrong, stale, or attributed to a competitor’s strength	Your flagship feature described as a competitor differentiator
FM-4	Named under a competitor frame	Brand appears only as a runner-up, alternative, or “other options” entry	Present in a comparison but structurally subordinate to one rival

Two clarifications that keep this workable.

FM-3 isn’t a model bug. It’s almost always the model accurately reporting something a third party asserted. Classify it as a content defect on someone else’s site before you classify it as a model defect, because the two have opposite remedies.

FM-4 is the mode most teams fail to record, because they filter their audit views to “responses that mention us” and stop reading. It’s also the mode with the clearest commercial signal: you’re in the comparison and you’re losing it.

The orthogonal axis: mentioned versus cited

Run the classification on two axes, not one. For each run ask separately:

• Are we mentioned in the answer text?
• Are we cited as a source?

That produces four cells, and each has a different remedy:

	Cited	Not cited
Mentioned	Fully present. Defend it.	FM-2 territory. You supplied the facts; someone else supplied the voice.
Not mentioned	Strongest signal in the whole audit. You are the evidence and the answer still excludes you. Someone is paraphrasing you without crediting you.	FM-1. Fix reach or retrievability.

The not-mentioned-but-cited cell is under-reported because it requires reading the source list of answers that never name you. It’s also where the most useful work tends to live, because you have already cleared every access and authority hurdle - the model chose you and then didn’t print your name.

How this relates to what Microsoft now ships

In October 2026 Bing added a Topics Portfolio beta that splits each topic into Your Grounding Queries - where you appear - and Opportunity Queries - “the strongest queries across your whole portfolio where other sites appear and yours does not” (Bing blog, 2026-10-13).

That’s a genuinely useful instrument, and it’s the first-party realisation of one cell of your matrix: FM-1, on one engine. Use it. Two cautions. It’s explicitly labelled beta, and Microsoft lists real limitations: classification is AI-generated and “can occasionally be imperfect”; topics with insufficient site presence “are not shown, so smaller sites may see a shorter list or may not even see this feature”; branded queries can surface as opportunity queries; and the numbers are approximate with citation volumes “deliberately shown as ranges.”

It doesn’t cover ChatGPT, Gemini or Perplexity. It doesn’t distinguish FM-2 from FM-4. And it gives you nothing about claim accuracy. The four-mode scheme plus the two axes is what you need to close the rest.

---

Stage 5: Route the Failure to a Root Cause

Here is the part that separates an audit from a report. Each mode has a small number of candidate mechanisms, and each mechanism has a cheap discriminating test. Run the test; do not guess.

RC-A - Crawl and access failure

The symptom: FM-1 everywhere, across every platform, including branded prompts.

If you’re absent even when someone asks for you by name, you have an access problem, not a visibility problem. The test isn’t a ranking check. It’s a log check.

Each platform now splits its crawlers by purpose, and conflating them is the most common cause of this failure:

• OpenAI runs four distinct user agents: OAI-SearchBot for search, GPTBot for training, ChatGPT-User for user-initiated fetches, and OAI-AdsBot for ad landing-page validation. OpenAI’s documentation states directly that “Sites that are opted out of OAI-SearchBot will not be shown in ChatGPT search answers” - while ChatGPT-User “is not used to determine whether content may appear in Search.” Blocking GPTBot to protect your content doesn’t block search indexing; blocking OAI-SearchBot by accident removes you from search entirely.
• Perplexity separates PerplexityBot, which surfaces and links sites, from Perplexity-User, which serves user requests and - per Perplexity’s own documentation - “generally ignores robots.txt rules” because a user requested the fetch.
• Anthropic publishes three as of April 2026: ClaudeBot (training), Claude-User (user-initiated) and Claude-SearchBot (search indexing), and states that disabling Claude-SearchBot “prevents our system from indexing your content for search optimization.”
• Google folds AI features into Googlebot’s crawl, and controls them with nosnippet, data-nosnippet, max-snippet and noindex (Google Search Central).

Two operational details that catch people out. Both OpenAI and Perplexity state that robots.txt changes take up to roughly 24 hours to apply, so a rule you fixed this morning is still live this afternoon - don’t re-run the audit to check. And Anthropic warns that IP blocking “may not work correctly or persistently guarantee an opt-out, as doing so impedes our ability to read your robots.txt file.” A WAF rule that returns 403 to a crawler by IP can therefore break the crawler’s ability to read the rule that would have allowed it.

The discriminating test: group your server or CDN logs by user agent over 30 days. Look for the search-class agents specifically. Then look at status codes. Cloudflare’s AI Crawl Control exposes exactly this - per-crawler request counts, a status-code distribution chart covering 2xx through 5xx including 403 blocks, and a Content Format chart comparing what crawlers request via Accept against what your origin serves via Content-Type (Cloudflare docs). A high-403 row for one operator with healthy rows for others is a firewall finding, and it is fixable the same day.

RC-B - Entity ambiguity

The symptom: FM-1 on unbranded prompts, but partial presence on branded ones. Or FM-3 that isn’t traceable to any third-party page.

The model can’t represent you because it can’t resolve what you are. Google’s documentation on Organization structured data (last updated 2026-09-08) describes the mechanism: markup on your home page “can help Google better understand your organization’s administrative details and disambiguate your organization in search results,” with some properties used behind the scenes specifically for disambiguation.

Three things to check, in order:

1. Do you publish Organization markup? Note Google’s careful framing: “There are no required properties; instead, we recommend adding as many properties that are relevant.” An empty Organization block isn’t entity work.
2. Is sameAs populated? Google defines it as the URL of a page on another site with information about your organisation - your profile on a review platform, a social profile, an industry registry. This is the bridge that connects your site to profiles you don’t control, and it’s the field most often left empty.
3. Does your logo meet the stated spec? At least 112×112px, crawlable and indexable, on a supported format.

And note the limit Google itself sets: “Google does not guarantee that features that consume structured data will show up in search results.” Entity markup is a representability investment, not a visibility purchase.

RC-C - Third-party source dominance

The symptom: FM-1 or FM-4 on category prompts, while your site is cited elsewhere. Or your not-mentioned-but-cited cell is populated.

You’re being read and losing the vote. Run the source analysis before you touch content: for every FM-1 category prompt, record who is cited and what kind of domain each is.

The pattern that emerges is usually one of four: major reference aggregators; review and comparison platforms; community forums; or a competitor’s own comparison page. Each has a different fix and none of them is “write more posts.”

Microsoft’s Topics Portfolio gives you the ecosystem shape of this directly: Topic Size (approximate total citations across all sites for the topic) and Sites Cited (approximate number of sites appearing for queries in that topic), which it describes as indicating “how crowded the topic is” (Bing blog, 2026-10-13). A topic with a large Topic Size and a high Sites Cited count is a crowded topic you aren’t in. That’s a distribution problem.

RC-D - Retrievability and structure

The symptom: FM-2. You’re found, but the retrieved passage doesn’t contain a claim you own.

The page was retrieved and didn’t carry the answer in quotable form. This is the fixable one, and the guidance is consistent across vendors. Google lists what still applies: content findable through internal links, “important content is available in textual form,” and “your structured data matches the visible text on the page.” Bing’s February post makes the mechanism concrete - “Clear headings, tables, and FAQ sections help surface key information and make content easier for AI systems to reference accurately” (Bing Webmaster blog).

The audit test: for each FM-2 run, open the cited page and ask whether the sentence the assistant produced could have been lifted from it. If not, your content is being retrieved as context rather than as evidence.

RC-E - Corroboration gap

The symptom: FM-3, and FM-2 that persists after RC-D is fixed.

If no credible source outside your site supports the claim, repeating it on your own pages won’t help. The model may hedge, credit someone else, or leave it out. This is the hardest cause to fix and usually the slowest. It’s why publishing more can leave claim integrity unchanged.

Microsoft’s guidance points here too: “Align text, images, and video so they consistently represent the same entities, products, or concepts,” and support claims with “examples, data, and cited sources” to build trust when content is reused in AI answers. Google’s list adds a practical one - keep Merchant Center and Business Profile information current, since those are independent surfaces that describe you to systems that never read your homepage.

---

The Routing Table

One table. Find the row matching your mode, run the test, and stop when it discriminates.

Mode	First test	If positive	Root cause	Fix class
FM-1 across all prompts incl. branded	Log user-agent breakdown, 30d	Search-class agents blocked or 403-heavy	RC-A	Unblock
FM-1 on unbranded only, partial branded presence	Entity audit: Organization markup, sameAs, logo	Incomplete or absent	RC-B	Establish
FM-1 on category prompts; you are cited in other prompts	Source-dominance analysis on failing prompts	Third-party domains fill the space	RC-C	Corroborate
FM-2	Can the cited page supply the sentence used?	No	RC-D	Expand
FM-2 that survives RC-D fix	Third-party corroboration check for your key claim	Nothing credible asserts it	RC-E	Corroborate
FM-3	Trace the claim to a specific source page	Traceable to a third party	RC-E, via RC-C	Correct
FM-3, untraceable	Compare against your own canonical pages	Your own pages are ambiguous	RC-B	Establish
FM-4	Cited-domain analysis on comparison prompts	Competitors dominate the citation field	RC-C	Corroborate
Not mentioned but cited	None - this is a standing condition	Always true	RC-E	Corroborate

The last row is the one teams skip because nothing looks wrong. You’re the evidence, the answer reflects your material, and your name is absent. The remedy isn’t technical.

---

The AI Visibility Gap Scorecard

Five dimensions, each scored 0–4, weighted to 100. Score them from evidence you collected, not from impression.

1. Access integrity (weight 25). Are the search-class crawlers actually fetching?

• 0 - search-class agents blocked or erroring at the CDN/WAF
• 1 - reachable but heavily throttled or intermittently failing
• 2 - healthy for some operators, degraded for others
• 3 - all major search-class agents fetching key templates cleanly
• 4 - clean fetches, low error rate, content negotiation working

2. Entity clarity (weight 20). Can a system resolve what you are?

• 0 - no entity markup, no corroborating profiles
• 1 - markup present but sameAs empty or logo non-compliant
• 2 - markup present, sameAs sparse
• 3 - complete markup, consistent name and logo, profiles aligned
• 4 - plus identifier properties populated and Merchant Center / Business Profile current

3. Presence (weight 20). Across the category-family prompt set specifically.

• 0 - below 10%
• 1 - 10–24%
• 2 - 25–44%
• 3 - 45–64%
• 4 - 65%+ with a stable time series

4. Claim integrity (weight 20). Of runs where you appear, how often is the claim yours?

• 0 - under 25% accurate
• 1 - 25–49%
• 2 - 50–69%
• 3 - 70–84%
• 4 - 85%+ with two-person agreement on the sample

5. Corroboration depth (weight 15). How many independent, credible, non-owned sources assert your key claims?

• 0 - none
• 1 - one weak or stale source
• 2 - two, at least one outside your control
• 3 - three or more across distinct domain types
• 4 - plus one that’s cited in answers where you aren’t mentioned

The arithmetic. Score each dimension 0–4, multiply by its weight, divide by four, and sum. The weights total 100, so a perfect site scores 100. A worked case: access 2, entity 3, presence 4, claim integrity 1, corroboration 2 gives (2×25 + 3×20 + 4×20 + 1×20 + 2×15) ÷ 4 = 240 ÷ 4 = 60.

That 60 is hiding the only number that matters. A site scoring 60 on access and 0 on presence isn’t a “content investment” finding - it’s invisible for a reason no content programme addresses, and a single blended number would have concealed that completely.

How to use it, and when not to. The scorecard’s job is to make the shape of your problem legible. In the worked case above, a claim-integrity score of 1 against a presence score of 4 says the brand is everywhere and understood nowhere - an attribution problem, and explicitly not a reason to buy more content.

---

Prioritising: Fix Classes and the Arithmetic

Once routing is done, sort findings into five fix classes. The ordering isn’t arbitrary - it reflects how much of the failure each class can remove on its own.

Class	Meaning	Typical lead time	Depends on
Unblock	Access and crawl failures	Hours to days	Engineering, CDN config
Establish	Entity, identity, structured content	Days to weeks	Engineering, marketing ops
Correct	Specific factual errors on third-party pages	Weeks	Comms, PR, support
Corroborate	Build independent, credible assertion	One to two quarters	Comms, partnerships, community
Expand	New or deeper content for uncovered intent	Ongoing	Content, subject experts

Then score each item:

priority = (prompts affected) × (intent weight) × (fix confidence) ÷ effort

Intent weight comes from your own prompt stratification - commercial and comparison prompts should outrank definitional ones unless your business says otherwise. Fix confidence is your honest probability that the change causes the improvement; set it low for Corroborate, where causality is genuinely uncertain.

Effort is systematically underestimated, and the distortion is predictable:

Work	Common estimate	Typical reality
Unblock a WAF rule	”An afternoon”	Correct - often under an hour, if you can find the rule
Fix sameAs and entity markup	”A sprint”	Roughly right, if the profiles exist
Correct a fact on a directory you don’t own	”A support ticket”	Weeks to months, often unfalsifiable
Get cited by an authoritative third party	”One PR push”	A quarter, and not guaranteed
Publish content for an uncovered intent	”Two weeks”	Two weeks of writing; indexing and citation on a different clock

Unblock and Establish go first, always. They’re cheap, they’re fast, and they change the denominator for everything downstream. A team that unblocks its crawlers and fixes its entity markup in week one will re-derive more of its audit than a team that spends the quarter publishing into a broken pipeline.

---

What This Audit Cannot Tell You

The limitations aren’t a disclaimer. They determine which conclusions you’re entitled to draw, and several of the most commercially tempting claims in this field fail here.

You can’t infer retrieval weights. No platform publishes them. Every causal claim your audit makes is an inference from an observed correlation, supported by a discriminating test but never confirmed by the platform.

You can’t attribute a single answer to a single cause with confidence. A page’s absence may be rank, freshness, availability at retrieval time, or a filter you can’t observe. This is precisely why the routing table requires a discriminating test per row rather than a plausible narrative per finding.

You can’t measure absence from the long tail. Google’s own API documentation warns it doesn’t return all rows. Your prompt set is a sample of a size you can’t state. Never present a category presence rate as a market coverage figure.

You can’t separate AI traffic from direct traffic reliably, at all times. This deserves more attention than it gets. Cloudflare’s crawl-to-refer metric - HTML crawler requests divided by HTML requests whose Referer names the platform - is elegant, and Cloudflare itself warns it “may overstate the respective ratios” because native-app referrals carry no Referer header, with the magnitude of the error described as unclear (Cloudflare blog, 2025-07-01; the figures in that post are from June 2025, so cite the method and not the numbers).

Referral volume is a lagging and noisy proxy for influence. Being cited without being clicked is a normal and substantial outcome.

• The published llms.txt adoption numbers contradict each other badly. One vendor’s September 2026 scan, with published methodology and soft-404 controls, found 9.3% of the top 1,000 and 8.3% of the top 10,000 Tranco domains serving a confirmed file, while explicitly stating it’s “not proof that an AI platform uses the file” and that publishing one “does not establish search-ranking improvements, AI citations, or training permissions” (Rankability, 2026-09-18). Other figures circulating in 2026 range from roughly 4% to a claim about 17.9 million websites, which can’t be reconciled with the measured figures. The llms.txt proposal itself claims “thousands of sites” with no published methodology (llmstxt.org, v2).

If you publish llms.txt, treat it as a cheap, low-risk experiment for coding agents and developer tooling - which is where its own authors say it’s used - and don’t list it as an AI search fix in a business case.

---

Running It on a Cadence Without Fooling Yourself

Monthly, not daily. Daily runs on a 50-prompt set generate data you can’t interpret and invite you to read noise as trend. Monthly matches both platform volatility and the reporting cadence your stakeholders can absorb.

Store everything, permanently. The dated snapshot is the asset. When a stakeholder asks in six months whether the Q3 change helped, the only defensible answer comes from the stored run, not from a re-run.

Version your prompt set explicitly. Changing a prompt invalidates its time series. Keep the old series, mark the break, never splice across it.

Watch your instruments, not just your numbers. The most instructive recent example: from the GPT-6 rollout, ChatGPT links briefly arrived without their utm_source parameter and without a referrer, which caused analytics to file the visits as Direct. Adobe’s Cedric Huesler examined roughly 500 pages where ChatGPT drove more than 5% of visits and found ChatGPT down about 31,000 visits per day and Direct up about 26,000 per day on the same pages, while Search fell 7% - in line with everywhere else. Direct rose on two thirds of the affected pages. The tagging was restored around 15 October as a bug (Search Engine Roundtable, 14–15 October 2026).

Two lessons. First, if your baseline moved and you can’t find a content-side cause, check the measurement before you check the market. Second, “AI referral traffic” is a constructed category maintained by other people’s tagging decisions, and it can break for a week without anyone telling you.

Re-run the discrimination, not just the score. The routing table’s tests should be re-run each cycle. A brand that was blocked in month one and fixed in month two shouldn’t still be carrying an access penalty into month four because nobody re-checked.

---

The One-Paragraph Version

Build a prompt set from questions your buyers actually ask, stratified by intent and by family. Run it against a fixed locale at a fixed cadence, three times each, and store the full answer, the citations, and every source consulted. Score presence, claim integrity and citation share separately - never as one number. Classify every run into one of four failure modes on two axes: mentioned versus cited. Diagnose the mechanism with a discriminating test per mode, starting with the cheapest: are the crawlers even getting through? Fix access and entity before you write a single word. Then spend your quarter on corroboration, which is the slow root cause that content volume doesn’t touch. No part of this guarantees a ranking in any system, and nothing in the current documentation suggests it could - what it gives you is a defensible diagnosis and a defensible order of operations, which is worth considerably more than a visibility score.

---

Sources and References

Crawler and access control

• OpenAI, Overview of OpenAI Crawlers
• Perplexity, Perplexity Crawlers
• Anthropic, Does Anthropic crawl data from the web, and how can site owners block the crawler? - dated 2026-04-07

Platform eligibility and measurement

• Google Search Central, AI Features and Your Website - page last updated 2025-12-10 UTC,
• Bing / Microsoft, Introducing AI Performance in Bing Webmaster Tools Public Preview - 2026-02-10
• Bing / Microsoft, New AI Visibility Insights in Bing Webmaster Tools: Intents, Topics, Citation Share, Compare - 2026-06-16
• Bing / Microsoft, Introducing Topics Portfolio (Beta) in AI Performance - 2026-10-13
• Bing / Microsoft, Elevating the Role of Grounding on the AI Web - 2026-02-12. Cited only for Microsoft’s own attributed claim.

Retrieval behaviour and APIs

• OpenAI, Web search | OpenAI API

• Perplexity, Agent API Models - worked example created 2026-05-21,

• Perplexity, Documentation overview Crawl diagnostics

• Cloudflare, Analyze AI traffic

• Cloudflare, The crawl before the fall… of referrals - 2025-07-01. Method cited; underlying figures are from June 2025.

Entity and structured data

• Google Search Central, Organization (Organization) structured data - page last updated 2026-09-08 UTC
• Google Developers, Search Analytics: query | Search Console API - page last updated 2026-08-11 UTC

Emerging standards

• llms.txt proposal, The /llms.txt file, v2
• Rankability, llms.txt Adoption Research Report - 2026-09-18. Third-party vendor research with published methodology; cited as such.

Measurement reliability

• Search Engine Roundtable, OpenAI Drops UTM Parameters From ChatGPT Results - 2026-10-14, updated 2026-10-15. Secondary report.