No. 033
How to check whether AI engines cite you, without buying a tool
Provider documentation is enough to build a manual citation check. Five hits in 20 runs carry a 95% interval of 11% to 47%, which is our own arithmetic.
11%vs47%
Why are mentioned, cited and linked three different things?
To the providers, a citation is a source shown on screen. Microsoft’s Bing Webmaster Tools help: “A citation indicates that your content was visibly referenced or shown in an AI-generated answer. It does not represent traffic, clicks, or user engagement.” Google Search Console counts clicks separately: “Clicking a link to an external page in AI Mode counts as a click.”
A brand name in the answer text is a third thing. Semrush, a vendor selling an AI visibility toolkit, sorted 3,981 domain appearances from its own product across 115 prompts, 14 countries and four engines: 61.7% linked without the brand named, 13.2% cited and mentioned, 25.1% mentioned with no citation. The 61.7% is a share of all appearances, of which 74.9% carried a citation at all, so as a share of cited appearances it is about 82%, our division and not Semrush’s.
Whether naming and linking are connected is open. Seer Interactive, an agency reporting on its client portfolio, found a 53.1% citation rate for brands named in a response against 10.6% for the same brands when not named across 541,213 responses for 20 brands, and calls the ordering “strongly supported behavioral evidence, not proven architecture.”
So the sheet needs three columns: the name in the text, the source link, the URL it resolved to.
What does the method consist of?
Six decisions, each forced by something a provider published.
-
A frozen query set, realistic rather than flattering, because Google says AI Overviews are shown only when its systems judge them additive to classic Search, “and as such, often don’t trigger”. Queries that never trigger one give a column of zeroes about Google.
-
Search forced on. OpenAI says ChatGPT “may search the web automatically when your question would benefit from current information”, and the manual route is on the same page: “Select View all tools. Select Search.” No rate is published for the automatic case, so leaving it to chance mixes searched and unsearched answers. Forcing search is our inference.
-
One fresh conversation per run, because Perplexity says that inside a conversation “Perplexity will remember the context of your previous queries”. A second query in that thread is not a second observation.
-
Memory off wherever there is a switch. OpenAI documents a temporary chat, “Select Unpersonalized if you do not want to use them”, and warns there that “Sources may not show every factor that shaped a response”. Its release note of 22 April 2026 says “Fast answers do not reference your past chats or memory”. Perplexity says “Memory and previous searches are always off when using incognito mode”.
-
One mode throughout. Perplexity describes Pro Search as going “beyond standard search capabilities by conducting thorough research across multiple sources” and says “Free users can enjoy a limited number of Pro Searches per day”. An account that runs out mid-session changed instrument mid-session.
-
The verbatim answer, with the date and the engine. Do not ask the model whether it cited you: Google’s Gemini help says “Gemini can misrepresent how it works” and names “citing sources” among the functions it misdescribes.
Then pick a cadence and keep it: a change of interval and a change of result look identical.
What breaks a naive sample?
OpenAI warns on its own search page that “Search results and citations can be incomplete, outdated, or incorrect”, and the same page explains how two testers get different answers. ChatGPT search “typically rewrites your query into one or more targeted queries”, and “If memory is enabled, ChatGPT may use relevant saved memories when rewriting a search query.” Location reaches that stage too: “A VPN or network location may affect the approximate location inferred from your IP address.”
Google documents the mirror image. AI Mode personalisation “is available if you’re 18 or older and have history and personalized recommendations enabled”, and a reader who nominated your site sees it marked: “your content can be highlighted with a ‘preferred’ badge”. The surfaces differ: “AI Mode and AI Overviews may use different models and techniques, so the set of responses and links they show will vary”.
Then the display moved. On 15 October 2026 Google announced “continued and substantial increases to the presence of inline links” and a tested display that puts links “in a dedicated section at the top of a response”. Parts are rolling out, parts are tests, so a series straddling that date measures your site and Google’s rollout together.
Two things the documentation will not carry. One is the current memory default for ChatGPT, Gemini or Perplexity: OpenAI’s only statement of one is a 2025 release note scoped to the EEA, the UK, Switzerland, Norway, Iceland and Liechtenstein, Google’s is a launch statement of 13 August 2025, and Perplexity’s Memory page states none. The other is whether a signed-out session displays citations like a signed-in one: OpenAI documents only that “People who are not signed in can also use web search.”
How wide is the interval on 20 runs?
Run the frozen set 20 times on one engine and count the runs where your domain appeared as a source. These are 95% Wilson intervals on that count, computed by us and taken from no source. They assume each run is an independent draw with one underlying probability, which is what the two studies below break. Clopper-Pearson exact intervals differ by at most 3 points.
| Runs where your domain was cited | Point estimate | 95% Wilson interval, our arithmetic |
|---|---|---|
| 0 of 20 | 0% | 0% to 16% |
| 5 of 20 | 25% | 11% to 47% |
| 10 of 20 | 50% | 30% to 70% |
| 60 of 100 | 60% | 50% to 69% |
Read the 5-of-20 and 10-of-20 rows together: 11% to 47% against 30% to 70%, overlapping across 17 percentage points. Doubling the count did not produce two findings you can tell apart.
Those intervals are a floor. Ronald Sielinski’s preprint “Quantifying Uncertainty in AI Visibility” queried Perplexity Search, OpenAI SearchGPT and Google Gemini with 200 queries per topic on three consumer topics, daily for nine days from 3 to 11 February 2026. On SearchGPT a 9.5% citation share for tomsguide.com carries a bootstrap interval of about 5.5% to 12.5%, wide enough that its 3.5-point lead over runnersworld.com at about 6.0% sits inside the noise. It is a preprint rather than peer-reviewed work, collected through APIs rather than the consumer apps, and the author’s affiliation is IQRush, which sells AI search visibility measurement.
Two runs of one query do not return one list. Grossman and five co-authors, in a preprint that its arXiv comments field says was accepted at ACM SIGIR 2026, which we did not verify with ACM, ran a consistency test on 100 queries with two runs per configuration. With device and location fixed, URL-level Jaccard similarity between runs was 0.66 for AI Overviews, 0.46 for Gemini and 0.78 for classic results, on a Google-only benchmark.
What can the free dashboards see?
| Dashboard | What the unit is | What it excludes | The provider’s own stated limit |
|---|---|---|---|
| GA4 AI Assistant channel | sessions where the referrer “matches a list of AI Assistants" | "It excludes Google’s AI Overviews and AI Mode” | that matching list is not published |
| Search Console generative AI report | impressions: links “shown to a user in a generative AI feature on Google Search” | clicks, CTR, queries; “AI Overviews, AI Mode” are combined | ”the usual data limitations (1,000 row limitation, time period, etc) for the Search performance report also apply” |
| Bing AI Performance | citations: “which pages are cited” plus the “grounding queries associated with your content" | "individual AI answers, exact prompts, or why a specific page was referenced”; no ChatGPT surface | ”The data shown represents a sample of overall citation activity” |
| Microsoft Clarity Citations | citations, plus “sessions on your site originating from AI assistants" | "traditional search rankings, impressions, or click-through rates” | AI Overview traffic “cannot be reliably separated from standard search, as they share the same referrer” |
| Cloudflare AI Crawl Control | AI crawler requests, plus “Top domains sending AI-driven referral traffic” | a crawler request is neither a visit nor a citation | the referral view is “available for customers on a paid plan” |
Three of the units here are impressions, citations and crawler requests, and none of those is a reader clicking. Neither session-based view can isolate Google’s AI surfaces either: GA4 files AI Overviews and AI Mode under Organic Search, and Clarity’s FAQ names the shared referrer as the reason.
The referral column holds one more trap. The only referral parameter in the provider pages we read is OpenAI’s, and its page has gone from the live help centre, so the citable copy is an Internet Archive capture of 22 December 2025: “ChatGPT automatically includes the UTM parameter utm_source=chatgpt.com in referral URLs.” The referrer lists that circulate for this, such as one practitioner’s regex of chatgpt.com, perplexity.ai and claude.ai, are suggestions, not provider documentation.
What do the paid tools add, and what can they still not see?
Cloudflare’s answer engine optimisation tab, described on 6 August 2026, is the clearest published example of the architecture. It probes “the leading assistants (today, Anthropic’s Claude and OpenAI’s GPT) with likely customer prompts” and reports Citation Rate, Mention Rate and Share of Voice. Two sentences define the trade. On variance: “AI assistants rarely answer the same question the exact same way twice.” On whose prompts those are: “we run this panel once per category and reuse the baseline across all accounts in that domain.”
The repeats are what you buy. The prompts are a category panel rather than your query set, two assistants are not all of them, and it is early access. Cloudflare is also blunt about what the category cannot report: “You can’t look this up like a search ranking. There’s no impression count and no missed-click report.”
The three vendor studies below ran those repeats, each with its limit.
| Vendor study | What it measured | Result | Stated limit |
|---|---|---|---|
| SparkToro with Gumshoe, Nov to Dec 2025 | brand lists in 2,961 runs of 12 prompts | ”a <1 in 100 chance” of the same brand list twice | brands, not cited URLs; volunteers on default settings; Gumshoe sells AI tracking, “Conflict of interest? Well, maybe” |
| Sistrix, Dec 2025 to Apr 2026 | domain churn, 82,619 prompts, one weekly snapshot each | about 56% of AI Mode and 74% of ChatGPT Search domains new each week | a weekly snapshot cannot separate real change from run-to-run randomness, our reading and not the vendor’s |
| SE Ranking, before Oct 2025 | AI Mode repeats of 5,000 US local keywords | same URLs about 19%, same domains about 35% | AI Mode only, US local, about thirteen months old |
What the sheet settles, and what it does not
It settles whether your domain appeared as a visible source: one frozen query set, one engine, one mode, one location, a stated window, an interval attached. That is a real measurement and nobody is selling it to you.
It does not settle why: Bing says its own data does not show “why a specific page was referenced”, and a manual run sees less than Bing does. It does not settle whether a person read the citation, which Microsoft states for its own figures and no provider documentation we read gives for the three assistants, or whether a change you made caused a change on the sheet, because 25% and 50% on 20 runs each are not distinguishable. And it does not settle how you compare with the engine next door: Benjamin Tannenbaum, in a preprint by a measurement vendor’s founder, reports a mean cross-engine URL Jaccard of 0.0079 from 15 prompts on one day in June 2026, warning that “The prompt count is small”.
A number that carries its interval and its window can be checked. The version without them is what the dashboards and the tools hand you.
Method and corrections
Every page linked here was opened on 23 October 2026 quoted strings matched against the text served. The only arithmetic is ours: the Wilson intervals above, and 61.7 divided by 74.9.
Three figures that circulate for this are deliberately absent: Profound’s drift percentages (June to July 2025), Strauss and co-authors’ attribution figures (older models in the LMArena setting), and any required number of runs, because the only published thresholds are one preprint’s platform-specific results and one practitioner’s suggestion. If a quoted string changes, or we have one wrong, we will correct it and say so.