What's Actually Happening With AI Search, and What You Can Honestly Measure

No Comments
What's actually happening with ai search, and what you can honestly measure

TL;DR

You cannot yet prove an AI answer made someone buy, but you can measure whether you are in the conversation, whether your share is growing, and whether what is being said is true. Those are 3 different questions, and most of the confusion about AI visibility comes from asking them as one.

  • Attribution from a mention to a sale: mostly not answerable today. Say so.
  • Visibility: measurable, as a share, on a fixed list of questions, repeated over days.
  • Accuracy: measurable with the same run and a different scoring column. In regulated businesses, I think it is the bigger risk.
  • New since I first drafted this: Google now offers every site a report on its AI impressions, and Bing added a topic view to its AI report.
  • Check first that AI crawlers can reach your pages. In my September 2026 test, 40 of 184 top sites did not serve real content to a client identifying as GPTBot.

The short version

For 20 years, search worked like a card catalog. You typed a few words, you got a list of doors, you picked one and walked through it. Everybody measured the same thing: how many people walked through your door.

Now people often ask a question in a full sentence and get an answer back. The list is still there, but on many searches it is no longer the first thing on the page. The answer is assembled from a lot of sources at once, and if it is good enough, many people go nowhere. Pew Research, on the browsing of 900 U.S. adults in March 2025, found people clicked a traditional result on 8% of visits with an AI summary, against 15% without one, and clicked a link inside the summary on 1%.

So the visit often disappears, and the influence does not. That is the whole problem in one sentence: you can still be the reason someone chose you, and there may be no click that proves it.

Why everybody is confused

Because 3 different questions are being called the same thing.

The questionStatus todayWhat to use
Did the AI mentioning us cause a sale?Mostly not answerableReferral clicks where they exist, and asking customers. Label anything else an estimate
Are we being mentioned at all, and more or less than before?Answerable, with disciplineFirst-party reports, plus a fixed list of questions, as a share
Is what it says about us correct?AnswerableThe same run, with a column for correct, incomplete or wrong
3 cards: attribution, did it cause a sale, mostly not answerable today; visibility, are we in the answer, answerable with discipline; accuracy, is what it says true, answerable. Below: first check whether ai crawlers can fetch your pages.
Measuring AI visibility is 3 questions, not 1, and the access check comes before all of them.

When someone asks how to measure AI visibility, they are usually asking all 3 at once. That is why every answer they get sounds evasive: it is one mushy answer to 3 separate questions. Split them up and 2 of the 3 become ordinary work.

On the first one, be precise. A referral click from an assistant can be followed, and you should look at them. But most influence leaves no click, so treat any single attribution number as an estimate and ask how it was built. The October 2026 analysis from Kevin Indig is a good warning: he checked 12 GA4 sites and found much of the growth in direct traffic was bots. Across 632 tests on 4 brands, one site showed a correlation of 0.72 between AI share of voice and direct traffic. It did not hold up when he checked it further, and he found no reliable link. He says an effect could still exist and that he could not isolate it. The positive case he cites, Ramp, looked at direct visits to long blog URLs nobody would type, compared them with organic visitors, and confirmed it with self-reported attribution.

The counting problem

Counting is the wrong unit. Ask more questions, get more mentions. The number goes up and nothing has improved. It is a vanity metric with extra steps.

What matters is share: out of everyone the answer named, how often was it you. And it has to be measured against the same fixed list of questions every time. Change the questions and the trend means nothing, because you are comparing 2 different things and calling it progress.

Repetition matters too. One person on r/Agentic_SEO tracked about 600 searches on a Saturday, Monday and Tuesday in September 2026. Of 550 with an AI Overview on all 3 days, only 13 cited exactly the same sites. That is one person's report, collected during a spam update, with Saturday gathered slightly differently, so read it as a warning and not a rate. A single run tells you very little.

The check that comes first

An AI can only tell people about your pages if it can read them. Many sites refuse those bots, and I suspect it is often by accident: a security rule or a rate limit doing its job a bit too well. And here is the trap: the visibility tool keeps showing you numbers anyway, because it is reading the answer, not checking the door. So the dashboard can look healthy while the site is out of reach.

Check the door first. It takes about 10 minutes: request your own pages the way each AI crawler would, from outside your own network, and look at what comes back. When I did this on 400 of the most visited sites, 184 qualified for comparison, and 40 of those did not serve real content to a client identifying as GPTBot. 20 of the 107 whose robots.txt never blocks GPTBot denied it at the edge anyway. A request that only claims to be a crawler cannot tell you what an IP-checking firewall does with the real one, so treat it as a first check. The commands are in my monitoring guide.

What the engines now show you

When I first drafted this in early August 2026, Google's AI report was still rolling out. It reached all sites on August 31, and Bing added a topic view on October 13. Any honest report should start here, because these numbers come from the engines themselves.

ReportWhat it showsWhat it does not
Google: Generative AI performance reportImpressions in AI Overviews and AI Mode, by page, country, device and date. For all sites since August 31, 2026Clicks and click rate are not in that report. No query list. It usually appears only with enough impressions
Bing: AI Performance and Topics PortfolioPages cited, the grounding queries behind them, and since October 13, 2026 a topic view with the queries where others appear and you do notTopics Portfolio is a beta, figures are approximate, and it needs enough data for your site
ChatGPT, Claude, PerplexityI did not find an equivalent report open to any site owner. Perplexity runs a publisher program I have not read in detailSo a fixed list of questions is still how you read these, and that number is your sample, not theirs

Read October 31, 2026. Google also has a Search generative AI control if you want to manage inclusion. It is a separate choice from measuring.

Use these to check third-party trackers. If a tracker and Google's own report disagree about direction on the same pages, believe neither until you know why.

The right way to think about it

Stop thinking about it as advertising. Think about it as word of mouth.

You cannot put a tracking code on a recommendation. Nobody has ever been able to. But companies have measured reputation for decades: are people talking about us, how much compared to the competition, and are they getting the facts right.

That is exactly the shape of this problem. It is not a performance marketing problem wearing a new hat. Once you say that out loud, the argument tends to stop. Nobody demands click attribution from a brand survey, and nobody should demand it here either, not yet.

What honest reporting looks like

  • First-party numbers first, from Google and Bing, for your own site
  • Share, not counts, measured on a fixed list of questions, per language
  • Repeat runs on different days, reported as a rate, with a failed run marked as not measured and never as not visible
  • Accuracy: is what is being said correct, incomplete or wrong
  • A control group: a set of questions you are not working on, so when everything moves at once you know it was the model that changed, not you
  • A trend, never a single number

And when someone needs a figure for a budget conversation: price the share you hold the way you would price the equivalent advertising. Label it as an analogue, because it is one. It is a number a finance team can read without anyone pretending it is attribution.

The part people get backwards

Everyone is trying to rank higher. In regulated industries, being described incorrectly is, in my view, the bigger exposure.

Wrong pricing, wrong coverage, wrong description of what you actually do, repeated confidently to someone making a decision about their health or their money. That is not a marketing problem. It is a legal and trust problem that happens to live in a marketing channel, and it is measurable with the same run of questions and a different scoring column.

Where this leaves you

You cannot prove an AI answer made someone buy. You can prove whether you are in the conversation, whether your share of it is growing, and whether what is being said is true.

That is less than the industry is promising and considerably more than most companies have. Start there. The how-to is in monitoring AI search visibility, the checks are in the AI readiness checklist, and everything I have measured on this is collected in the AI readiness hub.

FAQ

Can I prove an AI answer made someone buy?

Not for most businesses, not today. A referral click can be followed, but most influence leaves no click. The strongest case I have seen cited combined analytics with asking customers where they heard about you.

What does Google's Generative AI report show?

Impressions for AI Overviews and AI Mode, by page, country, device and date. It has been open to all sites since August 31, 2026. It does not give clicks or a click rate in that report, so it answers whether links to you were shown, not what that did.

Why share and not mentions?

Ask more questions and you get more mentions, so the number rises with nothing improved. Share, on a fixed list of questions, is what stays comparable from one month to the next.

How many times should I run each question?

More than once, on different days. One person's test of about 600 searches on r/Agentic_SEO found only 13 of 550 AI Overviews cited the same sites on all 3 days. Report a share across runs, never one answer.

Is direct traffic a sign of AI influence?

Not on its own. Kevin Indig checked 12 GA4 sites and found much of the growth in direct traffic was bots: total direct rose 140.9% from January to September 2026 while engaged direct changed by minus 1.0% in monthly counts. If you look at direct at all, look at engaged sessions.

Where is the practical how-to?

In my guide to monitoring AI search visibility, which has the commands, the routine and the limits. This page is the plain version of the thinking behind it.

Sources I read

Claude Vincent is a technical SEO consultant focused on crawlability, rendering, and AI-search visibility. He writes the field guides and case studies at SEO ProCheck, with a bias toward the durable, unglamorous work that decides whether search engines and AI answer engines can actually read and cite a site.

    About SEO ProCheck

    Technical SEO consulting and GEO strategy with 20 years of enterprise experience. Case studies, resources, and tools for search and AI visibility.

    Subscribe to our newsletter!

    More from our blog