Vector Index Presence

Element Code: TE-018

What vector index presence means

Assistants like ChatGPT, Perplexity, Copilot, and Gemini do not answer live questions from model weights alone. They run retrieval augmented generation: the query goes to a retrieval layer, that layer pulls candidate documents and chunks from search indexes and embedding based stores, and the model writes its answer from what came back, usually with citations. The embedding side is where the "vector" in this check's name comes from: text gets converted into numerical vectors so the system can match by meaning rather than exact keywords.

Vector index presence is the GEO equivalent of being indexed in Google. It asks whether your content ever made it into those retrieval layers. Concretely, each ecosystem has its own supply chain: ChatGPT's search feature draws on Bing's index plus OpenAI's own crawling, Perplexity maintains its own index via PerplexityBot, Copilot sits on Bing, and Google's AI Overviews and Gemini pull from Google's index. Absent from the supply chain means absent from the answer, every single time.

Why it matters

Every other GEO effort is downstream of this one. You can write perfectly structured, quotable, entity rich content, and none of it matters if the crawler that feeds the retrieval layer got a 403 at your front door. This is the classic indexation lesson replaying itself: in 2010 the silent killer was a stray noindex, today it is a blanket AI bot block someone added in 2023 and never revisited.

And plenty of sites did exactly that. When AI crawlers first showed up, blocking them all was a defensible default. Since then, the same user agents that feed training data have been joined by separate agents that feed live search retrieval and citations. Blocking the second kind does not protect your content from training; it just removes you from answers that would have cited and linked you. If AI visibility is a goal, that trade needs a deliberate decision, not an inherited robots.txt line nobody remembers writing.

Know your crawlers

The single most useful thing you can internalize here is that "AI bots" is not one thing. Training crawlers, search index crawlers, and user triggered fetchers are separate agents with separate robots.txt tokens, and the right policy often differs per type. This reference grid covers the ones that matter most:

How to check your presence

1. Grep your server logs. Search the last 30 days for the user agents above. Zero hits from OAI-SearchBot and PerplexityBot on a site that wants AI visibility is the red flag. Verify suspicious hits against the operators' published IP ranges, since these UAs are widely spoofed by scrapers.
2. Audit robots.txt and your WAF. Read what you actually serve at /robots.txt, then check CDN bot management settings. Cloudflare, for one, ships a toggle that blocks AI crawlers wholesale, and it is easy to enable without realizing what it covers.
3. Run retrieval probes. Ask ChatGPT with search and Perplexity questions that only your page answers well, including asking Perplexity to summarize a specific URL. Getting retrieved and cited is the ground truth this check cares about.
4. Confirm Bing indexation. Use Bing Webmaster Tools and a site: query. Sites obsess over Google and never once look at Bing, then wonder why Copilot ignores them.
5. Fetch your page as a bot. Run curl -A "OAI-SearchBot" https://yoursite.com/page/ and confirm you get 200 with real content in the raw HTML. If the substance only appears after JavaScript executes, most AI crawlers never see it, because they generally do not render JS the way Googlebot does.

How to fix gaps

First, set policy deliberately: decide per bot type whether you allow training crawlers, and separately whether you allow search and retrieval agents. Second, remove accidental blocks in robots.txt, WAF rules, and rate limiters. Third, make the served HTML self sufficient: server side rendering or prerendering for anything you want retrieved. Fourth, structure for chunking: descriptive headings, and sections that open with a direct answer so a chunk lifted out of context still makes sense on its own. Finally, keep Bing healthy with Bing Webmaster Tools and IndexNow, and put a monthly log check on the calendar so regressions get caught in weeks, not quarters.

What good looks like

Logs show regular, verified visits from the retrieval agents you have chosen to allow. Raw HTML fetched without JavaScript contains the full substance of each page. Bing has the site cleanly indexed. And when you probe assistants with questions your content answers, your domain shows up in the citations at a rate you track over time. That is presence. Everything else in GEO builds on top of it.

FAQ