Technical Setup signal · 15% of your score
Can AI actually read your site?
This is the one pass-or-fail signal. Before an AI engine can quote you, its crawler has to reach your pages and be allowed to read them. Plenty of good sites quietly fail here and never find out.
Why it matters
ChatGPT, Claude, and Perplexity read the web through their own crawlers (GPTBot, OAI-SearchBot, ClaudeBot, PerplexityBot), and Google's AI Overviews run off normal Googlebot indexing. A single line in your robots.txt, a stray noindex tag, or a JavaScript-only page can make your best content invisible to all of them. No amount of great writing matters if the crawler hits a wall first.
What the Tracker checks
- Whether your robots.txt blocks GPTBot, OAI-SearchBot, ClaudeBot, or PerplexityBot
- noindex tags or canonical issues on pages you actually want cited
- Whether the page renders real content without JavaScript (AI crawlers don't run it)
- Structured data: Article, FAQPage, HowTo, Organization, Person, and BreadcrumbList schema
How it's scored
Every site gets a 0 to 10 on this signal. Here's what each band looks like.
Strong
8–10Fully crawlable and indexable, AI crawlers allowed, clean canonicals, with valid schema as a bonus on top.
Mixed
5–7Reachable and indexable, but thin schema beyond CMS defaults, or one minor robots/canonical wrinkle.
Weak
0–4A real access problem: robots.txt blocks an AI crawler, or content pages are set to noindex and can't be read at all.
How to improve it
- 1Allow GPTBot, OAI-SearchBot, ClaudeBot, and PerplexityBot in robots.txt
- 2Remove stray noindex tags from content pages you want surfaced
- 3Make sure pages serve real HTML content, not an empty JavaScript shell
- 4Add Organization and Person schema so engines know what each page is
The Tracker doesn't just score this, it drafts the exact pages to close the gap, built in the structure engines reward. Run your site through all six signals and get the one highest-impact fix to start with.
Questions, answered
Which AI crawlers should I allow?
At a minimum GPTBot and OAI-SearchBot (ChatGPT), ClaudeBot (Claude), and PerplexityBot (Perplexity). Google's AI Overviews use normal Googlebot, so staying indexable covers those.
Do I need schema markup to get cited by AI?
No. Structured data helps an engine understand what a page is, but it isn't required. A fully crawlable, well-written page can be cited with no schema at all. Crawlability is the load-bearing factor; schema is a bonus.
Does blocking Google-Extended hurt my AI visibility?
Barely. Google-Extended only controls Gemini-app and Vertex grounding and training, not whether a page shows up in Google's AI search features. Those run off standard Googlebot indexing.
The other five signals
See where you stand on all six.
One scan scores your site on every signal, shows the pages an engine reads and skips, and hands you the fix worth making first.
$39/mo, cancel anytime