Technical GEO
Audit crawler access, rendering, structured data, and discovery.
- Best for
- Engineers responsible for site infrastructure, platforms, or technical SEO.
- Before you start
- GEO 101 Step 1 (definition).
- By the end
- You can audit and strengthen a site's crawler-facing delivery from access through rendering.
- The AI crawler landscape
Classify AI crawlers by purpose before setting access rules: model training, search retrieval, and user-triggered fetching require different decisions.
Time: 12 min
- Crawler profiles: GPTBot, ClaudeBot, and PerplexityBot
Compare GPTBot, ClaudeBot, and PerplexityBot by User-Agent, documented purpose, observed behavior, and opt-out mechanism. Let your own logs set the order.
Time: 12 min
- robots.txt and access control
Learn how robots.txt scopes rules by User-Agent, resolves competing paths, and differs from enforceable access control before changing crawler policy.
Time: 12 min
- llms.txt
Treat llms.txt as an emerging content-navigation convention, not an access-control file or a confirmed citation signal. Review the evidence before deploying it.
Time: 12 min
- Schema.org and JSON-LD
Use Schema.org to state entities and relationships explicitly, while keeping its role separate from ranking or citation claims. Compare serializations before implementing.
Time: 15 min
- Rendering: SSR vs CSR for AI crawlers
Check the HTML each crawler receives before choosing SSR, SSG, CSR, or edge rendering. Treat Core Web Vitals as a separate, platform-dependent concern.
Time: 12 min
- Sitemap & IndexNow
Sitemaps and IndexNow influence some AI surfaces through search indexes rather than direct ingestion. Map those dependencies before deciding what to submit.
Time: 12 min
- Hands-on: a crawler-access audit
Apply the preceding decisions to your own domain, then rank findings by dependency so access and rendering problems are addressed before downstream refinements.
Time: 30 min