Claude

1. What Claude is

Claude is Anthropic’s conversational AI product. When web search is available and the question calls for current or specialized information, Claude can retrieve live web sources, synthesize an answer, and attach direct citations. That makes its search-enabled experience a retrieval-augmented generative engine, not a conventional list of search results. It is one surface in the broader field of generative engine optimization.

The name “Claude” can refer to a product, a model family, or an API integration. Those meanings overlap, but they are not interchangeable. Anthropic operates the models and services. Claude chat is the consumer interface. The Messages API gives developers a separately configurable web search tool.

Surface	Live retrieval	How searching works	Citation presentation
Standard Claude chat	Conditional	Claude searches when the prompt needs current or specialized information, provided web search is enabled	Direct source links appear when web results support the answer
Claude Research	Multi-step	Claude runs successive searches across the web and, when connected, across authorized work sources	Citations accompany the resulting research answer
Messages API web search	Developer-configured and conditional	The request supplies the server tool; Claude decides when to call it within configured limits	Structured search-result and citation blocks are returned with the response
Claude model without a retrieval tool	No live web access	The model answers from its available context and prior training	No web-search citations are generated

Anthropic’s current consumer documentation says Claude processes multiple sources and provides direct citations, source links, and relevant quotations when appropriate. It also distinguishes web search from Web Fetch: search discovers sources, while Web Fetch retrieves a page supplied by URL. Research supports sustained, iterative investigation instead of relying on a single search.

ChatGPT Search is also retrieval-augmented chat, while Perplexity AI uses retrieval more consistently by default. Those product classes are useful starting points, but citation density and search invocation still need to be measured with the same prompts over the same period.

2. How Claude web search works

Claude follows the general answer loop: it interprets a prompt, decides whether retrieval is useful, generates one or more queries, processes search results, writes an answer, and attaches citations to supporting passages. A developer can make the tool available, and a consumer can enable it, without forcing Claude to call it for every question.

The path can be summarized as:

prompt → search decision → query generation → result retrieval and filtering → answer synthesis → citation attachment

The distinction between tool availability and actual use matters. Anthropic’s API launch explanation says Claude may generate targeted queries and conduct progressive searches when earlier results suggest a better follow-up. On a stable, general question, it may answer without searching. An explicit request for current information makes a search more likely, but Anthropic does not publish a deterministic routing formula.

Documented behavior	Practical implication
Search is conditional	A page cannot be retrieved in a turn where Claude never invokes the tool
Research can build one search on another	Claude may find a source through a narrower follow-up query rather than through the user’s original wording
Web Fetch accepts a direct URL	User-supplied pages follow a different discovery path from open-ended search
API requests can limit search calls and filter domains	An experiment’s configuration changes its candidate source pool
Citations are attached to supporting text	Retrieval and final citation are related but distinct events

Anthropic has not disclosed the full consumer search index, its ranking formula, or the weights used to choose one source over another. A high rank in an external search engine can be diagnostic, but it does not prove that Claude found the source through that search engine. Clear, self-contained passages improve citability once Claude can access the page, but no page structure can guarantee retrieval or citation.

3. Crawlers and user-agents

Anthropic documents three web agents with separate purposes. The distinction lets a publisher opt out of potential training collection without automatically refusing all search-related access.

Agent	Official purpose	Trigger	Likely effect of blocking
ClaudeBot	Collects public web content that could contribute to model training	Background collection	Requests that Anthropic exclude future material from its training datasets; it does not control search visibility
Claude-SearchBot	Navigates the web to improve search-result relevance and accuracy	Search indexing and quality improvement	Prevents indexing for search optimization and may reduce visibility or accuracy in Claude search results
Claude-User	Accesses a site in response to an individual user’s request	A user asks Claude to retrieve relevant web content	Blocks that requested retrieval and may reduce visibility in user-directed web search

These definitions come from Anthropic’s site-owner crawler guidance. Anthropic says all three honor standard robots.txt directives, do not bypass CAPTCHAs or other access controls, and support the nonstandard Crawl-delay extension where appropriate. Rules must be set on each applicable subdomain.

IP blocking is not a durable substitute for agent-specific robots.txt rules. Anthropic warns that blocking by IP may prevent a bot from reading those rules and may not reliably express the desired policy. Its live bot prefix feed lists combined Anthropic bot ranges, not a permanent range for each individual agent. Fetch the feed when validating logs instead of copying its addresses into a static allowlist.

The dedicated ClaudeBot reference explains Anthropic-specific controls, while AI crawlers provides a general method for verifying user-agents, IP ranges, rendering, and server logs. Publishers should configure training, search-indexing, and user-requested access separately instead of applying an unexamined User-agent: * block.

4. Citation preferences

Claude’s documentation explains how citations appear, but not how sources are ranked. Anthropic confirms that web-sourced answers include citations and that the API returns source locations. It does not publish universal weights for authority, freshness, page format, or domain type.

Evidence layer	What the evidence supports	What it does not support
Official documentation	Search can return direct source links; API searches can be limited by domain and location; citations identify supporting source text	A public ranking formula or a guaranteed content pattern
Current observational studies	Source mixes, search rates, and overlap with external result sets in a dated prompt cohort	A causal ranking factor or a universal rate for all Claude users
Controlled older-model research	How named model snapshots behaved when candidate documents were already supplied	Current claude.ai retrieval, indexing, or consumer citation share

Two content properties remain useful even though Anthropic has not identified them as ranking weights. E-E-A-T asks whether a source offers credible authorship, provenance, and evidence. Citability asks whether Claude can extract a complete claim and attribute it correctly. A page can be strong on one dimension and weak on the other.

The controlled evidence should also be interpreted narrowly. Wan, Wallace, and Klein found that Claude Instant and four other older models gave substantial weight to topical relevance when choosing between conflicting pages already placed in context. The GEO Wiki paper review explains the study’s limits, and the ACL 2024 paper provides the original method. It did not test 2026 Claude web search or prove that a query-shaped heading causes retrieval.

A source citation and a brand mention are different outcomes. Citation versus mention separates attributed evidence from an uncited appearance in generated prose. Track both, because a page may inform an answer without receiving a visible link, and a brand may be mentioned without its own page being retrieved.

5. API and integration

The Messages API exposes Anthropic’s web search as a server tool. This compact request uses the newest documented version as of August 28, 2026, caps the number of searches, restricts eligible domains, and provides approximate location context:

{
  "model": "YOUR_CURRENT_SUPPORTED_MODEL",
  "max_tokens": 1200,
  "messages": [
    {
      "role": "user",
      "content": "Compare the current evidence for this question."
    }
  ],
  "tools": [
    {
      "type": "web_search_20260318",
      "name": "web_search",
      "max_uses": 4,
      "allowed_domains": ["example.org", "docs.example.com/research"],
      "user_location": {
        "type": "approximate",
        "country": "US",
        "timezone": "America/New_York"
      }
    }
  ]
}

Use either allowed_domains or blocked_domains, never both. Anthropic’s web search tool documentation lists three active versions: web_search_20250305 for basic search, web_search_20260209 for dynamic filtering, and web_search_20260318 for response-inclusion control. Each version provides a different capability; older versions remain active when a newer one appears.

Response element	What it reveals
server_tool_use	The query Claude sent and the fact that a search call occurred
web_search_tool_result	The set of returned result blocks, including title, URL, page age, and encrypted content
web_search_result_location	The URL, title, encrypted index, and cited text associated with an answer span
cited_text	Up to 150 characters of source text supporting that answer span
response_inclusion	On the newest version, whether completed nested search blocks remain in the returned response

Web-search citations are always enabled. That differs from developer-supplied search_result blocks, where citations are off by default and must be enabled for the entire request. The broader citations documentation also covers grounding in uploaded documents, a separate way to supply sources.

As of the review date, Anthropic charges $10 per 1,000 web searches plus token costs; failed searches are not billed. Newer versions use code execution for dynamic filtering by default, which changes Zero Data Retention eligibility. Anthropic’s server-tools documentation says a newer search version can use a ZDR-eligible direct configuration by setting allowed_callers to ["direct"]. Model compatibility, hosted-platform support, organization settings, and retention eligibility can change, so record the applicable model, hosting platform, organization settings, and retention status for every experiment.

API output is valuable for AI citation tracking because it exposes queries and citation objects. It is still a proxy for consumer Claude. A different model, location, domain rule, tool version, or account setting can produce a different candidate set and answer.

6. History and timeline

Claude’s search and citation features arrived in several stages. The dates below describe launches or documentation changes, not a single permanent feature set.

Date	Change	GEO consequence
January 23, 2025	Anthropic’s release notes record the API Citations launch for developer-provided documents	Developers gained passage-level attribution before live API web search launched
March 20, 2025	Web search launched as a US preview for paid users; a May 27 update announced global availability on all Claude plans	Live public-web retrieval became part of consumer Claude
April 15, 2025	Research launched with iterative searches across the web and connected work context	Citation measurement expanded from single answers to multi-step research tasks
May 1, 2025	Integrations and advanced Research expanded the range of connected internal and external sources	Public-web citations could appear beside authorized private-source evidence
May 7, 2025	API web search launched with progressive search, domain controls, max_uses, and source citations	Developers gained a configurable surface for repeated tests
February and March 2026	Anthropic documented dynamic filtering and response-inclusion versions of the web search tool	Experiments now needed to record the tool version
April 7, 2026	Anthropic documented the current three-agent crawler taxonomy	Publishers could control training collection, search indexing, and user-requested retrieval separately

Launch model names, plan restrictions, and geographic limits in the 2025 announcements are historical facts. Consult the live documentation for current compatibility because Anthropic revises supported models and deployment environments more quickly than a static article can.

7. Measured citation behavior

No single study establishes Claude’s ranking logic. The most useful results state the model or product surface, prompt set, search setting, collection date, and data-access limits.

Evidence	Setup	Result	Boundary
Blyskal and Singh / Profound, July 2026	Web search enabled; prompts ranged from current product recommendations to basic explainers	Claude searched on 36.6% of tested prompts; across roughly 35,000 URLs tied to 400 queries, 79.2% of cited URLs also appeared in Brave’s top ten	Commercial observational study; the exact denominator used for the reported routing rate, dates, full sampling procedure, and raw data were not public
C-SEO Bench, 2025	Claude 3.5 Haiku ranked citations from documents already present in context	No tested white-hat rewrite was significantly effective for that model in the reported settings	It did not test live retrieval, the current consumer product, or search candidate generation
Wan et al., ACL 2024	Claude Instant v1 chose between conflicting candidate pages already supplied to the model	Topical relevance had a stronger effect than several stylistic credibility cues	The model snapshot predates current Claude versions, and retrieval had already occurred

The 2026 Claude analysis is useful because it reports both search invocation and result overlap. Its 36.6% search rate describes the study’s mixed prompt set, not Claude in general. The 79.2% Brave overlap makes Brave a plausible diagnostic comparison in that cohort, but the authors explicitly say it does not prove a direct causal dependency.

C-SEO Bench offers a different warning. Its NeurIPS 2025 experiment found no statistically significant improvement from the tested rewrites for Claude 3.5 Haiku, while context position was more consistently influential across the broader benchmark. Because the documents were already retrieved, the result weighs against claims that those rewrites work universally, but not against retrieval optimization in general.

Together, these studies support careful measurement but do not reveal a ranking formula. Record whether Claude searched, which queries it generated, which pages appeared in results, which passages were cited, and whether a brand was mentioned. Repeat the same prompt because Claude’s search decision and the number of generated queries can vary. Preserve the full configuration so a later change in model or tool version is not mistaken for a content effect.

8. Optimizing for Claude

Claude-specific optimization requires both reliable access and repeated observation. Each action below has a different evidence basis.

Tactic	Why it matters for Claude	Evidence level	Governing reference
Configure ClaudeBot, Claude-SearchBot, and Claude-User separately	Publishers can set training preferences independently of search indexing and requested retrieval	Official documentation	ClaudeBot
Make the public answer accessible in rendered HTML	Search and user-requested fetchers need to retrieve the relevant text	This is an official access mechanism, although its effect in a given implementation still requires testing.	AI crawlers
Write a direct, self-contained answer near the relevant heading	A complete passage is easier to quote and attribute after retrieval	This is a general citation principle, not a published Claude ranking weight.	Citability
Name the author, source, date, and original evidence	Readers can verify provenance, and Claude can preserve concrete attribution	This is an editorial trust practice, not a disclosed Claude ranking weight.	E-E-A-T
Cover current, comparative, and specialized questions your audience actually asks	Those question classes are more likely to require live retrieval than stable explainers	Official conditional-search behavior plus cohort-specific observation	Generative engine optimization
Keep claims and dates current	Claude’s search queries often seek recent evidence, and stale claims weaken the resulting answer	Observational and prompt-dependent	Generative engine optimization
Measure search, retrieval, citation, and mention as separate events	Visibility can be lost at any stage, and each stage requires a different diagnosis	Methodological	AI citation tracking

No official source says that llms.txt, Schema.org markup, a fixed word count, or one rewrite guarantees a Claude citation. Structured metadata can support ordinary discovery and interpretation, but it should be tested as one variable rather than presented as a direct ranking control. The older controlled studies show that a plausible content tactic can have no reliable effect in a specific model and context.

Run tests with a stable prompt set and record the date, location, surface, model, search setting, and tool version. For consumer Claude, note whether a search occurred at all. For the API, retain tool-use and citation objects, but do not merge API and consumer results into one share-of-voice number.

9. Why Claude matters for GEO

Claude combines three properties that affect measurement: search happens conditionally, citations appear when retrieved sources support an answer, and Anthropic provides separate controls for training collection, search indexing, and user-requested fetches. Publishers can therefore manage access and measure outcomes at several distinct stages.

Claude trait	Observable metric	Common interpretation error
Web search is enabled but conditional	Share of prompts that produce a search call	Assuming every eligible prompt searched the web
Search can generate several queries	Number of queries generated and consistency across repeated prompts	Optimizing only for the user’s literal wording
Retrieval precedes citation selection	Retrieved URLs compared with cited URLs	Treating every retrieved page as a citation
Crawler roles are separate	Verified visits by each user-agent	Treating ClaudeBot access as a requirement for current citations
API controls are explicit	Results by tool version, model, domain rule, and location	Treating an API run as the consumer ranking system

Compared with ChatGPT Search, Claude may choose a different retrieval path and source set for the same question. Compared with Perplexity, it may search less consistently because ordinary chat can answer without retrieval. Absolute claims about which platform cites more are weak unless they use the same prompts, model period, location, and search configuration.

Instead of guessing an undisclosed ranking formula, keep the relevant access paths open, publish verifiable passages that answer real questions, and measure each stage from search invocation through visible citation. That approach remains useful even as Claude’s models, tool versions, and source mix change.