ChatGPT Search Optimization: The Complete Technical, Content, and Brand Visibility Guide

Most advice about ranking inside ChatGPT is invented. Not deliberately - the writers usually believe it - but invented nonetheless. They list factors like “llms.txt adoption,” “schema markup depth,” and “brand mention frequency” as though they sat down with OpenAI and got a list. No such list exists. OpenAI has never published one, and its own help centre goes out of its way to say so: ChatGPT “ranks search results using multiple factors intended to help users find relevant, reliable information. Placement is not guaranteed.”

I’d start with the things you can control: crawler access, what your server logs show, how clearly your content supports its claims, and whether a machine can identify your business. Placement isn’t guaranteed, and the ranking factors aren’t disclosed. Chasing a particular position means guessing.

---

What OpenAI actually publishes

Before optimising for a system, it helps to know which parts of it are knowable.

Fully documented, safe to build on:

• Four named crawlers, their exact user-agent strings, and the stated purpose of each
• Four separate published IP-range JSON endpoints
• The two-part eligibility requirement (robots.txt permission and network access)
• A ~24-hour propagation window for robots.txt changes
• The distinction between training, search, ads validation, and user-initiated fetching
• The shape of a web-search response: search, open_page and find_in_page actions, plus url_citation annotations and a separate sources list

---

The four bots, and why the two you already know about are the wrong two

OpenAI’s crawler page defines four robots. The third-party SEO ecosystem has mostly absorbed two of them and gets the important distinction wrong.

OAI-SearchBot - the only one that governs search inclusion

From OpenAI’s crawler documentation: “OAI-SearchBot is for search. OAI-SearchBot is used to surface websites in search results in ChatGPT’s search features.”

The consequence is stated just as plainly: “Sites that are opted out of OAI-SearchBot will not be shown in ChatGPT search answers, though can still appear as navigational links.”

Note that second clause. Blocking this bot removes you from answers. It doesn’t necessarily remove you from the interface entirely - navigational mentions may persist. That’s a narrower penalty than total deindexing, which means blocking OAI-SearchBot is a worse trade than it first appears.

The documented user-agent string is:

Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36
(KHTML, like Gecko) Chrome/131.0.0.0 Safari/537.36; compatible;
OAI-SearchBot/1.4; +https://openai.com/searchbot

OpenAI also publishes a distinct variant used when fetching robots.txt itself, with an added robots.txt; marker. It explains why: “The marker helps site owners more easily distinguish requests for the robots.txt file from requests for other resources, especially when their logs do not include paths.”

That’s a genuinely useful operational detail and almost nobody uses it. If your log pipeline drops paths, you can still segment crawler traffic into “policy check” versus “content fetch” without any other changes.

Published IPs: https://openai.com/searchbot.json. I requested this endpoint on 2026-10-19; it returned a JSON document with "creationTime": "2026-01-02T11:00:00.000000" and a list of IPv4 prefixes. Cite the endpoint, never inline the list - it changes, and the file tells you when.

GPTBot - training, and nothing else

“GPTBot is used to make our generative AI foundation models more useful and safe. It is used to crawl content that may be used in training our generative AI foundation models. Disallowing GPTBot indicates a site’s content should not be used in training generative AI foundation models.”

User agent: GPTBot/1.4; +https://openai.com/gptbot. IPs: https://openai.com/gptbot.json (I observed "creationTime": "2026-09-22T02:00:07.000000").

Two things worth flagging that most coverage omits.

First, the two IP lists aren’t the same list, but they aren’t disjoint either. searchbot.json carries 39 IPv4 prefixes and gptbot.json carries 18. Six of the eighteen GPTBot prefixes appear verbatim in the searchbot list: 4.227.36.0/25, 74.7.175.128/25, 74.7.228.0/25, 74.7.230.0/25, 74.7.241.128/25 and 74.7.244.0/25.

That overlap has a concrete consequence. If your intent is “allow search, block training,” a robots.txt rule expresses that cleanly - but a network-layer block of GPTBot can’t be made complete on IP alone, because on those six prefixes you’d also be blocking OAI-SearchBot traffic. The separation OpenAI describes is exact at the robots.txt layer and only approximate at the network layer. Choose your mechanism deliberately rather than assuming they’re interchangeable.

Second - and this is the strategic hinge of the whole channel - OpenAI states: “Each setting is independent of the others – for example, a webmaster can allow OAI-SearchBot in order to appear in search results while disallowing GPTBot.” And: “If your site has allowed both bots, we may use the results from just one crawl for both use cases to avoid duplicative crawling.”

That’s OpenAI explicitly designing for the “presence without training” configuration: the crawl economics still work for them, so the trade-off isn’t being asked to carry their cost.

ChatGPT-User - the one almost nobody models correctly

“ChatGPT-User is not used to determine whether content may appear in Search. Please use OAI-SearchBot in robots.txt for managing Search opt outs and automatic crawl.”

Three properties from the same page: it’s not automatic (“ChatGPT-User is not used for crawling the web in an automatic fashion”); it fires when a user or a Custom GPT/GPT Action requests a page; and “Because these actions are initiated by a user, robots.txt rules may not apply.”

Compare that last sentence to Perplexity’s crawler documentation, which is blunter: “Since a user requested the fetch, this fetcher generally ignores robots.txt rules.” Both vendors are telling you that user-initiated fetching is a different control plane. Perplexity states it as policy; OpenAI states it as a possibility (“may not apply”).

Practical consequence: a robots.txt rule against ChatGPT-User is at best advisory. If you need a hard block on user-triggered retrieval, robots.txt isn’t the mechanism. Rate limiting and authorisation are.

One measurement from the published IP files makes this concrete. chatgpt-user.json lists 234 IPv4 prefixes - six times the 39 in searchbot.json - and its creationTime is 2026-10-07, the most recent of the four files as of 2026-10-19 by a wide margin. What it does establish is that ChatGPT-User isn’t a minor side channel you can ignore on infrastructure grounds: it’s, by published range count, the largest of the four.

OAI-AdsBot - the bot missing from your template

“OAI-AdsBot is used to validate the safety of web pages submitted as ads on ChatGPT… OAI-AdsBot only visits pages submitted as ads, and the data collected by OAI-AdsBot is not used to train generative AI foundation models.”

User agent: OAI-AdsBot/1.0; +https://openai.com/adsbot. IPs: https://openai.com/adsbot.json - a list of just 2 prefixes as of my 2026-10-19 check, with a creationTime of 2026-05-12. It shares no prefix with any other OpenAI bot, which makes it the cleanest of the four to handle independently.

If you run ads on ChatGPT, this bot will hit your landing pages. If you block it, ad landing page validation fails. Its practical significance is narrow and specific - but it’s the one bot most often missing from the copy-pasted robots.txt template circulating in 2026, because most sites have no reason to think about ad validation at all.

Quick reference

Bot	Governs	robots.txt effect	IP prefixes	creationTime
OAI-SearchBot	ChatGPT search answers	Binding - opt-out removes you from answers	39	2026-01-02
GPTBot	Model training	Binding for training only; no effect on search	18 (6 shared with searchbot)	2026-09-22
ChatGPT-User	User-triggered page retrieval	”May not apply” - advisory at best	234	2026-10-07
OAI-AdsBot	Ad landing page validation	Binding for ad submission	2	2026-05-12

---

Eligibility is a two-key lock, and most sites only turn one

Here is a detail with more operational impact than any content tactic, and it’s one sentence in OpenAI’s help centre. Under the heading “Make your website available in search results”:

“To make a website eligible for inclusion, allow OAI-Searchbot to crawl the site and confirm that the website host or content delivery network allows traffic from OpenAI’s published searchbot IP addresses.”

Two conditions. Permissive robots.txt is necessary and not sufficient.

Your robots.txt can allow OpenAI while your WAF blocks its IP ranges. The file looks fine. The site still isn’t eligible. You won’t necessarily see an error, so check what happens at the network edge too.

If you take one action from the technical section of this guide, make it this: confirm your CDN and WAF allow searchbot.json, and re-confirm it after every infrastructure change. Don’t assume it inherited from the day you deployed.

Also plan for the lag. OpenAI: “it can take ~24 hours from a site’s robots.txt update for our systems to adjust.” If you block and unblock in the same afternoon, you have no idea what state you’re in.

---

The robots.txt decision tree

Start at the top and stop at the first branch that applies.

START
│
├─ Q1. Do you sell products or services where a wrong answer
│      costs a human real money - legal, medical, financial,
│      safety-critical, or anything where you would want a lawyer
│      to review a wrong claim about you?
│   │
│   ├─ YES → Block GPTBot. Allow OAI-SearchBot. Do not block
│   │        ChatGPT-User at the robots.txt layer (it is advisory
│   │        anyway); use rate limits for abuse control instead.
│   │        Rationale: you want presence, you do not want your
│   │        proprietary material in the training corpus.
│   │
│   └─ NO  → Q2
│
├─ Q2. Is your content the product - news, research, analysis,
│      tutorials, a documentation set - such that unpaid
│      substitution in AI answers destroys your revenue model?
│   │
│   ├─ YES → Block GPTBot. Allow OAI-SearchBot.
│   │        Then read the syndication section of this guide
│   │        before assuming robots.txt protects you. It does not.
│   │
│   └─ NO  → Q3
│
├─ Q3. Do you actively license or syndicate your content to AI
│      companies, or participate in a revenue-share programme?
│   │
│   ├─ YES → Block GPTBot anyway unless the licence explicitly
│   │        covers training. An organic crawl is not a licence
│   │        and does not pay you. Allow OAI-SearchBot.
│   │
│   └─ NO  → Q4
│
├─ Q4. Do you have a legal, contractual or ethical reason to keep
│      your content out of training corpora - client
│      confidentiality, embargoed material, paywalled research,
│      personal data, minors' content?
│   │
│   ├─ YES → Block GPTBot. Allow OAI-SearchBot. Resolve the
│   │        tension with the real control (auth, paywall,
│   │        rate limit), not with robots.txt.
│   │
│   └─ NO  → Q5
│
├─ Q5. Would you be materially harmed if your content appeared in
│      an AI answer without attribution, in a context you cannot
│      correct or monitor?
│   │
│   ├─ YES → Block GPTBot AND OAI-SearchBot. Accept that you are
│   │        choosing presence over exposure, and record the
│   │        decision with a date so you revisit it deliberately.
│   │        Do not block OAI-AdsBot unless you do not run ads.
│   │
│   └─ NO  → Q6
│
└─ Q6. Default: allow OAI-SearchBot and OAI-AdsBot.
         Decide GPTBot on your own risk tolerance - it has no
         bearing on search visibility either way.
         This is the configuration OpenAI's documentation is
         explicitly designed to support, since it states that a
         single crawl can serve both purposes.

The invariant, across every branch: OAI-AdsBot is a decision about advertising, not about AI visibility. Never fold it into a blanket AI policy. And never assume that any choice here affects ChatGPT-User, which is on its own control plane.

One implementation note that applies to every branch: enforce any GPTBot decision through robots.txt, not the network layer. The two IP lists share six prefixes, so an IP-based block is incomplete by construction - you’d be silently blocking OAI-SearchBot traffic on those ranges.

The two-key point from the previous section isn’t visible in this tree because it isn’t a robots.txt decision at all. Run the tree, then separately verify the network. Both, every time.

---

How a ChatGPT Search answer gets assembled

OpenAI doesn’t document its retrieval ranking. It does document the shape of the process, in two places that most SEO writing never opens.

Query rewriting happens before retrieval

From the ChatGPT search help centre:

“To provide relevant responses to your questions, ChatGPT search sometimes partners with other search providers. When it does, ChatGPT search typically rewrites your query into one or more targeted queries that it sends those providers.”

The documented example: a researcher asking “what’s the latest on the development of drugs that target CCR8 for cancer?” may produce an initial query of “CCR8 immunotherapy drug development 2025,” and after reviewing results, a follow-up “CHS-114 conference 2025.” The help centre links to Microsoft and Shopify privacy statements as the relevant provider documents.

Two implications. Your page may be retrieved for a query you never saw and wouldn’t have written. And the retrieval layer is partly someone else’s index - which means classic search visibility in those providers isn’t irrelevant to ChatGPT visibility, a point that undercuts a lot of “Google SEO is dead” rhetoric.

The full consulted-URL list is larger than the citation list

The web search API documentation is the most detailed public window OpenAI has published into answer construction, and one detail there has immediate measurement consequences:

“To view all URLs retrieved during a web search, use the sources field. Unlike inline citations, which show only the most relevant references, sources returns the complete list of URLs the model consulted when forming its response. The number of sources is often greater than the number of citations.”

If you’re auditing visibility by counting inline citations, you’re counting a filtered subset. A page can be consulted and never appear as a visible citation. This is worth stating plainly because visibility-tracking tools generally only see what is rendered.

The same documentation describes three web-search modes - non-reasoning search, agentic search with reasoning models, and deep research - and three action types: search, open_page, and find_in_page. The last two are documented as supported in reasoning models, which is the mechanism by which a system retrieves a specific page and then searches within it rather than reading it end to end.

Two more structural details from the same page:

• Domain filtering. Up to 100 allowed_domains or 100 blocked_domains, Responses API only, subdomains included.
• Search context is capped at 128k regardless of the model’s larger context window. And search_context_size “does not set an exact token count or guarantee a specific number of sources or citations.”

What ChatGPT Search surfaces that a crawler audit will not show you

This is where the peer-reviewed evidence is more informative than the documentation, because the documentation is silent.

A 2026 audit of 26,266 unique cited URLs across ChatGPT, Copilot, Gemini and Perplexity - published as arXiv:2605.23684, from Northwestern University and carrying a journal reference to AIES 2026 - contains two findings that matter directly for what you publish:

Non-text media is cited. Of the sources the researchers couldn’t scrape, “14.2% of the sources we could not scrape were URLs linking to PDF files.” Separately, they couldn’t fetch “16.2% of the overall sources cited by generative search engines because they appear to be in other modalities” - video and images, including images hosted on Facebook.

Read carefully: this is evidence that ChatGPT Search surfaces and cites PDFs, video and images. It’s not evidence of an indexing rule. What it does establish is that “we publish a website of HTML pages” isn’t the full surface area ChatGPT draws on.

The distribution is brutally long-tailed. Across all four engines, the aggregate Gini coefficient on citation frequency by domain was 0.68. Breaking it down per engine, ChatGPT had the highest concentration of the four at 0.648, ahead of Gemini (0.595), Perplexity (0.563) and Copilot (0.492). The top 25 domains accounted for 23.8% of all citations; 59.1% of cited domains were cited exactly once and 16.5% exactly twice.

That distribution is the strategic fact of this entire channel. It means a single placement tells you almost nothing, and that the dominant behaviour is reliable retrieval across many queries, not winning one. Sites optimising for a single lucky citation are optimising for a distribution that punishes exactly that.

One more finding worth carrying: 8.8% of the 2,848 audited responses carried no citations at all, with health queries worst at 11.6%. Some ChatGPT answers come from parametric knowledge, not from your page at all.

---

Entity signals: what the evidence actually supports

Two independent studies point the same direction. That convergence matters more than either result alone.

The CUNY/Columbia Authority Signals study ran 100 health questions sampled from HealthSearchQA into ChatGPT 5.2 Pro, then hand-coded all 615 resulting cited sources across four domains: who wrote it (author credentials), who published it (institutional affiliation), how was it vetted (quality assurance), and how does AI find it (digital authority). Over 75% of cited sources came from established institutional sources - Mayo Clinic, Cleveland Clinic, Wikipedia, NHS, PubMed.

The Northwestern audit reaches the same conclusion from the opposite direction, measuring domain concentration rather than source attributes. Within its top 25 most-cited domains, government sites accounted for 33.0% of citations, Wikipedia 15.2%, academic and scientific publishers 14.6%, social platforms 8.8% - and news media had “minimal representation.”

Two teams used two methods across two domains and found the same pattern: institutional sources dominate ChatGPT’s citations. Commercial editorial brands have a smaller presence. I’d look for a specific claim where your evidence is more detailed than theirs: a configuration, a regional price, a benchmark, or an edge case. Trying to outrank an institution across its whole subject is a much harder fight.

For entity work specifically, there’s one machine-readable path with a real specification: the Agentic Commerce Protocol Product Feed. It requires id, title, description, price, currency, availability and image_url, and recommends category, brand and product_type. It specifies TSV as the recommended format, UTF-8, a 10GB ceiling, and refresh cadence of up to every 15 minutes. It gives concrete image guidance (at least 800×800, JPEG/PNG/WebP, white or transparent background) and rating fields.

This is a feed specification for commerce surfaces. Use its title and description ranges to prepare a valid feed. They aren’t organic ranking factors.

---

Claim-level content structure

Generative search doesn’t rank pages. It retrieves passages and attributes them to claims. If your page is organised around topics, the retriever finds a section and you get a general citation. If it’s organised around claims, the retriever finds the one that answers the question and attributes that.

Four structures that work better than topic-organised prose in this environment:

Answer-first openings. Put the direct answer in the first two sentences under the heading, then support it. A retrieval window that captures the answer and part of the evidence produces a better-grounded citation than one that captures thesis and setup.

Self-contained sections. The find_in_page action means a system can search within a retrieved document rather than reading it whole. Sections that depend on context established twenty paragraphs earlier are effectively invisible to that mode. Make each section survive extraction.

Stable, quotable facts with explicit subjects. Write the number, the date, the entity name, the qualifier - in the same sentence. “Our p95 latency was 340ms across 4,000 requests in March 2026” extracts cleanly. “Performance has historically been a strength for the platform” extracts as nothing.

Date everything. ChatGPT Search is built for current information, and the help centre tells users to “review when it was published or updated.” An undated page is a page that can’t be confidently cited for a current question. A visible, machine-readable date on the page is cheap and it isn’t optional in a recency-weighted environment.

The failure mode is writing for the model instead of the reader. If a page reads like it was engineered for extraction, humans bounce and your engagement signals degrade - and you have optimised for a channel where you can’t verify the result.

---

The syndication layer your robots.txt does not cover

The most operationally important finding in the Tow Center’s sixteen-hundred-query study is one robots.txt rule can’t fix.

The study - published 6 March 2025, with tests conducted in February 2025 - covered eight generative search tools including ChatGPT Search, across 20 news publishers. Two findings undercut the comforting story that opting out works:

1. “While USA Today blocks ChatGPT’s crawler, the chatbot cited a version of its article that was syndicated in Yahoo News.”
2. “Generative search tools fabricated links and cited syndicated and copied versions of articles.”

Your robots.txt controls access to your URL. It doesn’t control a copy published on another domain. If your content turns up through syndication after you block access, the system can credit Yahoo or an unofficial mirror instead. You lose the attribution and the referral. That’s a poor result for a publisher trying to control reuse.

The same study found the inverse failure too: the San Francisco Chronicle permits OpenAI’s search crawler and sits inside a Hearst partnership, yet ChatGPT correctly identified only one of ten excerpts from it. ChatGPT also misidentified 134 articles across its 200 responses while signalling low confidence only fifteen times, and never once declining to answer.

The practical response is bounded, not magical: if you syndicate, canonicalise consistently and make the original the machine-attributable one; and accept that opt-out governs your URL, not your content’s existence on the web.

---

Measurement: the honest state of ChatGPT referrer data and logging

I want to be direct here, because the alternative is the single largest source of bad advice in this field.

OpenAI publishes no referrer scheme. No utm_source, no documented referrer header, no attribution parameter. The OpenAI crawler and web-search documentation contain no referrer specification.

So: treat any specific statement about ChatGPT’s referrer behaviour in either direction as practitioner report, not fact. You’ll find confident assertions. Treat them accordingly.

What you can do rests entirely on documented ground. OpenAI publishes exact user-agent strings, including a distinct robots.txt; marker variant. That gives you reliable log segmentation without any referrer at all:

OAI-SearchBot      → automated crawl, search surface
OAI-SearchBot + "robots.txt" marker → policy check, not content fetch
GPTBot             → training crawl
ChatGPT-User       → user-triggered retrieval
OAI-AdsBot         → ad landing page validation

Check each request against the published IP files. A user-agent string is easy to spoof. Remember the overlap: six prefixes serve both GPTBot and OAI-SearchBot, so an IP alone can’t separate those two. For the other three bots, the mapping is exact.

The second measurement method is prompt-panel tracking: a fixed set of queries run on a schedule, recording whether you appear in the citations. It’s the method the OpenAI API documentation practically points you toward, because sources gives you the full consulted-URL list while the rendered UI gives you only citations. Expect noise. The Tow Center study’s authors noted that chatbot responses are dynamic enough that “if someone ran the exact same prompts again, they would get different outputs.”

And accept the structural limit: a fixed prompt panel measures the queries you chose. It tells you about those. Nothing available to a publisher measures the distribution.

---

Documented, inferred, and unknown

A confidence table, because the difference between these three is the whole point.

Area	Status	Basis
Which bot governs search inclusion	Documented	OAI-SearchBot, stated explicitly
Training is governed separately by GPTBot	Documented	Stated explicitly
ChatGPT-User does not govern Search	Documented	Stated explicitly
Eligibility = robots.txt + network access	Documented	Help centre
~24h robots.txt propagation	Documented	Stated explicitly
Query rewriting before retrieval	Documented	Help centre, with example
Consulting third-party search providers	Documented	Help centre + launch post
Consulted-URL count exceeds citation count	Documented	API docs
PDF / image / video sources are cited	Documented (empirical)	arXiv:2605.23684
Citation distribution is heavily concentrated	Documented (empirical)	arXiv:2605.23684, Gini 0.648 for ChatGPT
Institutional sources dominate health citations	Documented (empirical)	arXiv:2601.17109
robots.txt can be circumvented via syndication	Empirical, dated	Tow Center, Feb 2025 conditions
Any specific ranking factor	Unknown	Never published
JavaScript rendering behaviour	Unknown	Never documented
Canonical / sitemap handling	Unknown	Never documented
Referrer or UTM scheme	Unknown	No official documentation found
Effect of llms.txt on ChatGPT Search	Unknown	No vendor statement

On llms.txt, one clarification worth making because it gets garbled constantly. The llms.txt proposal is a community document. Its author reports that “thousands of sites” publish one and that OpenAI, Anthropic and Gemini each publish one for their own developer docs. https://developers.openai.com/llms.txt returns HTTP 200 with real content, while https://platform.openai.com/llms.txt returns 404.

OpenAI publishing llms.txt for its own documentation doesn’t show that ChatGPT Search reads the file on other sites. The spec gives the files different jobs. The robots.txt file sets access rules. The llms.txt file supplies information when an agent needs it. I’d keep those jobs separate when deciding what to implement.

---

A working checklist

Crawl-access layer, in priority order:

1. Read your current robots.txt. Check whether a broad User-agent: * group with a Disallow rule is catching OAI-SearchBot by default. Under standard robots.txt semantics an unlisted bot is unrestricted - so the actual risk is inheritance from a wildcard group, not omission. Either fix the wildcard group or add an explicit User-agent: OAI-SearchBot + Allow: / before it, since the most specific matching group wins.
2. Confirm OAI-AdsBot handling matches whether you actually run ChatGPT ads. If you don’t, it costs you nothing to leave it unblocked.
3. Decide GPTBot deliberately, on the decision tree above. Record the date and reason. Implement it as a robots.txt rule - not as a network block, because six prefixes are shared with OAI-SearchBot.
4. Verify your CDN and WAF allow every prefix in searchbot.json. Re-verify after every infrastructure change, and re-read the file’s creationTime to see whether OpenAI has moved it since you last checked.
5. Separate the four bots in log analysis using the documented user agents, including the robots.txt; marker. Validate against the published IP files, and remember that IP alone can’t separate GPTBot from OAI-SearchBot on the six shared prefixes.

Evidence layer:

6. Reorganise priority pages so each section opens with its direct answer.
7. Make every section self-contained enough to survive find_in_page extraction.
8. Put subject, number, date and qualifier in the same sentence on every fact you want quoted.
9. Put a visible, machine-readable publication or update date on time-sensitive pages.

Entity layer:

10. If you sell products, implement the ACP Product Feed with accurate brand, category and product_type. Treat it as feed hygiene, not a ranking lever.
11. For non-commerce entities, prioritise a stable, well-linked presence on the institutional sources your category actually cites - not on the sources that cite you most often.
12. Audit your syndicated copies. Make sure the original is the machine-attributable version.

Measurement layer:

13. Build a fixed prompt panel against real customer questions and run it on a schedule. Expect variance.
14. Where you can, compare rendered citations against the sources list to see the gap you aren’t seeing.
15. Don’t build reporting on a referrer parameter. Segment from server logs.

The through-line: OpenAI gave you four documented levers and told you the outcome isn’t guaranteed. Take the levers - they’re real - and hold the rest loosely.

---

Sources and References

OpenAI primary documentation

• Overview of OpenAI Crawlers - bot definitions, user-agent strings, IP endpoints.
• Web search | OpenAI API - search modes, actions, sources field, domain filters, context limits, deprecations.
• Searching the web with ChatGPT - eligibility rules, query rewriting, search providers, “placement is not guaranteed.”
• Introducing ChatGPT search - published 2024-10-31, last updated 2025-02-05. Third-party providers, partner list, publisher opt-in language.
• Ads | OpenAI Developers and In production | Agentic Commerce - commerce surfaces and network requirements.
• Product Feed Specification - required and recommended fields, formats, cadence.
• OAI-SearchBot IP ranges and GPTBot IP ranges - creationTime values 2026-01-02 and 2026-09-22 respectively.

Independent research

• Allaham, M. et al. (2026), “Synthetic Sources? Auditing Generative Search Engine Citations for Evidence of AI-Generated Sources”, arXiv:2605.23684, AIES 2026. Submitted 22 May 2026.
• Jacques, E. et al. (2026), “Authority Signals in AI Cited Health Sources”, arXiv:2601.17109. Submitted 23 January 2026.
• Jaźwińska, K. and Chandrasekar, A. (2025), “AI Search Has a Citation Problem”, Tow Center for Digital Journalism, Columbia Journalism Review. Published 6 March 2025; tests conducted February 2025.

Comparison vendors

• Does Anthropic crawl data from the web? - Anthropic Help Centre, page dated 7 April 2026.
• Perplexity Crawlers Standards proposal
• The /llms.txt file, v2 - community proposal.
• https://developers.openai.com/llms.txt - Existence of the file is not evidence that ChatGPT Search reads llms.txt on third-party sites.