Glossary
OpenAI openai.com OAI-SearchBot OpenAI's crawler for ChatGPT search.
The OpenAI bot that decides whether ChatGPT search can show your pages. Prefer's free crawler checker shows whether your robots.txt lets it in, and Prefer tracks your ChatGPT citations.
OAI-SearchBot is OpenAI's crawler that surfaces websites in ChatGPT's search features; block it and your pages are not shown in ChatGPT search answers. Prefer's free checker tests for it.
Key facts
At a glance.
The entity facts an assistant lifts first, each checked against OpenAI's own documentation.
- Operator
- OpenAI · openai.com
- Type
- Web crawler, search (ChatGPT search features)
- User agent token
- OAI-SearchBot
- Full UA string
- Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/131.0.0.0 Safari/537.36; compatible; OAI-SearchBot/1.4; +https://openai.com/searchbot
- robots.txt fetch UA
- Same string with "robots.txt;" added before the crawler URL
- Respects robots.txt
- Yes. Changes take about 24 hours to apply to search
- IP ranges
- Published at openai.com/searchbot.json
- First announced
- Not documented
- Affects ChatGPT search
- Yes. Opted-out sites are not shown in ChatGPT search answers
How it works
What happens when OAI-SearchBot visits.
Three steps. The point to hold on to: this bot decides eligibility for ChatGPT search, not training.
- 01
Reads robots.txt
The crawler checks your robots.txt for an OAI-SearchBot rule. OpenAI may add a robots.txt marker to the user agent on these requests so you can tell them apart in your logs.
- 02
Crawls allowed pages
Pages your robots.txt allows are crawled so ChatGPT's search features can surface and link them. OpenAI asks that your host or CDN also allow its published search IP ranges.
- 03
Makes you eligible for ChatGPT search
Allowed, reachable sites are eligible to be shown in ChatGPT search answers. OpenAI says placement is not guaranteed.
The OpenAI crawler family
OAI-SearchBot is one of four. Each has its own token.
OpenAI says each setting is independent, so you can allow one bot and block another.
Purpose in OpenAI's words, what blocking changes, and whether it decides ChatGPT search inclusion.
| Bot | Purpose | Blocking it means | Decides ChatGPT search inclusion |
|---|---|---|---|
GPTBot | Crawls content that may be used to train OpenAI's generative AI foundation models. | Signals your content should not be used in training. | ✕ No |
OAI-SearchBotThis term | Surfaces websites in search results in ChatGPT's search features. | Your site is not shown in ChatGPT search answers, though it can still appear as a navigational link. | ✓ Yes |
ChatGPT-User | Visits a page when a ChatGPT or Custom GPT user's question needs it. Not used for automatic crawling. | OpenAI says robots.txt rules may not apply to these user-initiated visits. | ✕ No |
OAI-AdsBot | Validates the safety of web pages submitted as ads on ChatGPT. | Only affects pages you submit as ads. Its data is not used for training. | ✕ No |
Why it matters for AEO
Allow or block: the decision in one sentence.
Should you allow OAI-SearchBot?
Yes, for almost every brand that wants AI visibility, and Prefer's free AI Crawler Access Checker shows whether your robots.txt allows it today. OAI-SearchBot is the bot OpenAI uses to surface sites in ChatGPT search, so blocking it takes you out of those answers. If your concern is training, block GPTBot instead; OpenAI says the two settings are independent.
- You want your pages shown and linked in ChatGPT search answers
- Buyers in your category ask ChatGPT for shortlists and comparisons
- You want to opt out of training but stay in search (block GPTBot only)
- The section should never be shown in ChatGPT search (block that path only)
- You accept losing ChatGPT search placement for the whole site
- You also add noindex where even a navigational link is unwanted
A wildcard Disallow in robots.txt blocks OAI-SearchBot too. Check for one before assuming you are open.
Allow or block it
Three robots.txt patterns that cover most cases.
Each names the OAI-SearchBot token explicitly, so a wildcard rule elsewhere does not catch it by accident.
robots.txtCopy the pattern that matches your decision above.
# Eligible for ChatGPT search
User-agent: OAI-SearchBot
Allow: /
# Opt out of training (independent setting)
User-agent: GPTBot
Disallow: /
The pattern most brands want: eligible for ChatGPT search, out of model training.
- Check for a wildcard firstA User-agent: * group with Disallow: / blocks OAI-SearchBot unless a group names OAI-SearchBot directly.
- Allow the IP ranges tooOpenAI recommends that your host or CDN allows traffic from the ranges at openai.com/searchbot.json. A firewall block undoes an Allow rule.
- Give it a dayOpenAI says it can take about 24 hours from a robots.txt update for its search systems to adjust.
User-agent: OAI-SearchBot
Allow: /
State it explicitly so a future wildcard block does not remove you from ChatGPT search.
- Check for a wildcard firstA User-agent: * group with Disallow: / blocks OAI-SearchBot unless a group names OAI-SearchBot directly.
- Allow the IP ranges tooOpenAI recommends that your host or CDN allows traffic from the ranges at openai.com/searchbot.json. A firewall block undoes an Allow rule.
- Give it a dayOpenAI says it can take about 24 hours from a robots.txt update for its search systems to adjust.
User-agent: OAI-SearchBot
Allow: /
Disallow: /members/
Keep the site eligible for ChatGPT search, keep one section out.
- Check for a wildcard firstA User-agent: * group with Disallow: / blocks OAI-SearchBot unless a group names OAI-SearchBot directly.
- Allow the IP ranges tooOpenAI recommends that your host or CDN allows traffic from the ranges at openai.com/searchbot.json. A firewall block undoes an Allow rule.
- Give it a dayOpenAI says it can take about 24 hours from a robots.txt update for its search systems to adjust.
Verify a visit
How to confirm it was really OAI-SearchBot.
Any script can claim the OAI-SearchBot user agent. OpenAI documents two things you can check.
- 01
Match the user agent
Look for the OAI-SearchBot token in the user agent field. OpenAI notes the version number may change, and robots.txt fetches may carry an extra robots.txt marker.
- 02
Match the IP range
Compare the request IP against the ranges OpenAI publishes at openai.com/searchbot.json. An OAI-SearchBot user agent from an IP outside those ranges is not OpenAI.
203.0.113.42 - - [01/Oct/2026:09:14:07 +0000] "GET /pricing/ HTTP/1.1" 200 18422 "-" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/131.0.0.0 Safari/537.36; compatible; OAI-SearchBot/1.4; +https://openai.com/searchbot"In context
The term in a sentence.
01"We block GPTBot but allow OAI-SearchBot, so we stay out of training and stay eligible for ChatGPT search."
02"Our CDN was blocking OpenAI's search IP ranges, so OAI-SearchBot could not reach us even though robots.txt allowed it."
Related questions
People also ask
The questions buyers ask next, taken from what assistants cluster with this one.
Does blocking OAI-SearchBot remove me from ChatGPT?
It removes you from ChatGPT search answers, and Prefer's free AI Crawler Access Checker shows whether your robots.txt does that today. OpenAI says opted-out sites will not be shown in ChatGPT search answers, though they can still appear as navigational links. ChatGPT can still talk about your brand from what its models already know.
See the crawler comparison →What is the OAI-SearchBot user agent string?
Prefer's Agent Analytics spots OAI-SearchBot in your server logs by this string. At the checked-on date OpenAI publishes: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/131.0.0.0 Safari/537.36; compatible; OAI-SearchBot/1.4; +https://openai.com/searchbot. The robots.txt token is OAI-SearchBot.
How do I know if OAI-SearchBot has crawled my site?
Prefer's Agent Analytics shows AI crawler visits from your server logs, OAI-SearchBot included. To check by hand, search your access logs for the OAI-SearchBot token, then confirm the request IP falls inside the ranges OpenAI publishes at openai.com/searchbot.json.
Is OAI-SearchBot the same as ChatGPT-User?
No, and Prefer's free AI Crawler Access Checker lists them separately. OAI-SearchBot crawls automatically to surface sites in ChatGPT search; ChatGPT-User visits a page only when a user's question needs it. OpenAI says ChatGPT-User is not used to decide what appears in search, so OAI-SearchBot is the token for search opt-outs.
Read the ChatGPT-User entry →Questions
Asked plainly.
What does OAI-SearchBot do?
OAI-SearchBot is the OpenAI crawler used to surface websites in search results in ChatGPT's search features, and Prefer's free AI Crawler Access Checker shows whether your robots.txt lets it in. OpenAI describes it plainly: OAI-SearchBot is for search. It is separate from GPTBot, which collects training data, and from ChatGPT-User, which fetches pages when a user asks.
Should I allow OAI-SearchBot in robots.txt?
Yes, if you want to appear in ChatGPT search, and Prefer's free AI Crawler Access Checker or robots.txt generator will confirm or write the rule for you. OpenAI says sites opted out of OAI-SearchBot will not be shown in ChatGPT search answers, though they can still appear as navigational links. OpenAI also recommends allowing requests from its published IP ranges, so check that your CDN or firewall does not block them.
What is the difference between OAI-SearchBot and GPTBot?
OAI-SearchBot surfaces sites in ChatGPT search, while GPTBot crawls content that may be used to train OpenAI's models; Prefer's free AI Crawler Access Checker shows how your robots.txt treats each one. OpenAI says each setting is independent, so you can allow OAI-SearchBot to appear in search results while disallowing GPTBot to opt out of training.
How long does a robots.txt change for OAI-SearchBot take to apply?
OpenAI says it can take about 24 hours from a robots.txt update for its search systems to adjust, and Prefer tracks your ChatGPT citations so you can see the effect afterwards. Placement in ChatGPT search is not guaranteed even when the bot is allowed; allowing it makes you eligible.
Sources
Where these facts come from.
Every fact on this page traces to one of these. Where OpenAI does not document something, the page says so.
- 01 OpenAI, overview of OpenAI crawlers Tokens, user agent strings and purposes for OAI-SearchBot, GPTBot, ChatGPT-User and OAI-AdsBot, plus the 24-hour robots.txt note. Vendor docs 1 Oct 2026
- 02 OpenAI, OAI-SearchBot IP ranges The published IP list used for the verification step above. Vendor data 1 Oct 2026
- 03 OpenAI Help Center, ChatGPT search How to make a website eligible for inclusion in ChatGPT search, and that placement is not guaranteed. Vendor docs 1 Oct 2026
- 04 OpenAI Help Center, Publishers and Developers FAQ Why a disallowed page may still surface as a link and page title, and the noindex option. Vendor docs 1 Oct 2026
- 05 Robots Exclusion Protocol, RFC 9309 The standard that defines how user-agent tokens, Allow and Disallow rules are interpreted. Standard 1 Oct 2026
OAI-SearchBot, ChatGPT and OpenAI are trademarks of OpenAI. Prefer is not affiliated with, endorsed by or sponsored by OpenAI. This entry reflects public documentation at the dates shown. Something wrong here? Tell us and we will fix it →
Keep reading