Applebot-Extended
Operated by Apple · Governs Apple Intelligence training use
Applebot-Extended is not a crawler — it never fetches a page itself. It's a robots.txt token that determines whether content already crawled by Applebot may be used to train Apple's generative AI foundation models, including Apple Intelligence.
What it does
Apple's documentation is explicit that Applebot-Extended does not crawl webpages. It is only used to determine how data already crawled by the Applebot user agent may be used — specifically, whether it may help train Apple's generative foundation models.
Apple states that webpages disallowing Applebot-Extended can still be included in search results — this token controls training use only, independently of Applebot's search crawling.
Why this is optional to block
- It's a training-use switch, not an access control — it never fetches your pages
- Apple states disallowing it has no effect on search inclusion
- Whether to allow your content into Apple Intelligence training is a content-strategy decision, not a technical requirement
Identification & robots.txt
Applebot-Extended
Applebot-Extended has no independent crawl to verify by IP or request pattern — it's a named group in your robots.txt file. Additional detail is available in Apple's official documentation.
User-agent: Applebot-Extended
Disallow: /