← all sites

apache.org

apache.org currently blocks none of the 16 AI crawlers we track. Snapshot: 2026-08-25

Category
top1k
robots.txt
readable — view live file
Default (*) policy
restricted
llms.txt
none found
Machine-readable
JSON for this site

All 16 tracked AI bots

BotOperatorPurposeStatusHow it's set
GPTBotOpenAItrainingrestricted*default (*) rules
ChatGPT-UserOpenAIfetchrestricted*default (*) rules
OAI-SearchBotOpenAIsearchrestricted*default (*) rules
ClaudeBotAnthropictrainingrestricted*default (*) rules
Claude-UserAnthropicfetchrestricted*default (*) rules
Claude-SearchBotAnthropicsearchrestricted*default (*) rules
Google-ExtendedGoogletrainingrestricted*default (*) rules
Applebot-ExtendedAppletrainingrestricted*default (*) rules
PerplexityBotPerplexitysearchrestricted*default (*) rules
Perplexity-UserPerplexityfetchrestricted*default (*) rules
CCBotCommon Crawltrainingrestricted*default (*) rules
BytespiderByteDancetrainingrestricted*default (*) rules
meta-externalagentMetatrainingrestricted*default (*) rules
AmazonbotAmazontrainingrestricted*default (*) rules
DuckAssistBotDuckDuckGosearchrestricted*default (*) rules
MistralAI-UserMistralfetchrestricted*default (*) rules

Policy history

Tracking began 2026-08-25 (index founding). Changes to this site's AI policy will appear here as daily crawls accumulate.