← all bots

Claude-SearchBot

Indexes pages to improve Claude's search citations. Operator docs.

Operator
Anthropic
Purpose
search
Blocked by
93 of 652 readable sites (14.3%) — 53 name it explicitly, 40 block it via default-deny rules

Sites blocking Claude-SearchBot

SiteCategoryHow
adform.nettop1kdefault-deny robots.txt
allrecipes.comfoodnamed explicitly
amazon.catop1knamed explicitly
amazon.co.jptop1knamed explicitly
amazon.co.uktop1knamed explicitly
amazon.co.zatop1knamed explicitly
amazon.comecommercenamed explicitly
amazon.com.autop1knamed explicitly
amazon.com.brtop1knamed explicitly
amazon.detop1knamed explicitly
amazon.estop1knamed explicitly
amazon.frtop1knamed explicitly
amazon.intop1knamed explicitly
amazon.ittop1knamed explicitly
apnews.comnewsnamed explicitly
arstechnica.comnewsnamed explicitly
baidu.comtop1kdefault-deny robots.txt
blackhub.teamtop1kdefault-deny robots.txt
bloomberg.comnewsnamed explicitly
bonappetit.comfoodnamed explicitly
chatgpt.comtop1kdefault-deny robots.txt
cnet.comtop1knamed explicitly
cnn.comnewsnamed explicitly
creativecdn.comtop1kdefault-deny robots.txt
crwdcntrl.nettop1kdefault-deny robots.txt
dailymail.co.uknewsnamed explicitly
dailymail.comtop1knamed explicitly
daum.nettop1kdefault-deny robots.txt
economist.comnewsnamed explicitly
elpais.comtop1knamed explicitly
epicurious.comfoodnamed explicitly
facebook.comsocialdefault-deny robots.txt
fb.comtop1kdefault-deny robots.txt
figma.comtechnamed explicitly
gosuslugi.rutop1kdefault-deny robots.txt
hihonorcloud.comtop1kdefault-deny robots.txt
huffpost.comnewsnamed explicitly
ign.comentertainmentnamed explicitly
imdb.comreferencedefault-deny robots.txt
imgsmail.rutop1kdefault-deny robots.txt
instacart.comecommercedefault-deny robots.txt
instagram.comsocialdefault-deny robots.txt
investopedia.comreferencenamed explicitly
launchpad.nettop1knamed explicitly
lefigaro.frtop1knamed explicitly
lemonde.frtop1knamed explicitly
mail.rutop1kdefault-deny robots.txt
mailchi.mptop1kdefault-deny robots.txt
marketwatch.comfinancedefault-deny robots.txt
meraki.comtop1kdefault-deny robots.txt
msn.comtop1knamed explicitly
myfritz.nettop1kdefault-deny robots.txt
naver.comtop1kdefault-deny robots.txt
nbcnews.comnewsnamed explicitly
newyorker.comnewsnamed explicitly
nikkei.comtop1knamed explicitly
nypost.comnewsnamed explicitly
nytimes.comnewsnamed explicitly
people.comtop1knamed explicitly
pinimg.comtop1kdefault-deny robots.txt
pinterest.comsocialdefault-deny robots.txt
quantserve.comtop1kdefault-deny robots.txt
quora.comreferencenamed explicitly
reddit.comsocialdefault-deny robots.txt
registrar-servers.comtop1kdefault-deny robots.txt
reuters.comnewsdefault-deny robots.txt
rocketmoney.devtop1kdefault-deny robots.txt
sentinelone.nettop1kdefault-deny robots.txt
seriouseats.comfoodnamed explicitly
sharethrough.comtop1kdefault-deny robots.txt
sohu.comtop1kdefault-deny robots.txt
spiegel.detop1knamed explicitly
stbid.rutop1kdefault-deny robots.txt
t.cotop1kdefault-deny robots.txt
telegraph.co.uktop1knamed explicitly
theatlantic.comnewsnamed explicitly
theguardian.comnewsnamed explicitly
theverge.comnewsnamed explicitly
threads.comsocialdefault-deny robots.txt
tiktok.comsocialnamed explicitly
tiktokw.ustop1kdefault-deny robots.txt
twitter.comtop1kdefault-deny robots.txt
unsplash.comtop1knamed explicitly
uol.com.brtop1knamed explicitly
usatoday.comnewsnamed explicitly
vkontakte.rutop1kdefault-deny robots.txt
vox.comnewsnamed explicitly
weibo.comtop1kdefault-deny robots.txt
wikihow.comreferencenamed explicitly
wired.comnewsnamed explicitly
wsj.comnewsdefault-deny robots.txt
x.comsocialdefault-deny robots.txt
yelp.comfoodnamed explicitly