← all sites

E-commerce sites vs AI crawlers

22 tracked e-commerce sites, 15 with readable policy. Snapshot: 2026-08-25

33.3%
block ≥1 AI crawler (5/15)
Bytespider
most-blocked bot here (4 sites)
2
publish llms.txt
Sitellms.txtGPTBotClaudeBotGoogle-ExtendedPerplexityBotCCBotBytespider
amazon.com — blockedblockedblockedblockedblockedblocked
instacart.com — blocked*blocked*blocked*blockedblocked*blocked*
ebay.com — blockedblockedrestricted*blockedblockedblocked
chewy.com — restricted*restricted*restricted*restricted*restricted*restricted*
temu.com — restrictedrestrictedrestrictedrestrictedallowedblocked
adidas.com — unknownunknownunknownunknownunknownunknown
aliexpress.com — restricted*restricted*restricted*restricted*restricted*restricted*
bestbuy.com — restrictedrestrictedrestrictedrestrictedrestrictedrestricted
costco.com — unknownunknownunknownunknownunknownunknown
doordash.com — unknownunknownunknownunknownunknownunknown
etsy.com — restricted*restricted*restricted*restricted*restricted*restricted*
homedepot.com — restricted*restricted*restricted*restricted*restricted*restricted*
ikea.com — restricted*restricted*restricted*restricted*restricted*restricted*
kroger.com — unknownunknownunknownunknownunknownunknown
lowes.com — unknownunknownunknownunknownunknownunknown
nike.com — restricted*restricted*restricted*restricted*restricted*restricted*
sephora.com — unknownunknownunknownunknownunknownunknown
shein.com yes restricted*restricted*restricted*restricted*restricted*restricted*
target.com yes restricted*restricted*restricted*restricted*restricted*restricted*
ubereats.com — unknownunknownunknownunknownunknownunknown
walmart.com — restricted*restricted*restricted*restricted*restricted*restricted*
wayfair.com — restricted*restricted*restricted*restricted*restricted*restricted*