← all sites
E-commerce sites vs AI crawlers
22 tracked e-commerce sites, 18 with readable policy. Snapshot: 2026-10-04
27.8%
block ≥1 AI crawler (5/18)
Bytespider
most-blocked bot here (4 sites)
3
publish llms.txt
| Site | llms.txt | GPTBot | ClaudeBot | Google-Extended | PerplexityBot | CCBot | Bytespider |
|---|---|---|---|---|---|---|---|
| instacart.com | — | blocked | blocked | blocked | blocked | blocked | blocked |
| amazon.com | — | blocked | blocked | blocked | blocked | blocked | blocked |
| ebay.com | — | blocked | blocked | restricted | blocked | blocked | blocked |
| chewy.com | — | restricted | restricted | restricted | restricted | restricted | restricted |
| temu.com | — | restricted | restricted | restricted | restricted | allowed | blocked |
| adidas.com | ? | unknown | unknown | unknown | unknown | unknown | unknown |
| aliexpress.com | ? | restricted | restricted | restricted | restricted | restricted | restricted |
| bestbuy.com | — | restricted | restricted | restricted | restricted | restricted | restricted |
| costco.com | — | restricted | restricted | restricted | restricted | restricted | restricted |
| doordash.com | — | restricted | restricted | restricted | restricted | restricted | restricted |
| etsy.com | yes | restricted | restricted | restricted | restricted | restricted | restricted |
| homedepot.com | — | restricted | restricted | restricted | restricted | restricted | restricted |
| ikea.com | — | restricted | restricted | restricted | restricted | restricted | restricted |
| kroger.com | ? | unknown | unknown | unknown | unknown | unknown | unknown |
| lowes.com | ? | unknown | unknown | unknown | unknown | unknown | unknown |
| nike.com | — | restricted | restricted | restricted | restricted | restricted | restricted |
| sephora.com | ? | unknown | unknown | unknown | unknown | unknown | unknown |
| shein.com | yes | restricted | restricted | restricted | restricted | restricted | restricted |
| target.com | yes | restricted | restricted | restricted | restricted | restricted | restricted |
| ubereats.com | — | restricted | restricted | restricted | restricted | restricted | restricted |
| walmart.com | — | restricted | restricted | restricted | restricted | restricted | restricted |
| wayfair.com | ? | restricted | restricted | restricted | restricted | restricted | restricted |