← all sites
Food sites vs AI crawlers
7 tracked food sites, 6 with readable policy. Snapshot: 2026-08-25
83.3%
block ≥1 AI crawler (5/6)
ClaudeBot
most-blocked bot here (5 sites)
0
publish llms.txt
| Site | llms.txt | GPTBot | ClaudeBot | Google-Extended | PerplexityBot | CCBot | Bytespider |
|---|---|---|---|---|---|---|---|
| yelp.com | — | blocked | blocked | blocked | blocked | blocked | blocked* |
| allrecipes.com | — | restricted | blocked | blocked | blocked | blocked | blocked |
| bonappetit.com | — | restricted* | blocked | blocked | blocked | blocked | blocked |
| epicurious.com | — | restricted* | blocked | blocked | blocked | blocked | blocked |
| seriouseats.com | — | restricted | blocked | blocked | blocked | blocked | blocked |
| foodnetwork.com | — | restricted* | restricted* | restricted* | restricted* | restricted* | restricted* |
| opentable.com | — | unknown | unknown | unknown | unknown | unknown | unknown |