← all sites
Entertainment sites vs AI crawlers
15 tracked entertainment sites, 13 with readable policy. Snapshot: 2026-08-25
76.9%
block ≥1 AI crawler (10/13)
Bytespider
most-blocked bot here (8 sites)
0
publish llms.txt
| Site | llms.txt | GPTBot | ClaudeBot | Google-Extended | PerplexityBot | CCBot | Bytespider |
|---|---|---|---|---|---|---|---|
| ign.com | — | blocked | blocked | restricted* | blocked | blocked | blocked |
| soundcloud.com | — | blocked | blocked | blocked | blocked | blocked | blocked |
| gamespot.com | — | blocked | blocked | blocked | allowed | blocked | blocked |
| netflix.com | — | restricted | restricted | restricted | restricted | blocked* | blocked* |
| polygon.com | — | blocked | blocked | blocked | blocked | blocked | blocked |
| roblox.com | — | blocked | blocked | blocked | restricted* | blocked | blocked |
| kotaku.com | — | restricted* | restricted* | restricted* | restricted* | blocked | blocked |
| pcgamer.com | — | restricted* | restricted* | restricted* | restricted* | restricted* | blocked |
| disneyplus.com | — | restricted | restricted | restricted | restricted | restricted | restricted |
| hulu.com | — | restricted | restricted | restricted | restricted | restricted | restricted |
| epicgames.com | — | unknown | unknown | unknown | unknown | unknown | unknown |
| max.com | — | restricted* | restricted* | restricted* | restricted* | restricted* | restricted* |
| minecraft.net | — | unknown | unknown | unknown | unknown | unknown | unknown |
| spotify.com | — | restricted* | restricted* | restricted* | restricted* | restricted* | restricted* |
| steampowered.com | — | restricted* | restricted* | restricted* | restricted* | restricted* | restricted* |