← all sites

Entertainment sites vs AI crawlers

15 tracked entertainment sites, 12 with readable policy. Snapshot: 2026-10-04

66.7%
block ≥1 AI crawler (8/12)
Diffbot
most-blocked bot here (7 sites)
0
publish llms.txt
Sitellms.txtGPTBotClaudeBotGoogle-ExtendedPerplexityBotCCBotBytespider
ign.com — blockedblockedrestrictedblockedblockedblocked
netflix.com — restrictedrestrictedrestrictedrestrictedblockedblocked
polygon.com — blockedblockedblockedblockedblockedblocked
disneyplus.com — restrictedrestrictedrestrictedrestrictedrestrictedrestricted
hulu.com — restrictedrestrictedrestrictedrestrictedrestrictedrestricted
soundcloud.com — blockedblockedblockedblockedblockedblocked
kotaku.com — restrictedrestrictedrestrictedrestrictedblockedblocked
pcgamer.com — restrictedrestrictedrestrictedrestrictedrestrictedblocked
epicgames.com ? unknownunknownunknownunknownunknownunknown
gamespot.com ? unknownunknownunknownunknownunknownunknown
max.com — restrictedrestrictedrestrictedrestrictedrestrictedrestricted
minecraft.net ? unknownunknownunknownunknownunknownunknown
roblox.com — restrictedrestrictedrestrictedrestrictedrestrictedrestricted
spotify.com — restrictedrestrictedrestrictedrestrictedrestrictedrestricted
steampowered.com — restrictedrestrictedrestrictedrestrictedrestrictedrestricted