← all sites

Education sites vs AI crawlers

10 tracked education sites, 7 with readable policy. Snapshot: 2026-09-13

28.6%
block ≥1 AI crawler (2/7)
meta-externalagent
most-blocked bot here (2 sites)
1
publish llms.txt
Sitellms.txtGPTBotClaudeBotGoogle-ExtendedPerplexityBotCCBotBytespider
ted.com — blockedblockedblockedrestrictedblockedblocked
coursera.org yes restrictedrestrictedrestrictedrestrictedrestrictedrestricted
chegg.com ? restrictedrestrictedrestrictedrestrictedrestrictedrestricted
duolingo.com — restrictedrestrictedrestrictedrestrictedrestrictedrestricted
harvard.edu — allowedallowedallowedallowedallowedallowed
khanacademy.org — unknownunknownunknownunknownunknownunknown
mit.edu — restrictedrestrictedrestrictedrestrictedrestrictedrestricted
quizlet.com ? unknownunknownunknownunknownunknownunknown
stanford.edu — restrictedrestrictedrestrictedrestrictedrestrictedrestricted
udemy.com ? unknownunknownunknownunknownunknownunknown