← all sites
Real estate sites vs AI crawlers
4 tracked real estate sites, 3 with readable policy. Snapshot: 2026-08-25
66.7%
block ≥1 AI crawler (2/3)
meta-externalagent
most-blocked bot here (2 sites)
0
publish llms.txt
| Site | llms.txt | GPTBot | ClaudeBot | Google-Extended | PerplexityBot | CCBot | Bytespider |
|---|---|---|---|---|---|---|---|
| zillow.com | — | restricted* | restricted* | restricted* | restricted* | restricted* | restricted* |
| realtor.com | — | restricted* | restricted* | restricted* | restricted* | restricted* | restricted* |
| apartments.com | — | unknown | unknown | unknown | unknown | unknown | unknown |
| redfin.com | — | restricted* | restricted* | restricted* | restricted* | restricted* | restricted* |