DP.HE

Free tools

robots.txt checker

Reads the file at /robots.txt and works out, rule by rule, which crawlers may fetch your site root. Most sites that can't get cited by AI engines are blocking the crawler by accident.

http:// is optional.

These run against the live URL from my server. Nothing you enter is stored or logged beyond the request itself.

Want the fixes, not just the findings?

The checks tell you what's there. Working out what to do about it is the consulting.