robots.txt tester

Enter a URL to see whether crawlers may fetch it, which line of robots.txt decides, and what’s wrong with the file.