ROBOTS.TXT VALIDATOR
Testing Tool
Test and validate your robots.txt file. Enter your rules below and check whether certain URLs are blocked (disallowed) or allowed for different user agents.
1
Result
Matching rule
-
Line -
Checked user agent
-
Press TEST to start
Common questions
How does the matching work?
Bots first look for the most specific "User-agent" block. If there's a block for "Googlebot", for example, Google uses that. If there isn't, it falls back to "*". Within the block, usually the longest matching rule wins (longest match rule).
What is a robots.txt validator?
A robots.txt validator is a tool with which you can check whether your
robots.txt file is syntactically correct and whether it actually blocks or allows the intended URLs. It simulates the behavior of search engine bots (like Googlebot) and shows you which rule is applied for a certain URL. That helps avoid SEO problems where important pages are accidentally locked out.