Morten A. Giraffe
← All tools

Website Toolkit / Inspect

Robots.txt tester

Check which pasted rule applies to a crawler and path.

Your input stays in this page’s memory. It is not submitted to a server or saved by this tool. Refreshing clears it.

Output / Waiting for input

Make the next step clear.

Start with your own content or try the example. Your result will appear here.

How this works

Selects the longest matching user-agent group, combines equally specific groups and evaluates Allow/Disallow path rules. Supports * and terminal $; the longest matching rule wins, with Allow winning ties.

What this cannot tell you

This is a local rule simulation. It cannot confirm the file a crawler fetched, server access, indexing or every crawler-specific extension. Crawl-delay is not evaluated.

Read the Technical SEO Field Guide →