robots.txt generator & safety checker

Choose a starting point, customise crawl rules and check for common high-risk conflicts. Everything runs locally.

Select a preset and generate your file.

Generated robots.txt

Generated rules will appear here.

robots.txt controls crawling, not guaranteed indexing or access security. Never use it to protect sensitive information; use authentication and server permissions. Test important rules in your search-engine webmaster tools before publishing.