1
What it creates
Robots.txt gives crawler-specific instructions about which site paths may or may not be requested.
Generator
Robots.txt Generator
Define per-bot rules, disallow paths, and ship a clean robots.txt to your site root.
Rule 1
robots.txt
User-agent: * Disallow:
Upload the file to your site root so it's served at /robots.txt.
Quick guide
Understand your result
2
Read the result
An empty Disallow allows crawling. A slash blocks the whole site, so review broad rules carefully before publishing.
3
What to do next
Save the file at /robots.txt, include your sitemap address, and test important pages with the Robots.txt Tester.
Save your report
Download a full SEO audit for your page
Enter a page URL and save every recommendation — including the Robots.txt Generator checks — as a file you can keep or share.