Robots.txt & Sitemap Generator
FreeGenerate robots.txt and XML sitemap files for your website, free.
Runs entirely in your browser. Nothing you enter is uploaded or stored.
These are generic starting points, not a guaranteed-correct configuration for your site — review and adjust the paths for your own structure before publishing.
User-agent: *
How it works
What does robots.txt actually control?
It tells search engine crawlers which parts of your site they may or may not crawl. Worth stating plainly since it's a common misconception: robots.txt is a voluntary convention, not a security or access-control mechanism. Well-behaved crawlers (Googlebot, Bingbot, and most legitimate search engines) respect it, but a Disallow rule doesn't actually block anyone from requesting that URL — anyone with the link, or a crawler that ignores the file entirely, can still access it. Never rely on robots.txt to keep something private or secure; use real authentication or access controls for that.
What does sitemap.xml do?
It helps search engines discover the pages on your site and understand their relative structure and priority — especially useful for large sites, or pages that aren't well-linked internally. It doesn't guarantee those pages get indexed or ranked; it's a discovery aid, not a promise.
Where do these files need to go?
Both belong at the root of your domain — for example, example.com/robots.txt and example.com/sitemap.xml. A robots.txt placed anywhere else (like example.com/pages/robots.txt) is not read by crawlers.
Is anything I enter here sent anywhere?
No — every file is generated locally in your browser. Nothing is uploaded or stored.