Robots.txt Generator
Create robots.txt files with crawl rules for Googlebot, Bingbot, disallows, and sitemap URL.
How to Use Robots.txt Generator
Set permissions for Googlebot, Bingbot, or all crawlers (*).
Add paths to disallow (e.g. /admin/, /api/).
Include your XML sitemap URL and download robots.txt.
Frequently Asked Questions
Where does robots.txt belong?
Robots.txt must be accessible at the exact root of your domain: https://example.com/robots.txt.
Related Developer Tools
Explore more free client-side developer utilities in this category
Meta Tag Generator
Generate HTML SEO meta tags: title, description, keywords, viewport, canonical, and robots.
Open Graph Tag Generator
Generate Open Graph (OG) and Twitter Card tags with live social media card previews.
Sitemap Generator
Generate XML sitemaps for Google Search Console with priority, changefreq, and lastmod tags.
QR Code Generator
Generate high-resolution QR codes for URLs, WiFi credentials, vCards, text, and email.
Barcode Generator
Generate standard Code 128, EAN-13, and UPC barcodes with customizable height and download.
Favicon Generator
Convert images or emojis into multi-resolution favicon packs with HTML link tags.