AI Crawler Robots.txt Generator
Allow or block GPTBot, ClaudeBot, Google-Extended, PerplexityBot & more. Free, private and runs in your browser.
AI crawlers (ChatGPT, Claude, Gemini, Perplexity…)
Block specific search engines & SEO crawlers
robots.txt
About the AI Crawler Robots.txt Generator
Decide which AI companies can crawl your content. Allowing AI search bots can bring citations and traffic from ChatGPT, Perplexity and Google AI; blocking training bots keeps your content out of model training sets. Toggle each bot and get a ready robots.txt.
How to use the AI Crawler Robots.txt Generator
- Choose a preset (allow all, block training only, block all AI) or toggle each bot.
- Add your normal rules and sitemap URL.
- Copy or download robots.txt and upload it to your site root.
Frequently asked questions
Does blocking Google-Extended affect my Google rankings?
No. Google-Extended only controls use of your content for Gemini training/grounding; Googlebot search crawling is unaffected.
Do all AI bots respect robots.txt?
Major providers (OpenAI, Anthropic, Google, Perplexity, Common Crawl) state that they do, but robots.txt is voluntary and cannot technically stop a bot.
Is the AI Crawler Robots.txt Generator free?
Yes. AI Crawler Robots.txt Generator on Tooliphy is 100% free with no signup, no watermark and no usage limits.
Is it safe to use? Are my files uploaded?
Your data is processed directly in your browser and is never uploaded to our servers, so it stays completely private.
Does it work on mobile?
Yes, the tool works on Android phones, iPhones, tablets and desktops in any modern browser.