# ================================= # Branex Super-Clean Optimized Robots.txt # ================================= # Block low-value / sensitive URLs User-agent: * Disallow: /wp-admin/ Disallow: /email-newsletter/ Disallow: /solution/ Disallow: /hire-developer-for-product/ Disallow: /lp-app-mockup-design/ Disallow: /product-focused-hire-developers/ Disallow: /revamp/ Disallow: /backup/ Disallow: /blog/tag/ Disallow: /blog/author/ Disallow: /author/ Disallow: /category/ Disallow: /page/ Disallow: /search/ Disallow: /?s= Disallow: /*?replytocom= Disallow: /*?utm_ Disallow: /*?fbclid= Disallow: /*?gclid= # Allow everything else Allow: / # Sitemap Sitemap: https://branex.com/sitemap-index.xml # Specific AI / LLM / SEO Crawlers (explicit allow) User-agent: ChatGPT-User User-agent: Google-Extended User-agent: ClaudeBot User-agent: anthropic-ai User-agent: oai-searchbot User-agent: oai-chatbot User-agent: OpenAI User-agent: DeepSharp User-agent: AhrefsBot User-agent: SemrushBot Allow: /