# Allow all crawlers by default User-agent: * Allow: / User-agent: Googlebot Allow: / User-agent: bingbot Allow: / User-agent: OAI-SearchBot Allow: / User-agent: GPTBot Allow: / User-agent: ChatGPT-User Allow: / User-agent: Googlebot Allow: / User-agent: Bingbot Allow: / User-agent: ClaudeBot Allow: / User-agent: Claude-Web Allow: / User-agent: PerplexityBot Allow: / # Block specific paths if needed (examples) # Disallow: /admin/ # Disallow: /private/ # Disallow: /*.pdf$ # Sitemap location (optional but recommended) Sitemap: https://rentinvoice.in/sitemap.xml