Spaces:

VIDraft
/

AI

Running

AI / robots.txt

robots.txt: explicitly welcome AI crawlers (GPTBot/OAI-SearchBot/PerplexityBot/ClaudeBot/Google-Extended) + Sitemap directive

55b505a verified 4 days ago

533 Bytes

	User-agent: *
	Allow: /

	# AI search engines & LLM crawlers are explicitly welcome.
	# Structured summary for AI: https://vidraft.net/llms.txt
	User-agent: GPTBot
	Allow: /

	User-agent: OAI-SearchBot
	Allow: /

	User-agent: ChatGPT-User
	Allow: /

	User-agent: PerplexityBot
	Allow: /

	User-agent: Perplexity-User
	Allow: /

	User-agent: ClaudeBot
	Allow: /

	User-agent: Claude-SearchBot
	Allow: /

	User-agent: Google-Extended
	Allow: /

	User-agent: Applebot-Extended
	Allow: /

	User-agent: CCBot
	Allow: /

	Sitemap: https://vidraft.net/sitemap.xml