After publishing the 12-point Claude Technical SEO Audit Checklist at kulbhushanpareek.com/blog/claude-technical-seo-audit-checklist, I ran the same checklist on this site using the identical prompts. The audit identified 4 specific issues that had not been addressed previously. The same checks now run automatically in my free Claude SEO audit tool.
Check 1 (robots.txt): The robots.txt had no explicit rule for ChatGPT-User. Bingbot was allowed. GPTBot was not explicitly specified. The fix: added explicit Allow rules for ChatGPT-User, PerplexityBot, and ClaudeBot. GPTBot was explicitly set to Allow for now pending a training data policy decision.
Check 10 (internal links): Three recently published posts had only 1 internal link each pointing to them from established pages. The posts were ranking at positions 35 to 45 in GSC, consistent with low internal authority. Fix: added contextual internal links from the 47 prompts post and the best Claude SEO checker post to each of the 3 affected pages.
Check 12 (AI crawler): Bing Webmaster Tools was already verified. Confirmed via Bing Webmaster Dashboard that new posts from the last 30 days had been indexed. No issues found.
Check 4 (index coverage): 6 URLs still showing "Crawled not indexed" for insight posts published in the last 2 weeks. Applied individual URL Inspection and Request Indexing for each. Not using Validate Fix.
The robots.txt ChatGPT-User finding on my own site is the one that will generate the most follow-up. A site I have been optimizing for AI visibility for months had no explicit ChatGPT-User rule. The default behavior (allow) means ChatGPT was accessing the site. But an explicit rule signals deliberate intent rather than accidental permission. Deliberate intent may improve crawl priority for AI retrieval systems. I cannot verify this directly but the effort cost of adding 3 lines to robots.txt is zero. It is now done. The internal link additions are verifiable: those 3 pages should see position improvements in 4 to 6 weeks. Want me to run this exact audit on your site? Book a free 30-minute call.
Leave a Comment
Your email address will not be published. Comments are moderated before appearing.