Quick Reference
Element Code: XM-003
Issue: Sitemap includes URLs blocked by robots.txt
Impact: Conflicting signals, wasted crawl attempts
Fix: Remove disallowed URLs from sitemap
Detection: Screaming Frog, Google Search Console
What Is This Issue?
Including robots.txt blocked URLs in sitemaps sends contradictory signals. The sitemap says "index this" but robots.txt says "do not crawl."
Why This Matters for Your Website
Google may report these as errors in Search Console. It wastes crawl budget on URLs that cannot be accessed.
How to Fix This Issue
- Remove from sitemap: Exclude disallowed URLs
- Or unblock: If URLs should be crawled
- Configure generator: Respect robots.txt in sitemap generation
Tools for Detection
- Google Search Console: Reports sitemap errors
TL;DR (The Simple Version)
Your sitemap includes URLs blocked by robots.txt. Either remove them from the sitemap or unblock them.
About SEO ProCheck
Technical SEO consulting and GEO strategy with 20 years of enterprise experience. Case studies, resources, and tools for search and AI visibility.
Work With Me
Technical SEO audits, GEO strategy, site migrations, and international SEO. Hourly consulting for teams who need hands-on support, not just reports.
Subscribe to our newsletter!
Recent Posts
- No Social Schema December 7, 2025
- Missing Social Profile Links December 7, 2025
- Social Image Wrong Size December 7, 2025