403 Forbidden URL in XML Sitemaps: How to Fix It

What a 403 in a sitemap means

An XML sitemap lists URLs you are explicitly asking search engines to crawl and consider for indexing, so every URL in that file is effectively a recommendation. A 403 Forbidden response means the server understood the request but refused to authorize it, so the crawler is denied access entirely. When a URL returns 403 and still appears in your sitemap, you send a contradictory message: pointing Google toward a door and then locking it.

Google's own guidance is clear that a sitemap should contain the canonical URLs you actually want shown in search results, and those URLs should return a clean 200 status. A 403 is the opposite of that. It is not a page Google can index, render, or rank, so its presence in the sitemap serves no useful purpose and actively undermines the quality of the file.

Why it matters

Sitemap hygiene and trust

Search engines use sitemaps as a trust signal about which pages you consider important. When a sitemap is full of errors such as 403s, 404s, redirects, or noindex pages, that trust erodes. As Sitebulb notes on this exact issue, if search engines repeatedly find this kind of dirt in your sitemaps, they may begin to discount the file as a reliable source of crawling and indexing signals. A clean sitemap that consistently resolves to indexable pages is one Google learns to rely on.

Crawl budget

Every URL a crawler attempts to fetch consumes a slice of your crawl budget. On a small site this is negligible, but on a large or frequently updated site, repeatedly directing crawlers to forbidden URLs means fewer requests are spent on the pages that actually deserve attention. Listing only accessible, indexable URLs keeps that budget focused where it counts.

Quality signal

A sitemap riddled with inaccessible URLs reads as neglected. It suggests the file is auto-generated without validation and not maintained in step with the live site. That impression works against you, because the sitemap is one of the most direct lines of communication you have with search engines.

Common causes

A 403 in a sitemap usually traces back to one of a handful of access-control mechanisms:

Authentication-gated pages. Member areas, account dashboards, staging environments, and other login-protected URLs return 403 to anyone not signed in, including crawlers. These pages should generally never be in a public sitemap in the first place.

Hotlink protection and directory rules. Server rules that protect assets or directories, often set in .htaccess, can inadvertently forbid access to legitimate pages or referenced resources.

Bot blocks and over-zealous firewalls. Security platforms, WAFs, and DDoS-protection services sometimes detect crawling activity and treat it as an attack, returning 403 to the crawler even when a normal browser loads the page fine. Screaming Frog notes that the 403 a crawler sees may differ from what a real search engine receives, so this needs verification rather than assumption.

How to diagnose

Start by confirming the status code yourself. A quick request from the command line tells you exactly what the server returns:

curl -I https://example.com/the-flagged-url

HTTP/2 403
server: nginx
content-type: text/html; charset=UTF-8

Next, run a crawl in Screaming Frog or Sitebulb in list mode against your sitemap URLs. Both tools surface every URL that returns 403 and group them together, making the scope obvious at a glance. If the page loads normally in a browser but returns 403 to the crawler, test with a search-engine user-agent and from a different IP to check whether a firewall or bot rule is the culprit. You can also use the URL Inspection tool in Google Search Console to see how Googlebot itself fares when it fetches the page.

How to fix it

There are two valid resolutions, and the right one depends on whether the page is meant to be public.

Option 1: Remove the URL from the sitemap

If the page is intentionally restricted, such as a login-only account page or a private resource, it does not belong in a public sitemap. Remove it. If your sitemap is generated by a CMS or SEO plugin, exclude that page or post type from sitemap generation rather than editing the file by hand, so the fix survives the next regeneration.

Option 2: Fix the access restriction

If the page should be public and the 403 is a mistake, fix what is blocking it. Review your server rules, for example an over-broad rule in .htaccess, and make sure legitimate user-agents are allowed through:

# Audit .htaccess for rules that may forbid crawlers
# Look for overly broad Deny / RewriteRule [F] directives, e.g.:

RewriteCond %{HTTP_USER_AGENT} bot [NC]
RewriteRule .* - [F,L]   # blocks anything with "bot" in UA -- too broad

# After loosening, confirm a clean 200:
curl -I https://example.com/the-fixed-url   # expect HTTP/2 200

If a WAF or security platform is responsible, allowlist verified search-engine crawlers. Once the page reliably returns 200, re-run your audit to confirm the issue has cleared.

Best practice: only 200 canonical URLs in sitemaps

The durable fix is a discipline, not a one-off cleanup. A healthy XML sitemap contains only URLs that return 200, are self-canonical, are indexable, and are not blocked by robots.txt. No 403s, no 404s, no redirect chains, no noindex pages, no parameter duplicates. Each sitemap should also stay within Google's limits of 50,000 URLs and 50MB uncompressed. Validate your sitemap on a regular schedule, ideally as part of every audit cycle, so errors like this are caught before search engines find them.

FAQ