Duplicate URLs (Technical): How to Consolidate Them

What this check flags

This check fires when the same content is reachable at several different URLs because of technical variations — a trailing slash here, an uppercase letter there, a tracking parameter, or the http and https versions both resolving. Each variant is a distinct URL to a crawler, so Google can split ranking signals across them, waste crawl budget on near-identical copies, and sometimes index the wrong one. This is a canonicalisation and redirect problem, not a "rewrite your content" problem: the words are fine, the addresses are a mess.

A real failing example

An e-commerce category page was live at half a dozen addresses that all returned 200 OK with byte-identical HTML:

https://example.com/shoes/running/        ← the one you want
https://example.com/shoes/running         ← no trailing slash
https://example.com/Shoes/Running/        ← mixed case
http://example.com/shoes/running/         ← insecure protocol
https://example.com/shoes/running/?ref=nav
https://www.example.com/shoes/running/    ← www vs non-www

Every one of those served a 200 and carried a self-referencing canonical, so Google saw six competing pages. Internal links pointed at a mix of them, scattering link equity. The fix was to pick one canonical form and enforce it with 301 redirects at the server, plus a consistent canonical tag pointing home:

# Force https, non-www, lowercase, trailing slash → one canonical URL
RewriteEngine On
RewriteCond %{HTTPS} off [OR]
RewriteCond %{HTTP_HOST} ^www. [NC]
RewriteRule ^ https://example.com%{REQUEST_URI} [R=301,L]

# On every page, one canonical that matches the enforced form:
<link rel="canonical" href="https://example.com/shoes/running/">

After deploy, the five variants returned 301 → https://example.com/shoes/running/, and a crawl confirmed only one indexable version remained. Parameters like ?ref=nav were handled by keeping the canonical clean so the parameterised URL folded into the canonical rather than competing with it.

The common variant types and their fix

Rule of thumb: if the two URLs are technically interchangeable and you never want both indexed, use a 301 redirect so only one is ever served. If the variant needs to stay reachable for users (a legitimate filtered or tracked URL), use a canonical tag pointing at the clean version instead.

How to detect it

1. Screaming Frog. Crawl the site, then check the URL tab for the Uppercase and Parameters filters, and the Response Codes tab to confirm variants return 200 instead of redirecting. The Canonicals tab shows where variants self-canonicalise instead of pointing to one form.
2. curl the variants directly. Test each form and read the status code:

   curl -s -o /dev/null -w "%{http_code} %{redirect_url}n" -I 
     http://example.com/shoes/running
   # want: 301 https://example.com/shoes/running/

   A 200 where you expected a 301 means the variant is a live duplicate.

3. Google Search Console. The Pages (indexing) report has a "Duplicate without user-selected canonical" and "Alternate page with proper canonical tag" bucket — open those to see which variants Google is grouping and which one it picked.
4. site: search + URL Inspection. Run site:example.com/shoes/running to see if multiple variants are indexed, then use URL Inspection on a variant to read Google's chosen canonical.

How to fix it

1. Choose one canonical form for the whole site — protocol, host, case, and trailing-slash policy — and write it down so it's enforced consistently.
2. 301-redirect the interchangeable variants at the server (rules above). Verify each variant now returns a single 301 to the canonical, not a chain. See trailing slash redirects and URL works on HTTP and HTTPS for the two most common offenders.
3. Canonical the parameter variants that must stay live (tracking, session, sort) to the clean URL — details in the URL parameters reference.
4. Fix internal links. Point every internal link at the canonical form so you stop generating variant traffic in the first place — redirects are a safety net, not a substitute for clean links.
5. Update the sitemap. List only canonical URLs; drop every variant.

FAQ

How is this different from "URLs with duplicate content"?

This check is about the same content served at multiple technical variants of one address — a canonicalisation/redirect fix. URLs with duplicate content is about substantively similar body text on genuinely different pages, which is a consolidation/merge decision. Different problem, different fix.

Should I use a canonical tag or a 301 redirect?

301 when the variant should never be visited (case, protocol, www, trailing slash) — it removes the duplicate entirely. Canonical when the URL must stay reachable for users or tracking but shouldn't compete for indexing. When in doubt, a 301 is the stronger, cleaner signal.

Do URL parameters always create duplicates?

No — a parameter that changes the content (like ?page=2) is a distinct page. The duplicates are parameters that don't change the meaningful content: tracking, session IDs, and re-sorts of the same items. Canonical those to the clean URL.

Does the trailing-slash choice matter for SEO?

The choice itself doesn't — slash or no slash is fine. What matters is that you pick one and enforce it, so the same page never lives at both /page and /page/ returning two 200s.

Related: Canonical Tags: The Complete Reference