URLs with Exact Same Content: How to Fix Them

What this check flags

This groups URLs whose body content is byte-for-byte identical — the same page reachable at two or more addresses. It is not a copywriting problem and not a penalty; it is a plumbing problem. Google has to pick one URL to index and consolidate signals to, and every extra address dilutes link equity and wastes crawl budget until you tell it which one wins.

A real failing example

The classic offender is the same page served under multiple technically-different URLs. All four of these returned identical HTML with a 200 and no canonical:

http://example.com/product/widget      200  (same bytes)
https://example.com/product/widget     200  (same bytes)
https://www.example.com/product/widget 200  (same bytes)
https://example.com/product/widget?ref=nl 200 (same bytes)

The fix is to declare one canonical URL and make every other address point at it — 301 the protocol/host variants, self-reference the canonical, and let tracking parameters resolve up to the clean URL:

<!-- on the one true page -->
<link rel="canonical" href="https://www.example.com/product/widget">

# server: force one host + protocol
http://  → 301 → https://
example.com → 301 → www.example.com
?ref=… variants → canonical to the clean URL (no separate index)

301 where the duplicate should not exist as its own address; rel=canonical where the URL must stay reachable (parameters, print views) but should not be indexed separately.

301 vs canonical: which lever for which case

How to detect it on your own site

1. Screaming Frog: crawl, open the Content tab and filter to Exact Duplicates. The tool hashes rendered content and lists URL pairs whose hash matches — that is your exact-duplicate list, ready to export.
2. Sitebulb: the Duplicate Content report separates "Exact duplicate" from "Near duplicate" so you can act on the certain cases first.
3. Google Search Console: open the Pages (indexing) report and look for "Duplicate without user-selected canonical" and "Duplicate, Google chose different canonical" — those buckets are Google telling you it found the same content at several URLs.
4. Manual: load the suspected twins side by side and diff the HTML. Identical bytes with two live URLs and no canonical confirms it.

How to fix it

1. Pick the canonical URL you want to keep — the shortest, cleanest, https+preferred-host version.
2. 301 every duplicate that has no reason to exist as its own address.
3. Self-reference rel=canonical on the winner and canonical the must-stay-live duplicates to it.
4. Force one protocol and one host at the server so variants can never resolve 200 again.
5. Re-crawl and confirm the Exact Duplicates filter is empty and GSC reassigns canonicals over the next crawl cycle.

Where exact duplicates come from

Almost none of these are written by hand. They are generated by the way the site serves URLs, which is why the fix lives in server config and templates rather than in the editor. The usual sources:

• Protocol and host sprawl: http and https, www and non-www, all resolving 200 instead of collapsing to one. This is the single most common cause and the easiest to close with two redirect rules.
• Trailing-slash inconsistency: /page and /page/ both live. Pick one form and 301 the other so a crawler never indexes both.
• Query parameters: tracking tags, session IDs, sort and filter values that change the URL but not the body. Canonical these to the clean address.
• Index files exposed: /dir/ and /dir/index.html returning the same page. Redirect the file form to the directory form.
• Staging or CDN hosts leaking into the index: a subdomain or preview host that mirrors production. Block it in robots or password-gate it.

Map the source before you act. If the duplicates come from parameters, no amount of per-page canonical editing helps until you handle parameters as a class. Fix the pattern, not the pages.

Related checks

Exact duplicates caused by URL variants are covered from the parameter angle in duplicate URLs (technical) and the consolidation playbook in URLs with duplicate content. When the pages are close but not identical, you have a judgment call — see similar content. Tag-level twins are handled in duplicate title tags. Reference: the canonical tag glossary and the full canonical tags reference.

FAQ

Does exact duplicate content trigger a Google penalty?

No. Google treats it as a canonicalisation task, not a violation. It picks one URL to index; the cost is split signals and wasted crawl budget, not a manual action.

Should I redirect or canonicalise the duplicate?

301 if the duplicate URL has no reason to stay live. Use rel=canonical if the URL must remain reachable (parameters, print views) but should not be indexed on its own.

Will Google just pick the right canonical on its own?

Sometimes, but it may choose a URL you did not want. Declaring the canonical yourself removes the guesswork and keeps signals on the page you actually want ranked.

Two products are genuinely identical — is that duplicate content?

If they live at separate URLs with the same body, yes, for crawl purposes. Consolidate to one, or differentiate the pages if both need to exist for merchandising reasons.

Do URL parameters create exact duplicates?

Often. Tracking, session, and sort parameters can serve the same body under many addresses. Canonical them to the clean URL so only one version gets indexed.