URLs with Duplicate Content: How to Consolidate Them

What this check flags

This check fires when two or more genuinely different URLs carry substantively the same body content — not a trailing-slash variant of one address, but real pages whose main text overlaps heavily. That splits ranking signals across near-twins, makes Google guess which one to show, and burns crawl budget re-reading the same paragraphs. The fix is consolidation, but with a rule this site holds firmly: enrich and merge, never just delete. Thin near-duplicates get folded together and made richer, not silently binned.

A real failing example

A B2B site had three pages targeting almost the same intent, each thin and 60–80% overlapping:

/services/crm-consulting/          (620 words)
/crm-consulting-services/          (580 words, near-identical body)
/solutions/crm-implementation/     (540 words, same three paragraphs reused)

All three ranked on page 2 for the same query and cannibalised each other — Google kept swapping which one it showed, and none had the depth to break through. Deleting two would have thrown away the internal links and the little authority each had. Instead:

1. Keep /services/crm-consulting/ as the canonical target.
2. Merge the unique bits (a case-study snippet, an FAQ, a pricing note)
   from the other two INTO it → ~1,600 words, genuinely deeper.
3. 301-redirect the two weaker URLs to the survivor.
4. Repoint every internal link to the survivor.

One stronger page replaced three weak ones. Nothing of value was thrown away — the good paragraphs from the retired URLs were carried into the survivor, and their link equity flowed in through the 301s. Within a couple of crawl cycles the consolidated page moved from page 2 to the top of page 1 for its target query.

Consolidate, canonicalise, or leave alone?

The deciding question is intent. If two pages answer the same user question, consolidate. If they answer different questions but happen to share boilerplate, make each one distinct rather than merging — that's the enrich path, and it's the right call for location and template pages.

How to detect it

1. Screaming Frog. Enable near-duplicate detection in the crawl config (it uses a similarity threshold), run the crawl, then open the Content → Near Duplicates report. It lists URL pairs and a similarity percentage so you can see exactly which pages overlap and by how much.
2. The site's duplicate-content tool. Paste suspected pages into the duplicate content checker to get a similarity score between two URLs before you decide to merge.
3. Search Console cannibalisation signal. In the Performance report, filter to a query and look at the Pages tab — if several of your URLs collect impressions for the same query and swap positions over time, they're competing near-duplicates.
4. Exact-phrase site search. Copy a distinctive sentence and search site:example.com "that exact sentence". Multiple results means that block is duplicated across pages.

How to fix it

1. Pick the survivor. Choose the URL with the best links, cleanest slug, and strongest existing rankings to be the consolidation target.
2. Merge, don't delete. Pull every unique, useful passage from the weaker pages into the survivor so it ends up genuinely deeper than any single original. This is the enrich-and-merge rule — the retired pages donate their content, they aren't just erased.
3. 301 the retired URLs to the survivor so their link equity and any bookmarks/backlinks carry over.
4. Repoint internal links and the sitemap at the survivor so nothing keeps feeding the retired URLs.
5. For distinct-intent duplicates, differentiate instead. Add unique, specific content to each page rather than merging — for filtered/parameter-driven near-twins, see faceted navigation SEO.

FAQ

Is duplicate content a Google penalty?

No — there's no dedicated "duplicate content penalty" for ordinary overlap. What actually happens is milder but real: Google filters near-identical URLs down to one in results and splits your signals across the copies, so you underperform. The remedy is consolidation, not fear. The full picture is in Duplicate Content: Causes, Myths, and Fixes.

Should I just delete the weaker pages?

No. Deleting throws away their content and link equity and can strand backlinks on a 404. Merge their unique value into the survivor and 301-redirect them — enrich and consolidate, never just prune.

How is this different from "duplicate URLs (technical)"?

This check is about real, distinct pages with overlapping body content — a consolidation decision. Duplicate URLs (technical) is about one page reachable at multiple technical addresses — a redirect/canonical fix. Fix the technical variants first; then look at genuine content overlap.

How similar is "too similar"?

There's no official percentage, but pages sharing most of their main body and targeting the same intent are the ones to consolidate. Use a similarity score as a starting filter, then judge by intent — two pages at 70% overlap serving different searches can both stay if you differentiate them.

Can canonical tags replace merging?

A canonical helps when both URLs must stay live (syndication, print versions), but it doesn't make either page stronger. When two thin pages target the same intent, merging into one richer page beats canonicalising two weak ones.