Increase organic traffic exponentially using keyword clusters

A note on the source

This post was originally filed as a summary of an article on keyword clusters published by Keyword Insights, linked in
the source line at the foot of this page. That URL now returns HTTP 404 and the article is not retrievable at its original
address. Rather than paraphrase claims we cannot verify, the method below is written from practice: it is the standard SERP
overlap approach that Keyword Insights and comparable tools implement, described so you can execute or reproduce it
yourself. No figures, thresholds or results are attributed to the original article, because we cannot confirm what it
said.

Why clustering beats keyword lists

The default content planning workflow produces a spreadsheet of keywords sorted by volume, and then one article per row.
On any topic with real depth this reliably creates internal competition. You publish "best running shoes", then "running
shoes for beginners", then "top rated running shoes", and Google has to pick one of the three to show. It picks
inconsistently, rotating between them, and every rotation splits your click-through data and your internal link equity
across three URLs that should have been one.

Clustering inverts the process. Instead of asking what keywords exist, you ask how many distinct jobs those keywords
represent. The answer is almost always a much smaller number than the keyword count, which means fewer pages, each of them
stronger.

Why SERP overlap is the right similarity measure

Most naive clustering methods group keywords by string similarity or by embedding distance. Both fail in predictable
ways. String similarity puts "running shoe repair" next to "running shoes" because they share tokens, when those are
completely different intents. Embedding similarity puts "cheap flights" next to "budget airfare", which is correct, but
also puts "how to fix a leaking tap" next to "how to fix a leaking pipe", which may or may not be one page depending
entirely on what Google is currently rewarding.

SERP overlap sidesteps the question of what the keywords mean by observing what Google already does. If the top ten
results for two queries are largely the same URLs, Google has judged one set of documents to satisfy both. That is an
empirical answer to an otherwise subjective question, and it updates itself as the SERP changes.

The method, step by step

1. Build the keyword set. Pull everything relevant from Search Console, a rank tracker, and competitor
gap analysis. Do not filter by volume yet; zero-volume queries frequently belong in clusters and contribute traffic
collectively.

2. Fingerprint every keyword. For each query, record the ranking URLs in the top ten as an ordered list.
Keep the position, because you will want it later for weighting. This is the only expensive step: it is one SERP request
per keyword, per location, per device.

3. Compute pairwise overlap. For each pair of keywords, count how many URLs appear in both top tens.
A short script does this in seconds:

from itertools import combinations

serps = {
    "running shoes":      ["a","b","c","d","e","f","g","h","i","j"],
    "best running shoes": ["a","b","c","d","x","f","g","h","y","j"],
    "running shoe repair":["p","q","r","s","t","u","v","w","k","l"],
}

for k1, k2 in combinations(serps, 2):
    shared = len(set(serps[k1]) & set(serps[k2]))
    print(k1, "|", k2, "=>", shared, "shared URLs")

4. Apply a threshold and build groups. Treat every pair above the threshold as an edge in
a graph, then take the connected components as clusters. Be aware that naive connected-component clustering chains: if A
matches B and B matches C, A and C end up in one cluster even when they share nothing. On large sets, require that every
member of a cluster meets the threshold against the cluster's highest-volume keyword rather than against any member.

5. Name the primary keyword. Within each cluster, the primary is the term with the highest volume whose
SERP most resembles the cluster average. That term becomes the H1 and the URL slug; the rest become subheadings, FAQ
entries and natural body variations.

Choosing a threshold

Thresholds are practitioner defaults, not published constants. Validate them against a sample of twenty pairs you can judge by hand before trusting them across thousands.

Two refinements pay for themselves. First, weight by position: an overlap in the top three is a stronger
signal than an overlap at positions nine and ten. Second, compare result types, not just URLs. Two queries returning the
same domains but one showing product listings and the other showing how-to videos are not the same intent, whatever the URL
count says.

Turning clusters into pages

A cluster is a page brief, not a page. The mapping that works:

The primary keyword sets the H1 and the slug. Secondary keywords with materially different phrasing become H2 or H3
sections, which is also what makes the page extractable for AI answer engines and featured snippets. Question-form keywords
in the cluster go into the FAQ block verbatim, because they are literally how people phrase the query. Keywords that are
pure syntactic variants need no dedicated treatment at all; they will be matched by the page as written.

Check the result before publishing. Our heading
analyzer shows whether the outline actually reflects the cluster structure, and the
keyword density analyzer is useful in
the negative sense: if one variant dominates the body text, you have written for a term rather than for the intent.

The consolidation case, which is where the traffic actually comes from

Clustering an existing site is usually more valuable than clustering a plan. Map every current URL to the cluster it
serves. Wherever two or more URLs land in the same cluster, you have found live internal competition. Consolidate them:
pick the strongest URL, merge the useful content from the others into it, and 301 the rest.

The gain here is not incremental. Three pages each ranking around position twelve for the same cluster, once merged,
frequently land inside the top five, because the merged page inherits all three link profiles and covers the intent more
completely than any of them did. Verify the redirects resolve in a single hop with our
redirect checker, and use the
content gap finder to confirm the merged
page picked up everything the retired pages covered.

Never delete the losing pages outright. Redirect them, so the links pointing at them keep contributing.

FAQ

Source: https://www.keywordinsights.ai/blog/keyword-clusters/