What Scraping 1.1 Million Search Results Taught Us About SEO

Authority Hacker conducted a massive study scraping and analyzing 1.1 million search results to identify patterns in what ranks and why. The research provides insights from one of the largest SERP analysis datasets assembled by an independent publisher.

Domain Rating Correlation

The study confirmed strong correlation between domain rating (DR) and rankings. Higher DR domains dominated top positions across most query types. However, the correlation was not absolute; lower DR sites could compete for less competitive queries or those requiring specialized expertise that larger sites hadn't targeted.

Content Age Patterns

First-page results averaged over 2 years old, consistent with other studies. However, certain query types, particularly news-related and trending topics, showed much fresher content in top positions. Understanding whether a query favors fresh or established content is crucial for realistic ranking expectations.

Word Count Analysis

Average word count for top-ranking content was higher than commonly assumed minimum thresholds. However, the research cautioned against interpreting this as causation. Longer content may rank better because comprehensive coverage naturally attracts more links and engagement, not because word count itself is a ranking factor.

Link Diversity Findings

Referring domain diversity showed stronger correlation with rankings than total backlink count. Sites with links from many different domains outranked sites with more total links concentrated from fewer sources. This supports diversified link building strategies over volume-focused approaches.

Source: Authority Hacker