Crawl Budget (33 articles)

Three step diagram showing how to find crawl waste, apply crawl controls, and free crawl budget for valuable pages.

How we optimized our Crawl Budget

A detailed case study on how Skroutz.gr, over 1.5-years, dealt with index bloat and crawl budget.
Learn More
Speeding up stubborn url migrations | ohgm

Speeding Up Stubborn URL Migrations | ohgm

A case study, and experiment, to identify stubborn old URLs that need to be deindexed
Learn More
Leverage pyramid for huge websites from template level down to page level, beside an operating model checklist.

Behemoth SEO: Search Strategy for Huge Websites

A case study of what happens when you remove duplicate or low-quality content from enterprise sites.
Learn More
Diagram showing crawl rate limit and crawl demand combining into crawl budget, the pages googlebot fetches per period.

What Is Crawl Budget?

An explanation of crawl budget with data and a smaller case study to show the impact…
Learn More
Sequence diagram of googlebot sending an if none match header and the server replying 304 not modified or 200 ok with a fresh etag.

Does Googlebot Use Etag Headers?

An experiment to detect if Googlebot uses Etags to help boost crawl budget efficiency.
Learn More
How we increased organic traffic (4x) with technical seo ξ ftf

How We Increased Organic Traffic (4X) With Technical SEO Ξ FTF

A case study showing how Technical SEO can help drive growth
Learn More

Get new blog posts by email: