Crawling (63 articles)

Crawl stats

Crawl Stats

Crawl stats are the data Google reports about its crawling activity on a site, available in…
Learn More
Speeding up stubborn url migrations | ohgm

Speeding Up Stubborn URL Migrations | ohgm

A case study, and experiment, to identify stubborn old URLs that need to be deindexed
Learn More
Diagram showing recovery of indexed pages after a javascript regression, moving from a drop in indexed pages through a raw versus rendered crawl to a server side fix and recovered coverage.

How Walker Sands used Sitebulb to fix JavaScript issues and reverse a huge drop in indexed pages

A case study from a Sitebulb client showing the impact of fixing JavaScript SEO issues.
Learn More
Diagram outlining a five step googlebot log file study, from collecting server logs to steering crawl toward revenue pages.

Here's what happened when I followed Googlebot for 3 months

An experiment which follows how Googlebot discovers and crawls pages on a website.
Learn More
Drag race style timeline comparing time from publish to indexed, where content in the html and server rendered pages finish fast while client side javascript waits through a render queue and finishes later.

The JavaScript Indexing Drag Race - StrategiQ

An experiment to investigate if JavaScript can cause problems for SEO.
Learn More
Diagram showing four technical seo health checkpoints that protect organic visibility: crawlability, indexation quality, site changes and monitoring.

The Story of TheSun.co.uk shows us why SEO is so important - SISTRIX

A case study which breaks down how removing a paywall impacted The Sun's SEO visibility.
Learn More

Get new blog posts by email: