Crawling (63 articles)

Diagram showing three ways to hide text with display none, visibility hidden, and a javascript toggle, and how each is treated for crawling and indexing.

Long Term CSS Javascript Hidden Text Experiment

An experiment that shows Google gives less weight to text hidden via CSS and JavaScript.
Learn More
Log file analysis study: what crawl data reveals

Log File Analysis Study: What Crawl Data Reveals

Research on log file analysis SEO reveals key patterns and actionable insights for SEO practitioners.
Learn More
Crawl rate

Crawl Rate

Crawl rate is how frequently a search engine's crawler sends requests to a site, usually expressed…
Learn More
Understanding googlebot's crawl budget allocation

Understanding Googlebot's Crawl Budget Allocation

Crawl Budget represents a critical component of modern SEO strategy. As search engines continue to evolve…
Learn More
Diagram of an xml sitemap index file referencing child sitemaps for products, posts, and pages within the 50,000 url and 50 mb protocol limits.

XML Sitemaps: The Complete Implementation Guide

Comprehensive guide to XML sitemaps covering implementation, best practices, troubleshooting, and advanced techniques.
Learn More
Comparison diagram of 404 not found, 410 gone, and 301 redirect responses and how each affects indexing.

404 vs 410 - The Technical SEO Experiment

An experiment showed 404 URLs were crawled more frequently than 410 URLs.
Learn More

Get new blog posts by email: