Crawling
Log file analysis means looking at your server's records of every visit, including visits from Google's…
Comprehensive guide to robots.txt covering implementation, best practices, troubleshooting, and advanced techniques.
A cloaking case study seeing if Google penalizes the website if it were intentionally decieved.
A case study from Sistrix on the impact of Twitter blocking Google from crawling and indexing.
Crawling is when Google's robot visits your website and reads all your pages. It's like a…
An experiment to find out how long it would take Googlebot to crawl JavaScript vs HTML…
Get new blog posts by email:
Technical SEO consulting and GEO strategy for teams who need real implementation support. 20 years of enterprise experience at Indeed, Yahoo, Oracle, McClatchy, and Toyota. Case studies, research, and resources for search and AI visibility.