The blog
Crawl budget
Crawl budget explained through server logs: how Google rations crawling, what your access logs reveal and what to fix.
Crawl budget
Crawl budget on a brand-new site: 1,722 crawler requests, analysed
We logged every crawler hit on our site during its first two weeks. 1,722 requests later: what our logs confirm, qualify and refute in Google's July 2026 crawl budget documentation.
Jul 27, 2026
Crawl budget
Crawl Budget: What Google's Rewritten Documentation Actually Changes
Google rewrote its crawl budget documentation on July 22, 2026. Shared capacity across all crawlers, an identical starting limit for every site, HTTP caching: what changed and what to check in your logs.
Jul 26, 2026