Google updates crawl budget doc with conservative default

Google has rewritten its crawl budget documentation, and the change that matters most for large sites is now stated plainly: every site starts on the same conservative crawl limit, and Google raises it only if demand is there and the site stays healthy. The updated page also spells out that this limit is shared across all of Google’s crawlers, so one hungry crawler can leave less capacity for the rest.

What happened

Google updated its “Optimize your crawl budget” page on 22 July 2026. The refreshed documentation now says: “Every site starts with the same default, conservative crawl capacity limit. If there is demand to crawl more and the site remains healthy, Google’s systems will automatically adjust this limit over time.”

It also clarifies how that budget is divided. According to the page, “While each crawler has a different crawl demand, the crawl capacity limit is shared across all crawlers. This means that high demand from one crawler can reduce the capacity available for others.” Googlebot, AdsBot and the Google Shopping crawler all draw from the same pool, and Google notes AdsBot and Shopping demand climbs when a site runs dynamic ad targets or has products in its merchant feeds.

Google reiterates who should care. Its own guidance limits the topic to “Large sites (1 million+ unique pages) with content that changes moderately often” and “Medium or larger sites (10,000+ unique pages) with very rapidly changing content.” Smaller sites, it says, are generally crawled efficiently already. The update also leans harder on server health, recommending faster server response times and support for the 304 (Not Modified) status code so Google can reuse a cached copy of a page that has not changed.

Why it matters

Crawl budget decides how quickly Google discovers new pages and refreshes existing ones. For a brand running thousands of location pages, a store locator, and a product catalog on one domain, that speed is the difference between a new store showing accurate hours the day it opens and lagging for a week. A conservative starting point plus shared capacity means the crawl budget is a resource to manage, not an entitlement, and a site that wastes it on duplicate or slow pages has less left for the pages that earn revenue.

What this means for multi-location brands

For a central marketing or web team, the job is to stop spending crawl budget on pages that do not need it and speed up the ones that do. That means keeping location and product data clean at the source so pages are not thrashing between versions, retiring or consolidating thin and duplicate URLs (faceted filters, tracking parameters, near-empty location stubs), and making sure the server answers fast and returns 304 when nothing has changed. Consistent, well-structured local business listings and a properly indexed store locator are what let Googlebot spend its limited budget on the pages that convert. Treat this as an enterprise SEO governance task across the whole estate, not a page-by-page fix.

Every site starts with the same default, conservative crawl capacity limit. If there is demand to crawl more and the site remains healthy, Google’s systems will automatically adjust this limit over time.

Google for Developers

The bottom line

Google has now put the rules in writing: budgets start small, grow with proven demand, and are shared across every crawler hitting your domain. For large multi-location sites, the winning move is unglamorous: faster servers, cleaner URLs, and accurate data at the source, so the pages that matter get crawled first.