Crawl Budget

Crawl budget is the amount of crawling a search engine will do on your site over a given period. It is shaped by two things: crawl capacity (how much your server can handle without slowing down) and crawl demand (how much Google wants to crawl, based on popularity and how often content changes).

It matters most for large sites — tens of thousands of URLs or more. If crawlers spend their budget on low-value URLs (faceted-navigation duplicates, endless parameter combinations, redirect chains), important new pages get discovered slowly.

You improve effective crawl budget by removing crawl traps, consolidating duplicates with canonical tags, returning fast responses, and keeping your sitemap clean so crawlers focus on canonical, indexable URLs. Small sites rarely need to worry about it — Google can usually crawl them fully.