2.6.1 What Is a Crawl Budget?
Crawl budget is the number of pages on your website a web crawler will crawl within a given timeframe.
Each time you hit the publish button, Google needs to crawl and index the content for it to start appearing in search results. Given the scale and volume of content on the internet, crawling becomes a valuable resource that needs to be budgeted and rationed for the most efficient use.
To put it simply, it’s hard for Google to crawl and index every single page on the internet everyday. So, Google crawls each website according to its assigned budget.
2.6.2 How Is Crawl Budget Assigned to Websites?
Crawl budget is assigned to websites based on two factors — crawl limit and crawl demand.
Crawl Limit
This is a website’s capacity and/or willingness to be crawled.
Upgrade to a premium plan to access the content.
COURSE ONLY (1 payment)
$695
- 7 chapters
- 53 modules
- 20+ hours of educational videos
- Monthly 1-on-1 coaching
- Monthly group office hours
- Ready-to-use templates
- Members-only articles
Fixed Price Guarantee
Upfront (1 payment)
$1500
- 7 chapters
- 53 modules
- 20+ hours of educational videos
- Monthly 1-on-1 coaching
- Monthly group office hours
- Ready-to-use templates
- Members-only articles
Fixed Price Guarantee
2 quarterly payments of
$750
- 7 chapters
- 53 modules
- 20+ hours of educational videos
- Monthly 1-on-1 coaching
- Monthly group office hours
- Ready-to-use templates
- Members-only articles
Fixed Price Guarantee
4 quarterly payments of
$375
- 7 chapters
- 53 modules
- 20+ hours of educational videos
- Monthly 1-on-1 coaching
- Monthly group office hours
- Ready-to-use templates
- Members-only articles
Fixed Price Guarantee