# Crawl budget

> How much of your site Googlebot is willing to fetch — only a constraint on large sites, but a real one there.

Googlebot fetches with finite patience: how many URLs it will crawl per visit depends on server speed and how much the site is judged worth fetching. Small sites never touch the limit — a few hundred pages all get seen. It becomes a constraint on sites with thousands of URLs, where crawl capacity spent on parameter URLs, faceted pages, tag archives and redirect chains is capacity not spent on pages that matter.

The maintenance is housekeeping: let robots.txt and canonicals keep junk URLs out, keep internal links pointing at final destinations instead of redirecting hops, keep the sitemap clean of 404s, and keep the server fast — crawl rate adapts to a site that answers quickly.

## Related terms

- https://dfieldsolutions.com/en/glossary/robots-txt.md
- https://dfieldsolutions.com/en/glossary/indexing.md
- https://dfieldsolutions.com/en/glossary/xml-sitemap.md
- https://dfieldsolutions.com/en/glossary/internal-linking.md

---

Source: https://dfieldsolutions.com/en/glossary/crawl-budget
DField Solutions — Dunakeszi, Hungary — dezso@dfieldsolutions.com
Booking: see https://dfieldsolutions.com/en/contact