SEO & Growth
Index Bloat Is a Crawl Budget You Already Spent
Faceted URLs are not free. Every one of them is a request against the same crawl budget your best pages need.
Most teams think of Google's index like infinite cloud storage: if you have four hundred thousand faceted filter URLs, why not let Google index all of them? You might capture long-tail search traffic.
Google does not think like cloud storage. Google thinks in crawl budgets and host load.
The math of a crawl budget
A crawler assigns your domain a finite number of requests per day based on your server response time, popularity, and historical content quality. If Googlebot allocates ten thousand fetches a day to your host, every faceted URL it crawls is a real page it did not look at.
Daily crawl capacity: 10,000 URLs
Faceted / junk URLs: 8,200 requests (82%)
Updated core products: 1,800 requests (18%)
Result: Key product updates take 6 weeks to re-index.How index bloat compounds
Index bloat is insidious because it degrades the quality score of the entire host. When Google crawls eighty thousand near-duplicate colour-and-size variations and finds identical body copy with slight parameter shifts, the site's overall quality score drops. Your core pages lose rank not because they are bad, but because the host is 90% thin content.
The clean remediation pattern
- Robots.txt disallows: block parameter combinations that have zero commercial search intent.
- Canonical consolidation: point multi-facet variations cleanly back to the root category.
- HTTP 410 Gone: aggressively purge expired or discontinued parameters rather than soft-404 redirects.
- Inspect Search Console crawl stats: verify that server response times drop as junk requests decline.