The brief on a large site almost always says “more”. More categories, more locations, more languages, more posts. And the site almost always has the opposite problem: thousands of URLs that no buyer ever asked for, competing with the few hundred that carry the business. On sites like that, the most valuable SEO work of the year is usually a deletion list.
What crawl budget costs you
A page is not free once it exists. Crawl budget is the amount of a site an engine is willing to fetch in a given period, and every URL spends some of it. Google’s own crawl budget guidance is blunt about the consequence: time spent on URLs that should not be crawled is time not spent on the rest of the site. Every near-duplicate splits links and signals with its twin. Every thin page lowers the average quality of the domain the engine is forming an opinion about. And every page is one more thing that goes stale, because nobody owns it.
The same arithmetic applies to the AI crawlers, which fetch less and forgive less. On a site of two hundred pages none of this matters. On a site of two hundred thousand it is the whole game.
Six page types that exist for no one
Together they are what index bloat means in practice: an index several times larger than the useful page count. Almost every large site has some of these. Few have owners for any of them.
- Filter and parameter URLs. Colour, size, sort order, session IDs. One product list becomes five thousand URLs. The engine sees five thousand copies of the same page.
- Location pages generated from a template. “Plumbing in [city]” times four hundred cities, with the city name as the only difference. These rank for nothing and describe nothing.
- Thin translations. A page translated because the site is multilingual, not because that market searches for it. It competes with the original and loses. Which pages deserve to exist per market is decision three in five decisions to make before entering a new market.
- The old blog. Eight years of posts written for keywords nobody remembers, many of them saying the same thing as each other. Three of them rank. The rest dilute those three.
- Docs, help and marketing answering the same question. Three systems, one query, and the wrong one ranking. I covered this for product companies in SEO for SaaS, and it gets worse with scale.
- Pages that exist because a department wanted them. A landing page for a campaign that ended, a microsite for an event, a “solutions” tree built for an org chart rather than a search.
Content pruning: merge, delete or hide, not “keep”
For every group of pages the question is the same: does this URL answer a question someone types, better than another URL on this site does. If yes, keep it and improve it. If no, there are three honest options.
- Merge. Several weak pages on one topic become one strong page, and the old URLs redirect to it. This is the usual answer for the old blog and for template pages.
- Delete. Nothing worth keeping, no links pointing in, no traffic. Return a real 404 or 410. The fear of deleting is almost always larger than the cost.
- Hide. The page has to exist for users but should not be in the index. Filters and internal search results live here.
noindex, or a canonical to the parent, depending on the case.
“Keep, just in case” is not an option, because that is the decision that produced the problem.
How to remove pages without losing what works
Removal at scale is a migration in everything but name, and it carries the same risk: rankings lost to a redirect map written too late. The checks are the ones in SEO migration without losing rankings, with one addition. Before touching anything, pull every URL that received a click or a link in the last twelve months. Those are protected. Everything else is a candidate.
Then work in batches, measure after each one, and stop if anything protected moves.
What to measure
Not the number of indexed pages, which only goes down and tells you nothing. Three things instead:
- Share of organic sessions landing on pages you decided to keep. It should rise as the noise leaves.
- Impressions on the protected list. Flat or up. If it drops, a redirect is wrong.
- Crawl stats in Search Console. Fewer requests to junk URLs, more to the pages that matter. This is the earliest signal, and usually visible within weeks.
Where the pushback comes from
Nobody in the building gets credit for deleting pages. Content teams are measured on output, product teams on features shipped, regional teams on having a local site. The person who says “fewer” is arguing against everyone’s incentives at once, and needs numbers to do it.
That is most of what I do as an enterprise SEO consultant on large sites: not writing the fortieth page, but building the case for the two thousand that should go, and then removing them without breaking what works. If your site has more pages than anyone can list, tell me how many pages you have, and I will tell you roughly how many of them are working against you. If the site is small and the audit is long, read what technical SEO can and cannot do first.