The Index and What Gets Excluded

Why this matters

Not every URL Google crawls ends up in the index — pages can be excluded for dozens of reasons, and most site owners do not know it is happening. Understanding what triggers exclusion (noindex, soft 404, duplicate content, low quality signals) gives you a diagnostic framework for pages that should rank but do not.

Try it yourself

1. In Google Search Console, open Index → Pages and look at the 'Why pages aren't indexed' breakdown — identify the top three exclusion reasons for your site and write down what each one means.
2. Pick one URL listed under 'Excluded' and run it through the URL Inspection tool to see Google's live assessment — note whether the exclusion reason is a tag, a server response, or a quality signal.
3. Check five of your lowest-traffic blog posts for a noindex meta tag using curl https://yourpost | grep 'noindex' — confirm none were accidentally excluded by a plugin or CMS setting.

References

• Google Search Central — Reasons a Page May Not Be Indexed