When a page does not show up in Google, the instinct is to ask "why is it not ranking?" That is usually the wrong question. Before a page can rank it has to be discovered, fetched, rendered, judged worth keeping, and stored under the right URL. Most of the problems I diagnose live in those earlier stages, and the fix is completely different depending on which one.
This is part of my SEO basics guide. It is the mental model I use before touching anything on a site, and the sequence I follow when something has gone wrong.
- Crawl
- Render
- Index
- Rank
Stage one: crawling
Crawling is discovery plus fetching. Googlebot keeps a list of URLs it knows about, decides which to visit and how often, requests them, and adds any new URLs it finds in the response to the list. URLs enter the list through links from already-known pages, XML sitemaps, redirects, and occasionally other signals such as URLs seen in Chrome or submitted through Search Console.
How Googlebot behaves
A few properties of Googlebot explain most crawl problems. It obeys robots.txt, so a disallow rule stops it fetching the page, though the URL can still be indexed from links alone. It crawls with a mobile user agent, so what your site serves to a phone is what Google sees. It will not log in, fill forms or click buttons. It follows anchor elements with an href; a div with a click handler is not a link. And it budgets attention per host, so on large sites not every URL is fetched every time.
What goes wrong at the crawl stage
Pages with no internal links pointing to them, so Googlebot never finds them. Sections blocked by a robots.txt rule added during staging and forgotten. Faceted navigation generating millions of filter URLs that soak up crawl capacity. Servers returning 5xx errors under crawler load, which makes Google back off. Redirect chains that end somewhere unexpected. None of these is visible if you only look at rankings.
The crawlability and indexability sub-guide goes deeper, and the orphan pages experiment shows what happens when you deliberately fix the "no links" problem on a set of pages.
Stage two: rendering
The fetched HTML is often not the page. On sites built with React, Vue, Angular or any client-side framework, the initial HTML may be an empty shell whose content only appears once JavaScript runs. Google handles this by rendering pages in a headless browser, executing the JavaScript, and processing the resulting DOM.
The render queue
Rendering is expensive, so Google does it in a second wave. The raw HTML is processed first; the page enters a render queue; it is rendered when resources allow. The delay is usually short but not assured, so anything that only exists after JavaScript runs is in a weaker position than content in the initial HTML. Links injected by JavaScript are discovered later.
What goes wrong at the render stage
JavaScript files blocked by robots.txt, so the renderer cannot execute them. API calls that need cookies or auth. Content that only loads on scroll or click. Timeouts on heavy pages. Canonical tags or noindex directives that differ between raw HTML and rendered DOM, leaving Google to choose which to believe.
Stage three: indexing
Indexing is not automatic. After rendering, Google analyses the content, checks for duplicates, and decides whether to store the page and under which URL. A fetched page that Google declines to index is a common and often misread outcome.
Index selection
Google does not index everything it crawls. Near-empty pages, duplicates, pages of low apparent quality relative to the rest of the site, and pages marked noindex are dropped or never admitted. On large sites, index selection is effectively a quality filter. Thin programmatic pages are the classic victim, which is the argument in programmatic SEO is not a shortcut.
Canonicalisation
When several URLs have the same or very similar content, Google groups them and picks one canonical URL to represent the group in results. Your rel="canonical" tag is a strong hint, not a command. Google weighs it against redirects, internal link patterns, sitemap inclusion, HTTPS versus HTTP, and which version it saw first. If your signals disagree, Google will choose, and it may not choose the URL you wanted. The canonical tags sub-guide covers how to make the signals agree.
What goes wrong at the index stage
"Crawled, currently not indexed" on pages the business cares about: usually a quality or duplication judgement. "Discovered, currently not indexed": Google knows the URL but has not prioritised fetching it. "Duplicate, Google chose different canonical than user": your hints are contradicted by something else. A noindex tag left over from staging. Soft 404s that return 200 but look empty.
Stage four: ranking
Only indexed pages can rank. When someone searches, Google retrieves candidates from the index and orders them using many systems working together. You do not need their internal names; you need to know what each is trying to judge.
Ranking systems in plain language
| What the system is trying to judge | What it looks at | What you control |
|---|---|---|
| Does this page mean the same thing the query means? | Language models that understand words in context, synonyms, entities and relationships | Clear, specific, well-structured writing that matches how people phrase the need |
| Does it satisfy the intent behind the query? | The type of result people want for this query: a product, a guide, a map, a video | Choosing the right page type; see search intent |
| Is it original and helpful rather than made for search engines? | Site-wide and page-level quality signals, depth, first-hand information, user behaviour over time | Publishing fewer, better pages with real expertise |
| Is the source credible on this topic? | Links from other sites, mentions, consistency of coverage, author and brand signals | Earning references; building topical authority |
| Is the experience acceptable? | Mobile usability, HTTPS, Core Web Vitals, intrusive interstitials | Technical hygiene; see Core Web Vitals |
Ranking is a weighted combination of these answers, not a checklist. A technically imperfect page can still win by being the clearly best answer, and most of what you control is upstream: the page type you build, the depth you write at, the links you earn.
What goes wrong at the rank stage
The page targets a query whose intent differs from what the page offers. Two of your own pages compete for the same query and split the signals, the cannibalisation problem addressed in keyword mapping. The page is thinner than the current top results. The site has no credibility on the topic yet. Or the page is simply new, competing against pages with years of accumulated signals.
Seeing each stage in Search Console
Search Console is where Google tells you directly what it did with your pages. Each stage has a view.
| Stage | Where to look | What to check |
|---|---|---|
| Crawl | Settings, then Crawl stats; also the URL Inspection tool | Total crawl requests over time, response codes, host status; for a single URL, the last crawl date and whether fetching succeeded |
| Render | URL Inspection, then Test Live URL, then View Tested Page | The rendered HTML and screenshot; compare them to what a browser shows; look for blocked resources and JavaScript errors |
| Index | Indexing, then Pages report; URL Inspection for the user-declared versus Google-selected canonical | Which reasons are keeping pages out; whether the Google-selected canonical matches yours |
| Rank | Performance report, filtered by page and query | Impressions and average position by query; whether the right page is appearing for the right query |
Using the pipeline as a diagnostic order
When a page underperforms, I check the stages in order and stop at the first failure. Is it crawled, and recently? Is the rendered content what we intended? Is it indexed, under the canonical we chose? Only then do I look at ranking. This is why I run a technical audit before any content programme, using the technical SEO audit framework: content built on a site with crawl or index problems never gets the chance to compete.
Ranking is the last stage of a pipeline. Most pages that fail to rank never got there.
Frequently asked questions
How long does it take Google to index a new page?
Anywhere from hours to weeks, depending on how often Google crawls your site, how the page is linked internally, and whether Google judges it worth indexing. A page linked prominently from a frequently crawled homepage is found quickly; a page only in a sitemap on a low-authority site may wait. Submitting through URL Inspection can speed up discovery but does not force indexing.
Why does Search Console say "Crawled, currently not indexed"?
Google fetched the page and decided not to store it, at least for now. On most sites this is a quality or duplication signal: the page is too similar to another, too thin, or the site has many such pages. Improving the page and the internal links to it is the usual fix; resubmitting an unchanged page rarely changes the outcome.
Does Google index JavaScript-rendered content?
Yes, but in a second wave and with more that can go wrong. Content present in the initial HTML is processed earlier and more reliably. For anything the page needs to rank on, I recommend server-side rendering or pre-rendering so it does not depend on the render queue.
Why does Google choose a different canonical than the one I set?
Because your canonical tag is one signal among several, and the others disagreed. Common causes are internal links pointing to the non-canonical version, the non-canonical version being in the sitemap, a redirect in the other direction, or the two pages not actually being duplicates. Make every signal point the same way.
Where this connects
This sub-guide sits under the SEO basics guide and is the model behind my technical SEO audit framework, which checks the stages in the same order. The render stage is tested directly in the JavaScript rendering test, and the technical SEO and content architecture case study shows a site where crawl and index problems were fixed before content was built on top. If you need these stages diagnosed and fixed on your own site, that is what my technical SEO service does.