Skip to content

Why Google Isn’t Indexing Your WordPress Site: A Hosting and Crawlability Checklist

A website owner and web professional review WordPress site diagnostics on a laptop, with abstract visual cues for crawl access, redirects and server responses.

A missing Google result is a symptom, not a diagnosis. A WordPress page may be undiscoverable, blocked from crawling, marked noindex, redirected elsewhere, treated as a duplicate, intermittently unavailable—or indexed but not visible for the searches you expect.

Just as importantly, not indexed is not automatically a hosting problem, and being crawled does not guarantee rankings. This checklist helps Canadian organizations, site owners and web designers separate the likely causes before changing settings or moving a website.

Start here: what is the actual problem?

Before troubleshooting, identify which of these situations applies:

  • Not indexed: Google has not added the URL to its searchable index.
  • Indexed but not ranking: The URL can be found with a direct search, but has little visibility for relevant searches.
  • An old version is shown: Google has not yet reflected a recent change, redirect or migration.
  • Only part of the site is affected: A section, template or page type may have a technical setting different from the rest of the site.

Search for the exact URL in Google and try site:example.com, replacing the example domain with your own. Then use Google Search Console:

  • URL Inspection for a specific important URL.
  • Page Indexing reports for patterns across the site.
  • Crawl Stats when availability, response time or server errors may be involved.

A sitemap submission or an indexing request can help Google discover a URL, but neither guarantees inclusion, timing or rankings.

A quick indexing decision tree

  1. Can Google discover and access the URL? Check links, XML sitemap inclusion, robots rules, password protection, firewall controls and DNS/SSL availability.
  2. Can Google index it? Check WordPress, page-level and HTTP-header noindex directives.
  3. Does the URL resolve as expected? Check status codes, redirect chains and redirect loops.
  4. Is Google selecting another version? Review canonical tags and duplicate HTTP, HTTPS, www, non-www, staging or old-domain URLs.
  5. Is the site reliably serving pages? Look for recurring 5xx or 429 responses, slow server responses and failed application or database processes.
  6. Is the page indexed but unseen? Review its usefulness, relevance and whether there is meaningful search demand.

1. Check whether WordPress is discouraging indexing

In WordPress, go to Settings → Reading and review “Discourage search engines from indexing this site.” WordPress documents that this setting can generate a noindex,nofollow robots meta tag.

This is a sensible temporary setting for a development site, but it is easy to leave enabled after a launch, restore, staging copy or migration. Check it first when an entire site is missing from search.

Then inspect the affected page itself:

  • Review page-level settings in your SEO plugin for a noindex choice.
  • View the page source and search for noindex.
  • Check whether the server sends an X-Robots-Tag: noindex HTTP header.
  • Compare a missing page with a similar page that is indexed.

Make changes carefully, with a backup and rollback plan. If core, plugin or theme updates are part of the fix, follow a backed-up, tested process such as our WordPress security update checklist.

2. Check crawl access, not just index settings

Google must be able to access a page before it can read a meta noindex directive. That is why robots.txt and noindex do different jobs.

Robots.txt primarily manages crawler access and traffic. It is not a reliable way to keep an already-known page out of Google. Blocking a URL in robots.txt while expecting Google to read its noindex directive creates conflicting instructions.

Common crawl-access blockers

  • A Disallow rule in robots.txt.
  • Password protection left on a live page or directory.
  • Firewall, web application firewall or bot-protection rules that deny verified Googlebot.
  • Temporary maintenance mode that remains active.
  • DNS, SSL certificate or hostname errors.
  • IP or geographic access restrictions that unintentionally affect search crawlers.

Do not broadly allow every bot that claims to be Google. Verify Googlebot before changing firewall rules, and ask for qualified help if you are unsure how your security tooling works.

3. Verify what the URL returns

A browser loading a page once does not prove that every request is healthy. Check the HTTP response for the preferred URL and for common variants. Search Console can identify issues, while a hosting or development team can help inspect server logs and request behaviour.

Expected situation Expected status Common mistake Next action
Live, indexable page 200 OK A page returns an error or a soft 404 instead. Confirm the page has useful content and loads reliably.
Permanent URL move 301 redirect Several redirects are chained together. Redirect the old URL directly to the final preferred URL.
Temporary move 302 redirect A temporary redirect remains in place after a permanent move. Review whether the move is truly temporary.
Removed page 404 Not Found An important live page returns 404. Restore the page or redirect it where there is a relevant replacement.
Unavailable-looking page Often 200, but treated as a soft 404 Thin, empty or “not found” content is served with 200. Provide the intended content or return an appropriate status.
Rate-limited access 429 Too Many Requests Rate limits persist for legitimate crawl requests. Investigate the rule, traffic pattern and available capacity.
Server or application failure 500 or 502 Errors are treated as isolated when they recur. Review logs, PHP/application processes, database health and configuration.
Temporary overload or maintenance 503 Service Unavailable A temporary response remains for days. Resolve the underlying condition and restore normal service promptly.

Google documents that 5xx and 429 responses cause crawlers to slow their crawling. Persistent server errors can eventually lead to URLs being removed from Google Search. That does not mean every brief outage causes deindexing, but recurring failures deserve investigation.

4. Inspect canonicals and redirects

Google may crawl your page yet select a different URL as the canonical version. This is common after site moves and when multiple versions of a page remain accessible.

Check for consistency between:

  • The URL in the browser.
  • The final URL after redirects.
  • The page’s rel="canonical" tag.
  • The URL included in the XML sitemap.
  • Internal links across the site.

Redirects and rel="canonical" are strong canonicalization signals; sitemap inclusion is a weaker signal. A canonical tag is not an absolute command—Google ultimately chooses the canonical it considers most appropriate. Avoid using robots.txt to try to consolidate duplicate URLs.

Variants worth checking

  • http versus https.
  • www versus non-www.
  • Trailing-slash and non-trailing-slash versions.
  • Old domains after a migration.
  • Staging, preview or temporary domains.
  • Parameter URLs created by campaigns, filters or plugins.

5. Know when hosting and performance need investigation

Hosting is one possible contributor, not the universal answer to an indexing issue. Google says server availability problems can prevent crawling and that Googlebot may reduce crawl activity when servers struggle to respond. Search Console’s Crawl Stats and URL Inspection tools can help identify whether that is part of the picture.

Ask for an operational review when you see repeated signs such as:

  • Recurring 500, 502, 503 or 429 responses.
  • Host availability or response concerns in Search Console.
  • Slow time to first byte that coincides with crawler or visitor problems.
  • Failed PHP processes, database errors or application-resource exhaustion.
  • Redirect loops, long redirect chains or inconsistent redirect behaviour.
  • Important image, script or other resources failing while a page renders.

Improving response and rendering speed may support more reliable crawling where serving capacity is demonstrably limiting access. It is not a promise of indexing or ranking gains. Tools such as AccelerateWP may be worth discussing as part of a broader performance review, rather than as a standalone SEO fix.

Advanced note for unusually bloated WordPress output

Google’s March 2026 crawler explanation says the initial HTML document is subject to a 2 MB limit and recommends placing critical elements—including the title, meta directives, canonical and essential structured data—high in the HTML. This is mainly a diagnostic consideration for unusually large or heavily modified WordPress output, not a reason for a typical small-business site to panic.

6. Clear migration and staging leftovers

Moves are a frequent source of conflicting search signals. After a WordPress migration, confirm that temporary crawl blocks are removed and that security controls are not blocking verified Googlebot. Take particular care if the site was copied from staging, restored from a backup or launched on a new domain.

After a site move: short verification list

  • Confirm the live WordPress Reading setting does not discourage indexing.
  • Check page and server-header noindex directives.
  • Remove temporary password protection and maintenance pages.
  • Review robots.txt on the live domain.
  • Test preferred URLs, redirects and canonical tags.
  • Confirm the XML sitemap uses the live canonical URLs.
  • Verify Search Console ownership and inspect priority URLs.
  • Review DNS and SSL behaviour for the final hostname.
  • Check that staging or temporary domains are not competing with the live site.

Moving a WordPress site? Ask Advantage Hosting about free migration and an SEO-safe launch checklist. A careful migration process can reduce avoidable technical confusion, although it cannot guarantee immediate indexing or rankings.

7. If technical checks pass, improve the page itself

A technically accessible page can still have limited search visibility. Google may choose not to show a page prominently—or at all for a particular query—when it does not offer enough distinct value, does not closely answer the searcher’s need or has limited demand.

If Search Console shows that Google can access the page and the technical signals are consistent, review:

  • Whether the page answers a clear question or serves a distinct purpose.
  • Whether it has substantial, accurate and useful information beyond near-duplicate pages.
  • Whether its title and visible content accurately describe the page.
  • Whether relevant internal links help visitors and crawlers find it.
  • Whether the business is targeting searches people actually use.

Do not do this

  • Do not use robots.txt as a substitute for noindex.
  • Do not block crawling and expect Google to read a blocked page’s noindex directive.
  • Do not leave 503 or 429 responses in place for days while crawl problems continue.
  • Do not assume an indexing issue proves the host is at fault.
  • Do not make DNS, firewall, server-header or PHP changes without a backup, rollback plan and qualified assistance where needed.

Copyable Google indexing triage checklist

  1. Inspect the exact URL in Google Search Console.
  2. Confirm whether the issue is not indexed, indexed but not ranking, or an old version.
  3. Check WordPress Reading settings and page-level noindex controls.
  4. Check source code and HTTP headers for noindex.
  5. Review robots.txt, passwords, maintenance mode and verified-bot access controls.
  6. Test the HTTP status and final destination of key URLs.
  7. Check canonical tags, sitemap URLs and internal links for consistency.
  8. Look for recurring 5xx/429 errors, slow responses and application failures.
  9. Review staging, DNS, redirects and temporary controls after any move.
  10. If the page is technically sound, improve its usefulness and search relevance.

When to ask for hosting help

Not sure whether Google is being blocked by WordPress, DNS, redirects or the server? Contact Advantage Hosting for a practical WordPress hosting and migration review. We can help investigate repeated server errors, capacity concerns, launch and migration checks, redirect or canonical confusion, and uncertainty around DNS or server logs.

For agencies, document the domain, DNS, Search Console, hosting and staging credentials as part of every client handoff. Clear access and ownership make technical diagnosis far faster when a site needs attention.

Sources and further reading

Tags: