SEO Status
A site can be closed to search engines in three separate ways, and two of them are invisible when you view the page:
| Signal | Where it lives | Visible in page source? | |---|---|---| | X-Robots-Tag | HTTP response header | No | | robots.txt | A separate file at your domain root | No | | <meta name="robots"> | Inside the page HTML | Yes |
Google applies the most restrictive of the three. So a site can carry index, follow in its HTML — the one signal you can actually see — while a header or a robots.txt quietly keeps it out of search entirely.
Open Settings → SEO Status to see all three read from your live site, in the Indexability group at the top of the page.
What the check reports
Search engine access — OK. All three signals agree and your site is indexable. Nothing to do.
Signals contradict each other. Some signals say "index" and others say "do not index". The three values are shown side by side so you can see which one is blocking. Remember that the most restrictive one wins — if any of them says no, your site is not being indexed.
An installed plugin is closing the site to search. Some plugins deliberately hide a site while it is being built — they send a noindex header and a blocking robots.txt, and they are meant to be removed at launch. The check names the plugin so you know exactly what to disable. If your site is not ready yet, this is expected behaviour and you can leave it alone.
The HTTP header is set outside jekcms. jekcms only sends an X-Robots-Tag header in maintenance mode and on a few internal endpoints (preview, cron, webhooks). If you see one on a normal page while maintenance is off, the software is not the source — which is why searching the files that ship with jekcms never finds it. Look instead at:
- a
.user.inifile or anauto_prepend_filedirective on your server - your environment configuration, which is not part of the installed package
- a layer in front of your site (CDN, WAF, or a server-level rule)
On your own server you can locate it with:
grep -rn "X-Robots-Tag" /path/to/your/siteIf nothing turns up there, ask your host whether a rule is being applied in front of the site.
robots.txt blocks the whole site. The live robots.txt contains User-agent: * followed by Disallow: /, which stops Google from crawling any page. Check your robots rules under Settings → SEO. If they look correct there, a physical robots.txt file on the server is overriding the one jekcms generates.
Cached copies do not carry response headers. Pages served from the full-page cache are returning different headers than freshly built ones. This matters because the pages that get cached are, by definition, your most-visited ones. Update jekcms, then clear the cache under Settings → Performance.
Why the headers matter for caching
From version 2.136.0 the full-page cache stores the response headers alongside the page body and replays them on every cache hit. Before that, a cached page could answer with different indexing and security signals than the same URL served fresh — so the busiest pages on a site were the ones most likely to lose them.
You do not need to configure anything for this. Existing cached copies from older versions are ignored and rebuilt automatically; nothing is served incorrectly during the transition.
Related
- Settings → SEO — robots rules, AI crawler policy, sitemap options
- Tools → SERP Identity Check — how your site name and identity resolve for search engines