Google: don’t let your CDN or firewall block Googlebot (Nov 2026)
Google’s Search Central blog says blanket firewall and CAPTCHA rules shouldn’t block its crawlers. If your site sits behind bot protection, check it this week.
What changed
On November 11, 2026, Google’s Search Central blog posted guidance on moving a site to a CDN (the network that serves your pages from servers close to the visitor) without overwhelming your own server or blocking Googlebot.
Google’s line on bot protection: “Don’t let blanket WAF or CAPTCHA rules block Google’s crawlers.” A WAF is the firewall that sits in front of your site and screens out bad traffic.
It also says a brand-new CDN starts with an empty (cold) cache, so the first request for each page goes back to your own server. In Google’s words: “Caches only shield you on the second visit.”
And speed matters to crawling: “Slower and slower responses decrease crawl rate.”
Who it affects
Any service business whose site has bot protection, a CAPTCHA or a firewall switched on, or that just moved to a new host or CDN. A lot of small sites have this turned on by a developer or a hosting plan and nobody looks at it again.
What to do this week
- Ask whoever runs your site one question: do our bot or firewall rules let Google’s crawlers through? Get a yes or no, not “it should be fine.”
- Open Google Search Console and look at Settings, then Crawl stats. A sudden drop in crawl requests, or a jump in server errors, after a hosting or security change is worth a call to your developer.
- In Search Console, run URL Inspection on your homepage and one service page and check that Google can fetch them.
- If you just moved hosts or switched on a CDN, expect your own server to take the first wave of requests while the cache fills.
- For the rest of the “can Google and AI actually read my site” checks, see why your site doesn’t show up in ChatGPT and the new website launch checklist. It’s what we check on website builds.
Justin’s take
This one’s boring and it matters. Your site can look perfect to you and still be throwing a challenge page at Google.
You won’t see it from your office. You see it in Search Console, so check it.
— Justin
Sources
- Google Search Central Blog, Moving to a CDN without overwhelming your origin (or blocking Googlebot) (November 11, 2026)
Checked against these sources on November 11, 2026.
For more like this, join the newsletter.