Google's Search Central team published guidance on 11 November on what happens when a site moves behind a CDN: requests to the original server go up rather than down, teams mistake the spike for an attack and switch on blanket bot protection, and Googlebot gets blocked with the bad traffic. Google says pages then drop out of Search, and the video published with it says products stop appearing in Google Shopping.
If you have a CDN, host or security change booked before Black Friday, move it. Getting the bot rules wrong in that window costs far more than waiting until January.
What happened
Google published guidance on 11 November on a problem it sees after sites move behind a CDN, the service that caches your pages worldwide so they load faster. Requests to the original server go up rather than down at first, because a new cache is empty and every first request for a page still reaches it, and because faster responses tell Google it can crawl harder. In Google's words, "a CDN does not immediately protect your origin server from every request". The common reaction is to switch on blanket bot protection, which challenges or blocks Googlebot along with everything else. Google says pages then drop out of Search, and the video published alongside the post says products stop appearing in Google Shopping. Google gives no figures for how often this happens.
What it means for you
The damage does not look like a bot problem. It looks like a redesign or a site move that lost you rankings, which is why it goes unnoticed for months. A block page served with a success code is worse again, because Google cannot tell it from your real page. The exposure is highest over the next 6 weeks, while sites get speed work before Black Friday.
What to do
- Ask whoever runs your CDN or firewall to allow verified Google crawlers, checked against Google's published IP ranges rather than the name the visitor gives, which anyone can fake.
- Before any CDN, host or security change, have someone load your main pages so the cache is filled when it goes live, then watch Search Console's crawl stats for a week.
- If the server is struggling, return a 429 or 503 response for a few hours instead of a blanket block, and check Merchant Center for product page unavailable errors.
Google Search Central
Worth doing something about this week.
More signals
Get a focused review of how your business is found, where competitors win attention, and what to change to generate more of the right enquiries. You leave with a plan you can act on, whether we do the work that follows or not.
