Google, Microsoft, OpenAI and Anthropic have backed a shared robots.txt rule that tells AI companies not to train on your site. Google plans to support it in early 2027.
Dominic Coutts
Nothing to change yet. When it arrives, most businesses will want to block training but stay visible in AI answers, and Bing will let you do both.
What happened
Google said on 30 October it will honour a shared control being drafted at the IETF, the body that sets internet standards. It's a single robots.txt rule that tells every participating AI provider not to use your site for training. Google aims to support it in early 2027.
Bing has firmer dates: new controls in Bing Webmaster Tools from 15 December, and the new preferences honoured from 1 February 2027.
What it means for you
Right now, blocking AI training means a separate rule for each company's crawler. From next year, one line should do the job. Bing also splits training from answers, so you can refuse training and still be used as a source in AI answers. That's the setting most businesses want once it's explained to them. Google's rule covers training only and doesn't affect whether you appear in search.
What to do
- Leave your current robots.txt rules in place. Nothing new is live yet.
- If your site uses old NoArchive or NoCache tags, Bing wants you moved across by 1 March 2027.
- Decide now whether you're happy for AI tools to train on your content, so the change is a quick job when it comes.
Google Search Central Blog
Real and worth watching, but nothing to act on yet.
More signals
Get a focused review of how your business is found, where competitors win attention, and what to change to generate more of the right enquiries. You leave with a plan you can act on, whether we do the work that follows or not.