TL;DR — the answer first
- What happened: OpenAI, Google, Microsoft and Anthropic announced on Oct 30, 2026 that they will honor one shared robots.txt token for AI training opt-outs.
- The token: Add
User-Agent: __usage_ai-trainingwithDisallow: /to your robots.txt file, and all four companies say they will treat it as a request not to use your site for AI model training. - What it does not do: It does not remove content already in training sets, it does not block search engines, and it is not yet a formal standard.
- Who should care: Website owners, bloggers and publishers who want one simple opt-out instead of four different company rules.
That is the short version. The rest of this article explains what changed, why it matters, and exactly how to use it.
A blogger, a robots file, and four different rulebooks
Imagine you run a small blog. You spent years writing articles, taking photos and building an audience, one post at a time. Then you hear that AI companies may be using website content to train their models, and you decide you would rather not take part. So you look up how to opt out, and that is where the simple idea turns into a headache.
Until now, there was no single way to say no. Each big AI company had its own crawler name, its own opt-out instructions, and its own timeline.
A website owner who wanted to opt out everywhere had to research each company separately, add each rule separately, and keep checking whether any of them had changed. For a big publisher with a technical team, that is annoying. For a one-person blog, it is a reason to give up and do nothing.
That is the problem the new announcement tries to solve. Instead of four separate opt-outs, the four companies have agreed to honor one shared token. One line, in one file, read by all four.
It sounds small, and in a way it is small. But for the millions of small websites that never had a legal team or a technical team, small and simple is exactly the point.
What was announced on Oct 30, 2026?
On Oct 30, 2026, OpenAI, Google, Microsoft and Anthropic announced that they will honor a single robots.txt policy token called __usage_ai-training. The idea is straightforward: a website owner puts this token in the site’s robots.txt file, together with a disallow rule, and the four companies will treat it as an opt-out from AI model training. You do not need to write four different rules for four different companies anymore.
The token is written as a User-Agent line, which is the normal way robots.txt files address a particular crawler or use. In this case, though, the User-Agent is not a crawler name. It is a policy label that all four companies have agreed to recognize. Think of it like a shared sign on a door: instead of writing a separate note for each visitor, you put up one sign that all four visitors have promised to read.
It is important to be precise about who is involved. The four companies in the announcement are OpenAI, Google, Microsoft and Anthropic. Meta is not one of the four in this announcement, even though many readers will wonder about it.
That does not mean Meta ignores website owners forever, but it does mean this particular shared token does not cover Meta. We will come back to that gap, because it matters.
Wait, what is robots.txt anyway?
If you have never touched a robots.txt file, here is the plain-English version. Every website can have a small text file, placed at the root of the site, called robots.txt. Automated visitors, like search engine crawlers, read this file before they crawl the site. The file tells them which parts of the site they may visit and which parts they should leave alone.
A typical robots.txt file is short. It has groups of rules, each starting with a User-Agent line that says who the rule is for, followed by Allow or Disallow lines that say what is permitted. For example, a rule can say that a particular crawler may not visit a private folder. Search engines have followed this system for years, and it is one of the quiet, boring pieces of plumbing that keeps the web working.
Here is the catch, and it is a big one: robots.txt is a request, not a lock. It works because well-behaved crawlers agree to follow it. It does not physically stop a badly behaved crawler, and it does not encrypt your content or hide it from anyone determined to take it. That distinction matters for everything that follows in this article.
If you are new to AI in general, our beginner guide explains the basics in simple words, including why AI models need training data in the first place. You can read AI Explained in Simple Words: What ChatGPT and Friends Actually Do for that background. With that in mind, let us look at what this new token actually controls.
What does the token control? Training, not everything
The new token controls one specific use: AI model training. When a company trains a model, it feeds it large amounts of text, images and other content so the model can learn patterns from that material. The __usage_ai-training token says: please do not use this site’s content for that training purpose. That is a meaningful, specific request, and it is narrower than many people assume.
It does not control search. If you add this token, your site can still appear in search results, because search crawling and AI training are treated as different uses. That is good news for most site owners, because disappearing from search would hurt traffic far more than most people intend. The token is designed to separate the two uses, so you can opt out of training without opting out of being found.
It also does not control every possible use of web content. The announcement is about training, and the token’s name says so: usage, then AI training. It is not a general “do not touch my site” signal, and it is not a copyright enforcement tool. Readers who want a total block on all crawling would need different rules, and even then, as we will see, robots.txt only works with crawlers that choose to respect it.
The timeline: not everyone, not all at once
One of the most practical details in this announcement is the timeline, because the four companies are not all starting on the same day. Anthropic has already started implementing support for the token, so for Anthropic, this is already real. OpenAI says it will support the token in the coming months, which is vaguer but still a commitment.
Microsoft has given a firmer date, targeting Feb 1, 2027 for its support. Google says it will support the token in 2027, without a specific month in the announcement. So if you add the token today, you should not expect all four companies to honor it instantly. Think of it as planting a sign now that each company will start reading on its own schedule.
That staggered timeline is normal for a change like this, but it is worth stating plainly. A website owner who adds the token in October 2026 and then checks in November should not be surprised if some companies are honoring it and others are still rolling it out. The announcement is a shared promise with different start dates, not a switch that flips everywhere on one day.
Is this a law, a standard, or just a promise?
This is where careful reading matters most, because the announcement is easy to overstate. The token is described in an individual IETF Internet-Draft, authored by Kevin Kelley of Anthropic. An Internet-Draft is a working document in the Internet Engineering Task Force process, and an individual draft is not the same as a formal, approved standard. In plain English, this is a proposal with backing from four big companies, not a finished global rule.
It is also voluntary in an important sense. The four companies have said they will honor the token, and that commitment matters, especially coming from companies of this size. But the token is not a law, and it does not create a legal enforcement system on its own. If a company that never signed up ignores the token, the token itself does not fine that company or block its crawlers.
That is not a reason to dismiss the announcement. Voluntary agreements by the biggest AI companies can still change behavior across the web, because those companies crawl at enormous scale.
It is, however, a reason to describe the token accurately. It is a shared industry commitment, built on a draft document, not a law and not yet a formal standard. Anyone who tells you otherwise is overselling it.
What it does NOT do: the limits you must know
Let us be blunt about the limits, because they decide whether this token is useful for you or just comforting. First, the token does not remove content that is already in training sets. If a company’s model was already trained on your content before you added the token, adding the token does not un-train the model. Training data, once absorbed into a model, cannot simply be deleted like a file from a folder.
Second, the token does not stop all AI-related crawling. As noted above, it targets training use, not search, and it does not cover every company. Meta, for example, is not part of this four-company announcement, so a site owner who only adds this token has not opted out of Meta’s training under this shared system. Other AI companies, smaller crawlers and research projects may also have their own rules, or none at all.
Third, the token does not guarantee compliance from bad actors. Robots.txt has always relied on good faith, and this token is no different. A crawler that ignores robots.txt will ignore this token too. The token makes it easier for honest, large-scale companies to respect your wishes, and it does not solve the problem of dishonest or careless crawlers.
Fourth, the token does not settle the legal questions around AI training. Courts, regulators and lawmakers are still working through what training on web content means under copyright and other laws. This token is a practical tool that exists alongside those debates, not a replacement for them. A site owner with a legal dispute should still talk to a lawyer, not rely on a robots.txt line alone.
Who benefits most from this?
The biggest winners are small and medium website owners. Before this token, opting out meant learning four systems, and many small sites simply never did it. Now, one line covers four of the largest AI companies, which lowers the effort so much that more sites are likely to actually use it. For a solo blogger, a small news site, or a niche publisher, that simplicity is the whole story.
Larger publishers benefit too, but in a different way. A big publisher with a technical team could already manage four separate opt-outs, so the new token saves time and reduces mistakes rather than making the impossible possible. It also gives publishers a single, clear signal to point to in conversations with AI companies. Instead of arguing about four different rulebooks, everyone can refer to the same token.
Regular readers benefit indirectly. If more websites can easily state their training preferences, the web becomes a place where creators have a clearer say over one important use of their work. That does not change what you read day to day, but it does change the balance of control behind the scenes. It is a small step toward a web where opting out is a normal, easy choice, not a technical obstacle course.
There is also a benefit for the AI companies themselves, and it is worth naming honestly. A single shared token is easier for them to implement and explain than four competing systems. It also reduces the public pressure that comes from stories about creators who could not figure out how to opt out. In other words, simplicity helps both sides, which is probably why four competitors agreed on it.
How to add the token: a simple step-by-step
Here is the practical part. If you own a website and want to use this token, these are the steps in plain order. You do not need to be a programmer, but you do need access to your site’s files or to someone who manages them for you.
Step 1: Find your robots.txt file.
Go to your website address, add /robots.txt at the end, and see what comes up. For example, a site called example.com would check example.com/robots.txt. If a file already exists, you will see its current rules. If nothing comes up, your site may not have one yet, and you will need to create it.
Step 2: Open the file for editing.
If you use WordPress or another content system, your robots.txt file may be managed by a plugin, by your hosting provider, or by editing a file on the server. If you are not sure, ask your host or your site manager where the file lives. Do not guess and edit the wrong file, because a mistake in robots.txt can affect search crawling too.
Step 3: Add the opt-out block.
Add these two lines, exactly as written, on their own lines in the file:
User-Agent: __usage_ai-training
Disallow: /The first line names the shared training policy token. The second line says that, for that token, the whole site is disallowed. The slash after Disallow means the entire site, not just one page. If you only want to opt out part of your site, you would use a different path, but most owners who want to opt out will use the slash for the whole site.
Step 4: Keep your existing rules untouched.
Do not delete your other robots.txt rules when you add this block. Your search engine rules, your sitemap references, and your other crawler rules should stay exactly as they were. The new block is an addition, not a replacement, and adding it in the wrong place or deleting other rules by accident is the most common mistake.
Step 5: Save, then check your work.
After saving, visit yoursite.com/robots.txt again and confirm the new lines are there, spelled exactly right, including the two underscores at the start of __usage_ai-training. A single typo in the token name means the companies will not recognize it. Take thirty seconds to double-check, because a silent typo is worse than no token at all.
Step 6: Remember the timeline and the limits.
After adding the token, remember that Anthropic is already implementing it, OpenAI is coming in the following months, Microsoft targets Feb 1, 2027, and Google is coming in 2027. Also remember that the token does not remove past training data and does not cover companies outside the four. Set a reminder to re-check in a few months, when more of the four have rolled it out.
What if you do NOT want to opt out?
Not every website owner will want to use this token, and that is a legitimate choice. Some creators want their work to be included in AI training, because they believe it helps their content reach new audiences or because they see AI tools as a normal part of the web. Others are undecided and want to wait and see how the token works in practice. Both positions are reasonable, and the token does not force anyone to opt out.
If you do nothing, nothing changes for you. Your site will be treated the way it is treated today, under each company’s existing rules. The token is an opt-out, not an opt-in, which means silence means the same as before.
That is an important detail, because some readers assume a new system automatically changes the default for everyone. It does not.
If you are undecided, the safest approach is to learn more before you act. Read our beginner explainer on how AI tools actually work, at AI Explained in Simple Words, and think about what matters most to you: reach, control, or a mix of both. You can always add the token later, and you can remove it later too. It is a file you control, not a permanent decision.
What this means for the AI industry
Zoom out for a moment, because this announcement also tells a story about the AI industry itself. Four fierce competitors, OpenAI, Google, Microsoft and Anthropic, agreed on a shared way for websites to say no to training. That kind of agreement is rare in a race this intense, and it suggests that all four companies felt the pressure to make opting out simpler. Public debate, creator complaints and regulatory attention all push in that direction.
It also suggests that the industry is moving from a phase of grabbing training data first and answering questions later, toward a phase where permission signals matter more. That shift is slow, incomplete and uneven, but it is real. A shared token does not end the debate about training data, but it does make the debate more concrete. Instead of arguing in the abstract about whether creators can opt out, everyone can now point to a specific line in a specific file.
There is a connection here to the business side of AI as well. Anthropic, one of the four companies, recently filed to go public, and its user-facing story is about trust, reliability and steady progress. We covered that angle in Anthropic Is Going Public — Here Is What It Means for You, Not for Investors.
A company that wants users and publishers to trust it has a reason to support clear opt-out tools. Trust, in this market, is becoming a competitive advantage, not just a nice word.
Frequently asked questions
What is the AI training opt-out token?
It is a shared robots.txt policy token, __usage_ai-training, that OpenAI, Google, Microsoft and Anthropic have agreed to honor. When a website adds it with Disallow: /, the four companies will treat it as a request not to use that site for AI model training.
How do I add it to my website?
Add these two lines to your robots.txt file, on their own lines: User-Agent: __usage_ai-training followed by Disallow: /. Keep your existing rules untouched, save the file, and check that the token is spelled exactly right, including the two underscores.
Does it block Google Search?
No. The token controls AI training use, not search crawling. Your site can still appear in search results after you add it. If you also want to block search crawling, that would be a separate robots.txt decision, and it would hurt your search traffic.
Does it remove my content from models already trained on it?
No. The token does not remove content already in training sets, and it cannot un-train a model. It is a forward-looking signal for future training, not a delete button for past training.
Does it cover Meta and every other AI company?
No. This announcement covers OpenAI, Google, Microsoft and Anthropic only. Meta is not part of this shared token, and smaller companies may have their own opt-out systems or none at all. Check each company’s own rules if you want broader coverage.
When does it take effect?
It depends on the company. Anthropic has already started implementing it, OpenAI says it will support it in the coming months, Microsoft targets Feb 1, 2027, and Google says it will support it in 2027. Adding the token now means each company will honor it on its own rollout schedule.
Is this a law or a formal standard?
Neither, at this stage. It is described in an individual IETF Internet-Draft by Kevin Kelley of Anthropic, which is a working proposal, not a formal approved standard. It is a voluntary shared commitment by four companies, not a law with its own enforcement system.
What if I make a typo in the token?
A typo means the token will not be recognized, so the opt-out will silently fail. Always double-check the spelling, especially the two underscores at the start of __usage_ai-training, and re-check your robots.txt file after saving.
Should every website owner add it?
Not necessarily. It depends on whether you want to opt out of AI training by these four companies. If you do, this is now the simplest way to do it. If you do not, or you are undecided, doing nothing leaves things as they are, and you can add or remove the token later.
The bottom line
The Oct 30, 2026 announcement is simple, useful and easy to overstate, so let us end with the accurate version. OpenAI, Google, Microsoft and Anthropic will honor one shared robots.txt token, __usage_ai-training, so website owners can opt out of AI training across all four with a single two-line block. That is a real improvement over four separate rulebooks, especially for small sites.
It is not, however, a magic shield. It does not remove past training data, it does not block search, it does not cover Meta or every other company, and it is not yet a formal standard or a law. It works on the good-faith system that robots.txt has always used, and it rolls out on different dates for different companies.
For a website owner who wants to opt out, the practical move is clear: add the two lines, check the spelling, keep your other rules, and remember the timeline. For everyone else, the move is just as clear: understand what the token does, decide whether it fits your goals, and do not let headlines turn a useful, limited tool into something it is not.
