Meta Rewrites Its Crawling Clause Before October 30: What Changes and What Stays Allowed on Your Site
On October 17 Meta removed the word "train" from the clause that lets it crawl sites tied to a business account. Crawling stays, test credentials stay, and there is no opt-out: here is what changed and what to do before October 30.
Three Days: From Nieman Lab's Report to Meta's Rewrite
Commercial Terms are the contract you accept when you use Meta's products for a business: ads, a catalogue, a Page or Business Suite. The version taking effect on October 30, 2026 adds a paragraph that did not exist in the text in force since March 2024: an explicit authorization to crawl domains and web pages with automated software.
| Date | What happened |
|---|---|
| October 15 | Nieman Lab publishes its report on the clause, with three lawyers reading the text. |
| October 16 | Meta responds through Andy Stone; the text still says "train". |
| October 17 | The US edition loses the sentence with "train"; the UK edition keeps it. |
| October 30 | The revised terms take effect. |
October 15: The Clause That Set Off the Alarm
Nieman Lab published its report after a reader's tip. The lawyers it consulted read the clause as permission for Meta to send scrapers, bots and crawlers to the domains "associated" with your business account and to use what they collected to "provide, develop, improve, research, train" its products.
October 16: Meta Responds
Andy Stone, from Meta's communications team, wrote on X that the reading was false and that nothing changes in how the company uses its crawlers: content owners can signal preferences with robots.txt. That day, captures of the legal site still showed "train".
Read also
October 17: "Train" Is Gone
In the US English edition the training sentence disappeared. The authorization now ends with a narrower purpose: "to provide, develop, improve, research, and maintain advertising and business products", plus product integrity and security. Meta said the crawling terms it consolidated only applied to its ads and business products.
October 30: The Date That Matters
The new terms are accepted in the least visible way: continuing to use the products for business purposes after the notice counts as acceptance.
What the Revised Text Says (and What It Left Alone)
The Authorization That Survived
The paragraph allows "automated software (such as web scrapers, bots, robots, spiders, crawlers and user agents)" and the creation of "test credentials" to access, index, cache, analyse, collect, retrieve or crawl the domains associated with your products or services, and to come back periodically. Crawling did not change; the declared use of what gets crawled did.
"Advertising and Business Products": The New Boundary
The text defines that category broadly: products to create and deliver ads, to inform users about your products and to connect you with potential customers. Reading it as "product pages only" has little support; reading it as "anything that feeds the advertising business" has plenty.
No Opt-Out, and Two Different Editions
There is no rejection box: the only exit is to stop using the products for commercial purposes. On top of that, a comparison of the two English editions made on October 17 around 12:30 UTC found the UK version still carried the training sentence. The text does not say which edition applies to which market.
Robots.txt Is Not a Contract: The Mechanism Almost Nobody Explains
A Technical Preference Is Not a Signed Authorization
robots.txt asks, it does not forbid: it is a convention the crawler chooses to honour, with no signature and no judge behind it. Commercial Terms are an authorization you grant by using the product. When the two collide, the question is not who is technically right, it is what the contract you accepted says.
What a Crawler Can Do Even When robots.txt Says No
Meta documents several agents: Meta-ExternalAgent crawls to train models and index content; Meta-ExternalAds crawls for advertising products; Meta-ExternalFetcher fetches a link at a user's request and, per that page, "may bypass robots.txt rules"; FacebookExternalHit builds previews and can also bypass them during security checks. Rules apply per user-agent token: blocking one does not block the rest.
Why "Develop" Still Worries People
Removing "train" does not close the debate: the experts Nieman Lab consulted warned that "develop" can be read as an umbrella word that includes training. Meta denies it. That is lawyers' interpretation, not a conclusion of the contract.
Who Falls Inside the Scope
The October version widens the definition of business use: ads, marketing messages, selling or listing products, a catalogue, developing apps, managing a Page or a Group for business purposes, Ads Manager, Business Suite and measurement services. The key sits in a phrase the text never defines: sites "associated with your products or services". A publisher can read that as its entire website; a shop can read it as product pages only.
What to Do With Your Site Before October 30
Review robots.txt by Crawler Type
The useful move is not a blanket block, it is deciding agent by agent: let the preview agent through so your links on Facebook and WhatsApp keep their card, and decide separately about the advertising crawler. The web scraping with Laravel tutorial covers the technical side.
Read Your Logs
Before changing anything, measure. Search the last few weeks for Meta user-agents and look at two things: the path they request and how often. A crawler hitting pages you never meant to publish decides your configuration far more than any press release.
Content Behind a Registration or Pay Wall
The text authorizes creating "test credentials" to access associated domains, without explaining what they are or where they would be used. The design consequence is simple: a registration wall is not an access control, it is friction.
When to Ask a Lawyer
If content is the main asset of your business, that conversation should happen before October 30. This article describes the text and the readings of it; it is not legal advice.
What to Watch After October 30
Four things: whether Meta publishes a formal version that mentions robots.txt, whether it unifies the English editions, whether a publishers' group challenges the terms, and whether crawling data contradicts the "nothing changes" promise. What is verifiable is the date, the text, the absence of an opt-out and the gap between editions; the rest is interpretation.
This week is enough for two cheap things: read your robots.txt by agent and check your logs. What a text file will not fix is content already cached, nor the fact that you accepted the contract by using the tool. If the topic interests you from the agents' side, the blog has a walkthrough of the MCP standard and its explicit permissions.
