By Elena Vasquez

Meta's updated Commercial Terms take effect Oct. 30 and grant the company the right to crawl websites of any business using its products and to use that content to train those products. The grant covers businesses that never buy an ad. A Facebook Page or a Meta Business Suite account is enough.

Nieman Lab reported the change on Oct. 15. Three legal professionals reviewed the terms for the outlet and concluded the language could support an argument by Meta that it may scrape news publishers' websites and train its AI systems on the content.

On Oct. 16, Meta spokesperson Andy Stone responded on X and called that summary false. He said the update exists so Meta can pull accurate information about products for sale on third-party websites when it makes ad and content recommendations. He added it is "not about scraping news sites."

Meta's ad terms already permit crawling advertisers' websites, including for training. The October version extends the reach to businesses that spend nothing.

What does the new clause allow?

The new crawling paragraph lets Meta, or anyone acting for it, use web scrapers, bots and other automated tools to collect content from domains and pages tied to a business's products or services. It also covers URLs surfaced through the business's use of Meta products, or through users' use of them.

The clause permits Meta to "create test credentials" to reach those pages. Meta may use the collected content to develop, improve, research and train its products.

The same crawl permission appears in the revised Self-Serve Ad Terms, also effective Oct. 30. Continued business use of Meta's products signals acceptance. The terms state that a business which disagrees should stop using Meta products for business purposes.

Media and intellectual property attorney Matt Topic, who represents The Intercept in its copyright case against OpenAI, told Nieman Lab there is no guarantee a court will enforce the provision. Publishers should assume it will be enforced, he said.

How much of this is new?

The current Commercial Terms, dated March 6, 2024, contain no crawling clause. Meta's ad terms do.

The current Self-Serve Ad Terms, dated Aug. 22, 2025, tie crawling rights to placing an ad order. Listed uses include developing advertising products, maintaining integrity and "research and training."

Meta has granted itself narrower crawl rights before:

• The Ad Creative Generative AI Terms (May 6, 2024) allow crawling advertiser-supplied URLs for its generative AI ad tools.
• The Product Catalog Terms allow periodic catalog updates from catalog URLs.
• The Seller Agreement, last updated May 26, 2026, already permits automated software, including creating test credentials, to obtain information from seller-provided URLs.

The difference is scope. Prior ad agreements tied permission to an ad purchase and covered the advertiser's domain or advertiser-supplied URLs. The new terms tie permission to any business use and extend to URLs made available through users' activity.

Meta has also announced in its Business Help Center that it will fold the Product Catalog Terms into the Commercial Terms and the Ad Creative Generative AI Terms into the Self-Serve Ad Terms to "reduce overlap." That announcement did not mention crawling or training.

Where does robots.txt fit?

Stone told Nieman Lab the new terms will not change how Meta crawls publishers, and that "content owners can still manage how their content is accessed by using robots.txt." Neither updated terms document mentions robots.txt.

The legal experts told the outlet the new language gives Meta an opening to argue its commercial terms supersede other blocking instructions. Cornell law professor James Grimmelmann said test credentials could technically let Meta sign up as a fake user to get past registration walls. Meta declined to say whether test credentials would bypass paywalls or other paid content.

Meta's crawler documentation lists several bots:

• Meta-ExternalAgent crawls for training foundation AI models and product improvement.
• Meta-WebIndexer improves Meta AI search results and helps Meta cite and link to content in responses.
• Meta-ExternalFetcher may bypass robots.txt when fetching pages at a user's request.
• FacebookExternalHit, which creates link previews, may also bypass it during security or integrity checks.

Why does this matter?

The Oct. 30 grant attaches to ordinary business use. Brands with a Facebook Page or a Business Suite presence but no ad spend fall inside it.

Meta points to robots.txt as the control mechanism. Operators can block Meta-ExternalAgent, the AI training crawler, while allowing Meta-WebIndexer, the search crawler. The terms do not say whether a business's robots.txt settings limit what it agrees to share by accepting them.

The clock runs short. Website operators have two weeks from the Oct. 15 report to decide whether to keep using Meta's products for business, and to review how their robots.txt files treat each Meta crawler.

The European Publishers Council is writing to the European Commission about the new terms. The EPC is specifically concerned that Meta may claim permission to use an article whenever a user shares a link to that story on social media.