Skip to content
Halloween 2026Explore the community map

Technology & AI

Meta Removes Disputed Commercial Terms Language Following Outcry Over AI News Scraping Rights

Menlo Park tech giant clarifies web crawler policies and robots.txt protections after legal scholars warned consolidated terms could bypass publisher blocks.

By East Bay Post · Syndicated from Lazy Tensor
Published · 2 min read

Meta removes language from terms that experts said allowed news site scraping — Editorial illustration for eastbaypost
Meta removes language from terms that experts said allowed news site scraping — Editorial illustration for eastbaypost

Meta Platforms removed controversial legal language from its commercial terms page on Saturday after journalism watchdogs and intellectual property scholars warned the clauses appeared to authorize scraping news websites to train artificial intelligence models. As reported by The Verge, analysts at the Nieman Journalism Lab and Cornell University digital and information law professor James Grimmelmann had flagged the consolidated wording earlier in the week. Legal experts interpreted the revised provisions as granting the Menlo Park technology company broad rights to ingest publisher content for AI training while superseding standard robots.txt exclusion instructions.

Following intense scrutiny from editorial boards across San Francisco, Oakland, and Walnut Creek, company officials deleted the disputed passages and issued a formal policy clarification. Meta spokesperson Daniel Roberts stated that the company heard publisher concerns and wanted to clarify that the consolidated crawling terms applied strictly to advertising and business products, adding that those boundaries have now been explicitly written into the terms themselves. Roberts emphasized that the contractual update does not alter how the company operates its automated web crawlers across the public internet.

Company communications representatives Andy Stone and Daniel Roberts had previously pushed back against legal critiques, asserting that there was no change to how Meta utilizes web crawlers and characterizing any suggestion to the contrary as false. In Saturday's expanded technical statement, Roberts affirmed that digital publishers and website administrators can continue deploying standard robots.txt protocols to signal their indexing preferences. Corporate legal counsel confirmed that the updated commercial service agreements do not grant the platform any new legal authority to override those technical blocking directives.

Regional media guilds, independent digital publishers in Contra Costa and Alameda counties, and intellectual property committees at Berkeley law institutes welcomed the removal of the ambiguous contract language. Industry directors noted that local newsrooms invest significant editorial funding into investigative reporting and rely on enforceable technical standards to prevent unauthorized dataset harvesting. Technology policy boards across the Bay Area continue to evaluate standardized machine-readable licensing frameworks that compensate journalism organizations when archival text is utilized by commercial language models.

Digital publishers, webmasters, and legal researchers can inspect the updated commercial terms of service and technical user-agent documentation through the corporate transparency portal at about.meta.com. Questions regarding robots.txt configuration standards or publisher indexing controls may be directed to corporate media relations at 1 Hacker Way in Menlo Park or by calling (650) 543-4800 during regular weekday business hours.

Comments (0)

No comments yet. Be the first to share a thought on this story.

The free newsletter

The East Bay, in your inbox.

The local stories worth knowing, delivered once a week.

From East Bay.

Get the newsletter

Free to read. Unsubscribe anytime.

More local stories