Inside the web infrastructure revolt over Google’s AI Overviews
It could be a consequential act of quiet regulation. Cloudflare, a web infrastructure company, has updated millions of websites’ robots.txt files in an effort to force Google to change how it crawls them to fuel its AI products and initiatives.
We spoke with Cloudflare CEO Matthew Prince about what exactly is going on here, why it matters, and what the web might soon look like. But to get into that, we need to cover a little background first.
The new change, which Cloudflare calls its Content Signals Policy, happened after publishers and other companies that depend on web traffic have cried foul over Google’s AI Overviews and similar AI answer engines, saying they are sharply cutting those companies’ path to revenue because they don’t send traffic back to the source of the information.
Related Knowledge Center: Artificial intelligence
On top of a failing business model, newsrooms must navigate the powerful — and not fully reliable — new capabilities of artificial intelligence and the deployment of large language models.
State of the business model | Copyright and intellectual property
Related Knowledge Center: Copyright and intellectual property
With copyright questions already complicated by the emergence of the web in the 1990s, artificial intelligence raised the stakes to a new level.