News

A profile of nonprofit Common Crawl, which has scraped billions of webpages since 2013, including paywalled ones, to build an archive used by OpenAI and others (Alex Reisner/The Atlantic)

Mediagazer · last updated

Alex Reisner / The Atlantic:
A profile of nonprofit Common Crawl, which has scraped billions of webpages since 2013, including paywalled ones, to build an archive used by OpenAI and others  —  “You shouldn’t have put your content on the internet if you didn’t want it to be on the internet,” Common Crawl’s executive director says.