A profile of nonprofit Common Crawl, which has scraped billions of webpages since 2013, including paywalled ones, to build an archive used by OpenAI and others (Alex Reisner/The Atlantic)
Alex Reisner / The Atlantic:
A profile of nonprofit Common Crawl, which has scraped billions of webpages since 2013, including paywalled ones, to build an archive used by OpenAI and others — “You shouldn’t have put your content on the internet if you didn’t want it to be on the internet,” Common Crawl’s executive director says.