Toward AI-Powered Source Audits
This is our second blog post about Unheard, a new project designed to help news organizations uncover potentially overlooked narratives by using AI to audit who is quoted in their articles.
Thoughtful sourcing is the backbone of every good story, the raw material from which narratives are built. Decisions about whom to quote are extremely delicate; the journalist shapes not only the perceived balance of a story, but also its framing, stance, the broader image of society it constructs—and the audiences it serves.
News organizations risk overlooking important narratives and injecting bias into their reporting without thoughtful sourcing. That’s why we’re building an AI-powered tool to conduct source audits on large corpora of news articles. The goal of the project is to equip newsrooms with data that helps them identify when coverage leans too heavily on a single type of source. Thanks to recent breakthroughs in generative AI, highly accurate quote-detection technology is now within reach.
In an experiment, we tested seven different generative AI models on their ability to extract quotes from articles published by the New York Times and the Associated Press. We found that the models performed surprisingly well, with some achieving accuracy scores in the mid- to high-ninetieth percentile.
Related Knowledge Center: Inside the newsroom
As newsrooms struggle with rickety business models and precarious finances — as well as new technologies — they are deploying an array of editorial responses.
The Times and its critics | Washington Post mortem
Related Knowledge Center: Artificial intelligence
On top of a failing business model, newsrooms must navigate the powerful — and not fully reliable — new capabilities of artificial intelligence and the deployment of large language models.
State of the business model | Copyright and intellectual property
Related Knowledge Center: Theoretical journalism
Journalism as a profitable product has been in decline for years, raising the questions of what the news is for and what people are willing to pay for.