How newsrooms are using AI to 'smell the data'

Add this term to your newsroom AI arsenal: RAG, or retrieval-augmented generation. It’s the architecture behind chatbots like the Financial Times’ AskFT and Bloomberg’s BloombergGPT: The bots are trained on the publisher’s own archives and use them to answer readers’ questions, with citations. Freelancer Josh Axelrod wrote for Nieman Lab this week:

RAG’s jargon-filled moniker belies a fairly simple approach — one that boosts reliability, key for journalists who find themselves in the reliability business. The model doesn’t create an answer from the vast expanses of Amazon reviews, medieval literature, and Reddit comments that general-purpose chatbots are typically trained on. Instead, a RAG-powered model retrieves information from a journalist-defined database, then uses that to augment what it generates with attributions to boot. The database can be a newsroom’s archives of fact-checked articles, a book of case law, or even a single PDF.

A RAG system can surface the most noteworthy elements in huge datasets, filtering and surfacing information journalists might miss otherwise. A journalist from Norway’s iTromskø calls it “smelling the data.”