AI authentication tools are built without adequate journalist input, new report finds

It has never been easier (or cheaper) to fake and alter media. The evidence that journalists rely on to verify facts — photographs, videos, documents, websites, and even audio calls with sources — can now be convincingly generated with AI.

In June, dozens of journalists, technologists, and academics gathered at NYU’s Arthur L. Carter Journalism Institute in Manhattan to discuss this growing threat to authentication and verification work. Hosted by Princeton’s Center for Information Technology Policy (CITP), the day-long workshop included expert panels, presentations, and breakout discussions. On Thursday, CITP published a report synthesizing the workshop’s findings and offering insight into the “fight for facts in the AI age.”

“Establishing one fact increasingly means establishing the facts beneath it,” write the report’s authors. “While AI tools can strengthen the verification process and introduce new opportunities to increase access to knowledge, there is an arms race with the volume of potentially fabricated facts.”

In response to the deluge of AI-generated media online, a host of verification initiatives, commercial products, and other tools have flooded the zone. The report suggests that many are not being developed with enough direct consultation from journalists.

One banner initiative in Silicon Valley to address authentication problems is C2PA, or the Coalition for Content Provenance and Authenticity. The initiative, launched in 2021 by Adobe, Microsoft, the BBC and others, embeds cryptographically signed “manifests” inside media files. It effectively creates a label the moment a photo is captured in a camera, recording its full history and origins as the image is edited, published, and circulated.

Despite gaining the support of major news organizations like the Associated Press and The New York Times, C2PA has hit major snags when it comes to newsroom implementation. One anonymous newsroom mentioned in the report said the software it used to ingest and edit photos stripped the manifest; it had to pressure the developer to patch the issue. Since the publisher’s CMS didn’t support C2PA, the manifest was viewable on images internally but deleted when published for readers. Most CMS vendors currently do not support C2PA.

“Will audiences believe the news organization has a chain of custody for its files if it cannot offer ‘full transparency’ of the manifest?” the report’s authors ask.

The report also notes that many social media companies strip C2PA and other metadata from images circulated on their platform, further eroding the initiative’s effectiveness.

There are also safety concerns. Cryptographic signatures could endanger journalists and sources by attaching identifying information to photographs or videos they take. This presents an immediate threat to those working in authoritarian environments, according to a recent study by the human rights organization Witness.

Similar challenges arise with other authentication tools, including AI image and video detectors. Many of these tools operate with confidence ratings — they might spit out a 80% likelihood that an image is AI-generated. The report explains that these types of ratings aren’t useful to most reporters, given that readers usually think about authentication as a binary: is it real, or is it fake?

“These confidence ratings are not (easily) translatable to audiences, since they want to know if an image is genuine or not and not a confidence rating that reporters often cannot even explain since the authentication tools are often ‘blackboxes,’” write the authors.

Given these challenges with bringing authentication tools into newsrooms, the report suggests that there needs to be more collaboration between technologists and the news industry to better meet the need for authentication tools.

“We urge technologists to work with reporters to develop the tools together and then let reporters evaluate their usefulness,” write the authors.

You can read the full report here, including more recommendations and an overview of how AI is shaping verification work at large.