Swing state journalists were trained to avoid the worst kinds of political coverage. Did it work?

In the lead-up to the 2022 midterm elections, a coalition of journalism groups trained 45 journalists in 21 “intensive” sessions over nine months. The editors and reporters came from 22 news organizations ranging from public radio stations to small-town newsrooms to large metro dailies.

Afterwards the groups behind the Democracy SOS trainings — Solutions Journalism Network, Hearken, and Trusting News — wanted, quite reasonably, to know: Well, did any of it work?

The trainers brought the question to a trio of prominent digital journalism researchers and the result is “How the Engagement Journalism Movement Is Changing Political News Content.” From that paper, published last month:

This intensive training is part of the massive, industry-wide retraining effort to move journalists from top-down, false-binary political coverage to one that is more relevant to different communities and less polarizing for constituents. We know that journalists are at least saying that their practices have evolved. Yet, one question lingers: Is the content produced by these re-trained journalists actually changing? Is all that training resulting in content that matches the training’s goals?”

For the paper, Sue Robinson (University of Wisconsin-Madison) and co-authors Margarita Orozco (University of Kansas) and Joshua P. Darr (Syracuse University) analyzed more than 5,000 news stories, ultimately drilling down on 1,388 political stories from 19 of the 22 news organizations that participated in the trainings. (The training organizations funded the “applied research” study but were not allowed to adjust how the research team articulated their findings or manipulate any data points.)

The researchers specifically wanted to look at horse-race framing, transparency, solutions-orientated coverage, and engagement practices. They found:

• A significant decrease in horse-race framing in political stories in 2022 compared to 2018 and 2020. The researchers found that “horse-race coverage” chiefly about which candidate is leading in the polls or fundraising decreased from 27% in 2018 to 13% in 2022.
• A large increase in transparency elements — which include reporter bios, sidebars that explain the reporting process, links to evidence, and mission statements about political coverage. About 20% of stories included at least one transparency element in 2022, compared to 6% in 2018 and 7% in 2020.
• A large increase in engagement reporting. Political coverage that used engagement practices such as reader call outs, community advisory boards, and integrating reader feedback rose from about 6% in 2018 to 15% in 2020 and 27% in 2022.
• A significant increase in solutions-oriented coverage, from 7% in 2018 and 8% in 2020 to 14% in 2022. Solutions journalism outlines possible answers to issues and strives to include evidence and context to help readers evaluate those potential responses.

As the researchers note, the trainings coincided with wider shifts in the news industry’s approach to election coverage that could have contributed to the changes in political news content. (The criticism of horse-race coverage, in particular, spans many election cycles.) Though they found meaningful changes in the political coverage of 2022, the researchers stopped short of declaring a direct causal link to the specific Democracy SOS trainings.

In an email, Robinson noted that the two easiest tasks for newsrooms to achieve were transparency (“because it is fairly easy to add to an article a few lines about how it was reported and who was reporting it”) and reducing horse-race coverage (“because it is fairly easy also to frame a story without using battle terminology, or reducing stances to binaries”).

The researchers and training partners were both surprised to see that “solution stories” did not see a larger increase. The trainers had the sense from the workshop sessions that the journalists and editors in attendance had been the most excited to adopt that approach.

“We theorized with our partner Solutions Journalism Network, that solution stories are actually pretty hard to do – or at least they represent one of the biggest departures from more traditional reporting,” Robinson said. “Solutions journalism requires a different kind of (and often more arduous) reporting and research.”

The training known as Democracy SOS has been renamed Advancing Democracy, said Solutions Journalism Network director of beacons and advance practice Linda Shaw. The program has grown to include revamped online training and $4,000 grants for newsrooms.

“We did worry that fewer newsrooms might have time to participate in this fellowship in an election year like 2024, but demand was strong – and continues to grow,” Shaw said in an email. “The changes we made to the fellowship were more a refinement about how best to support newsrooms in transforming their coverage, mainly through helping them take small steps they could build upon over time.”

“We also continued to emphasize one key element that newsrooms consistently said was among the most valuable parts of the fellowship: the time spent with peers,” she added. “Journalists appreciated the opportunity to problem-solve together, share successes, and build a sense of community with others working toward similar goals.”

Robinson is currently reviewing 2024 election coverage and said that, anecdotally, researchers are seeing “a lot more engagement-focused reporting” than previous years. Preliminary results from focus groups held in Pennsylvania, Wisconsin, and Nevada show that audiences — even right-leaning ones — notice and appreciate the shift to engagement-focused stories, she said.

“We defined engagement as a story having more than two sources that could not be considered political officials, public figures, pundits, campaign, strategist, and the like,” Robinson said. “Or the story had to contain information that showed the journalist were doing more intensive engagement work, such as a survey of the readers, a forum, some kind of call-in show, or something like that. We will have to see if the total numbers show these anecdotes to be widespread during the 2024 election.”

I asked Robinson what else Nieman Lab readers might want to know about her research. She wanted to sound a note of caution.

“None of this transformed content really matters much if newsrooms can’t get it in front of all kinds of audience groups,” Robinson said. “We must figure out a way to talk across our differences so that we can govern together using a shared set of facts,” she added. “We are very far from that right now.”