Your personal newsroom

For journalists, election polls are the gift that keeps on giving.

Consider: The New York Times on Sunday — two days before the election — published a new poll with strikingly good results for Democrats.

Is it accurate? Is it reliable?

Funny you should ask. Having written the initial polling story — that is, having created the news — the Times gets to ask and answer those questions, new stories covering the content they created. And other news organizations, having covered the initial poll, also get to follow up with the same questions.

Rest assured: After the votes are counted, we’ll be inundated with either fist-bumping or hand-wringing about the accuracy of the polls — as if they were ever the point. 

The Times on Oct. 28 published a story in which one of its editors, Patrick Healy, interviewed members of their polling team: Nate Cohn and William P. Davis.

Healy cuts to the chase:

Let’s start with a point-blank question. Should people trust these polls showing Democrats running ahead or neck-and-neck in red states?

NATE COHN: You can trust them to a degree. You can trust the polls to usually be in the ballpark, but you can’t trust them to be exactly “right.” Polls are inherently imprecise and subject to many sources of error.

So if by “trust” you mean “assume that the leading candidate will win,” then the answer is “no.” The polls have been wrong by this standard before and they will be wrong again. But if trust means “rely on the polls as the best source of information on the electorate, even if an inexact one,” then the answer is yes.

So, if you ask me if I’m telling you the truth, if you should trust me — it depends on what you mean by “trust.” You can trust me to a degree (yes, “trust” is doing a lot of work there).

Feeling good about this so far?

The article concludes with this, from Cohn:

A poll is not a fact. It’s a best measurement. There’s no guarantee that a “best” measurement is good enough — whether in terms of The Times’s credibility or whether it improves our understanding of politics. Historically, there have absolutely been elections when it didn’t feel like polling passed that test. The 2016 election is the easiest example; there will be another election like it again.

That said, readers come to The Times to understand American politics, which includes an understanding of public opinion during an election. For better or worse, polling is really the only quantitative measure of the attitudes of the electorate. There just isn’t an alternative. The polls will still drive the political conversation, even if The Times doesn’t conduct any of its own polling. Campaigns run on polls. Our coverage would inevitably be affected by the polls that others are conducting. If that’s going to be the case, it’s probably better that we’re conducting our own.

At no point does Healy ask what is, to me, the only question that matters: Why conduct election polls at all? Who does that serve? How do polls help me with my fundamental task as a citizen and community member: voting.

Cohn conflates public opinion surveys and election polling, but they are distinct animals. A healthy debate over whether there is such a thing as “public opinion” began more than a century ago.

Jill Lepore writes in her U.S. history, “These Truths”:

The most stinging critiques came from University of Chicago sociologist Herbert Blumer and Columbia political scientist Lindsay Rogers. Public opinion polling is not a form of empirical inquiry, Blumer argued, since it skips over the crucial first step of any inquiry: identifying what it is that is to be studied. As Blumer pointed out, this is by no means surprising, since polling is a business, and an industry run by businessmen will create not a science but a product. Blumer argued that public opinion does not exist, absent its measurement; pollsters created it: “public opinion consists of what public opinion polls poll.”

Blumer’s disdain did not derail the industry. But the man whose name is synonymous with public opinion polling, George Gallup, took a dim view of the other branch of polling: election predictions. According to Lepore:

Time coined the word “pollsters” in 1939, and, in the public mind, the word “polls” came to mean two different things: surveys of public opinion and forecasts of elections. When Gallup started, he conducted forecasts only in order to prove the accuracy of his surveys, there being no other way to demonstrate it. That aside, the forecasts themselves were pointless.

Congress called for an investigation. “These polls are a racket, and their methods should be exposed to the public,” Walter Pierce, a Democratic member of the House, wrote in 1939. (Pierce, like many critics of polling, which tended to show that Americans favored entering the war, was an isolationist.) Part of the concern was that polls were fraudulent. Another concern was that they were interfering with the proper function of elections and of government. Polls don’t represent the people, one congressman wrote; Congress does: “Polls are in contradiction to representative government.”

Many journalists equate election polling — the horserace — with the campaign itself: “The polls will still drive the political conversation,” as Cohn puts it. Which makes polling sound like an inevitability, an act of God, rather than invented content, a journalism choice — I’d argue a business choice dressed up in editorial language.

Resources committed to breathless stories about a neck-and-neck race are smart commerce at the expense of the actual point of the election: citizens deciding their future — for themselves, their communities, the nation.

Today, and in the coming days and weeks, we’ll find out what decisions we’ve made. That’s what elections — not polls — are for.

CNN: Can you trust the polls this time? (Nov. 2)

The New York Times: The Final Polls Show a Blue Surge. Can We Trust Them? (Nov. 1)

The New York Times Are Polls Overstating Democrats’ Midterm Leads? Times Pollsters Weigh In. (Oct. 28)

American Crisis (Margaret Sullivan): Polls, pundits and horserace coverage are NOT what we need for the midterms. (Aug. 24)

The New York Times Why Polling Chaos Could Create Big Problems for American Politics (Aug. 22)