On November 9, 2016, Americans, as well as the rest of the world, experienced an extraordinary shakeup. Despite polls, projections, and pundits stating the contrary, Donald Trump secured a victory over Hillary Clinton in the 58th presidential election. While some lamented and others rejoiced, most were left wondering what went wrong with the polls, which unanimously predicted a victory for Hillary Clinton.
While it’s easy to disregard this result, and the subsequent shock it elicited, as ‘bad’ or ‘wrong’ polling, the answer to why the polls were unable to represent their electorate’s opinions is more nuanced. It requires us to take a closer look at the history of polling public opinion.
A Brief History
Modern public opinion polls find their roots on the heels of the Great Depression, when statistician George Gallup’s American Institute of Public Opinion received national recognition for correctly predicting the victory of Franklin D. Roosevelt in 1936, beating out reputed polling source and magazine The Literary Digest’s numbers, which predicted Republican candidate Alfred Landon to win.
Nearly a decade later, at the end of the 1948 presidential election, newly appointed president Harry S. Truman held up the headline that morning on the Chicago Daily Tribune: “Dewey Defeats Truman,” telling the press, “That ain’t the way I heard it!” This was a big blow for pollsters across the country and ultimately led to the formation of the American Association for Public Opinion Research (AAPOR).
In addition to structure and organization, actual polling methods evolved from door-to-door surveys to national phone call campaigns. Pollsters, using public records, could randomly select phone numbers and collect a wide, diverse sample of information. With the arrival of the cell phone, however, phone numbers became increasingly private and caller ID more ubiquitous, limiting the people being surveyed. Pollsters now face the added competition of the Internet, where anyone can create a poll and gather data. Though the quantity of data has grown exponentially, the quality often goes unchecked, there are numerous factors within each data set to consider, and interpreting the results is rarely as simple as a headline makes it seem.
The Art of the Poll
“You cannot assess people’s behaviors strictly from quantitative data,” explains Peter Zandan, Global Vice Chairman at Hill and Knowlton Strategies and a social science expert. “You need a holistic view that incorporates both behavioral science and data science to be accurate when making predictions.”
The important thing to note about polling, is that it only provides a sample of responses from the public. This is why findings are often caveated with a margin of error. “That margin of error comes from lots of places. Not strictly the poll, but also sometimes how people report it and how the data is weighted and processed.” For this election, Zandan says, the bigger errors were most likely a result of the modeling and the interpretation of data, not the data itself.
This means that polling is better described as an art, not just a science. Organizations add varying degrees of weight to certain responses based on the participant and their characteristics, so the opinions of repeat voters were given more weight. In an election that rallied first-time voters, we now know that past behavior was significantly less predictive in 2016 than we expected. Likewise, the factors weighted to predict Brexit, another notorious shock for pollsters, suggested the wrong outcome as certain demographics were given more representation in models and hard-to-reach voters were under counted.
As the campaign season drew to a close, the media chose to focus on the final verdict rather than the potential for change. The surge in similar findings among different polling organizations quickly spun the prediction into fact, completely disregarding the likelihood of disruption from Trump’s supporters—something is campaign was known for.
Today’s Data-Driven Campaign
Those in marketing understand that it can be hard to predict the actions of consumers. Rather than relying on quantitative data to understand markets and the public, it’s important to take those findings and combine it with qualitative analysis. We cannot be dependent on one tool when studying human behavior. “It would be myopic to believe that one source of information can give you an accurate view of the world,” explains Zandan.
Journalists and the public should not disavow data, but instead approach it more critically. Data can help tailor a marketing campaign; however, we must beware our expectations and oversimplifications or risk overlooking entire groups. “If you feed your marketing teams the same tired demographic research, you’re painting a very restricted picture for them,” writes Alex Jutkowitz, CEO of Group SJR. “You’re setting them up to reach the most stereotypical conclusions, which can hurt your brand and your bottom line.”
Creativity, in addition to data from credible sources, means that marketers can see think outside the patterns shaped by the same weighted models. Polls and the media were blindsided by a disruptive candidate, but marketers who take a nuanced approach to data will remain the drivers of change.