Ask many poll-weary Americans what they think about the steady stream of numbers they’ll be force-fed over the next few months, and you’ll likely hear something to this effect: “I’ve never been surveyed in any of these polls, and neither has anyone I know. So who are they talking to?”
But whether a poll has surveyed you or your neighbors is irrelevant, say researchers. What matters is that the process they used to select respondents accurately reflects the population being surveyed.
Most media-run and other professional polls use a process called random sampling. This is designed to pick respondents at random, with an equal chance for every U.S. citizen to be chosen. So long as that equal chance is ensured, the poll’s sample is statistically sound. The process isn’t just used in polling either; it’s used for nearly anything that requires that a representative sample of a larger whole.
For polls, the random sampling theory is applied to the U.S. population, or to all registered voters or whoever is the subject of the poll. Using lists of phone numbers from all 50 states, pollsters use computers to pull random samples of American voters–representatives who reflect the makeup of the total pool. Theoretically, everyone has the same chance of being chosen for the poll, so it doesn’t matter who is actually chosen.
The margin of error–which is generally reported as a plus-or-minus percentage in popular polls–is the statistical chance that the sample was not completely random, and the extent to which the poll results could be affected if the sample was biased in some way. They margin is calculated using a mathematical formula that compares the sample size to the size of the entire population being surveyed. For most modern-day polls, the margin of errors is plus or minus 4 percentage points or fewer.
“Random sampling really is quite a reliable way to poll and get accurate results,” said Stephen Ansolabehere, a professor of political science at the Massachusetts Institute of Technology. “It’s definitely the most statistically sound way of conducting a poll.”
The process isn’t perfect, though, and pollsters spend a lot of time and money trying to compensate for small errors that can have a big effect on poll results.
One growing problem with polls conducted by random sample is non-response, or the rate at which contacted respondents refuse to answer the poll. As Americans continue to be inundated by phone polls and other solicitors, they are increasingly just saying “no.” That can throw off a sample’s accuracy significantly, according to pollsters and researchers.
“People just don’t want to answer questions anymore,” said Philip Meyer, Knight Chair in Journalism at the University of North Carolina, Chapel Hill. “It’s probably the biggest problem right now with sampling. We used to get 80 percent response from our samples; now we’re lucky to get 66 percent.”
To compensate, some polling organizations conduct surveys over longer periods of time–the typical medial poll is conducted over a few days–and generate larger phone number lists than they need. Still, Meyer said, there isn’t much that can be done to correct the problem. “You’re not getting a certain segment of the population in these polls, and they may vote and they certainly have opinions,” Meyer said. “Random sampling is the best thing we’ve got now for polling, but that flaw clearly makes it difficult to get completely accurate numbers.”
