It seems fairly obvious to state polling is a biased business. The aim is to get a cross-section of people and ask them the same question. From their answers, you could expect to extrapolate the views of a city, or a country, or an economic group etc.
But it is not quite as easy as it seems. For example, if you were to conduct a poll among the readership of the magazine Scientific American, you would expect to be reaching a certain demographic.
You would expect this group to be college educated, scientific in leaning, probably not religious and probably male. In fact, you would reach nearly 10% more women than men, you are reaching homeowners, and no surprise they all have college degrees.
Taking the readership of Scientific American then would not represent the views of middle America, nor would most people suggest it did.
How can you get a representative sample?
It is not easy. You cannot simply walk down the street. Asking someone on the street in Kansas will give you a different perspective than asking a person on the street in New York City, even if they voted for the same president.
Huge telephone list surveys will not work either. These days few people pick up the phone for a number they don’t recognize and the people who do are likely to be older, from the generation who did answer the phone. While their views are legitimate, you’re not going to get a decent cross sample of the population.
The larger, richer surveys get around this by using Random Digit Dialing. Computers generate numbers that could be phone numbers, equally they could not be phone numbers. But here’s the rub, your name is not on a list. They aren’t calling a list. If your number is called, it is because a computer generated it randomly and happened randomly on the seven digits that are yours (seven because you can assume an area code).
The idea is if the sample size is sufficient to represent the population as a whole, the view you get from the cross-section will give you an accurate view of the entire population, within a certain margin of error, reduced to a percentage size.
How the questions work
The sample size is, therefore, one factor which affects the accuracy of the poll, but just as significant is the way in which questions are worded and where they occur in the questioning. The point is if you want a specific result from a poll you can get to it by asking the questions in the right way.
The answers are also as insightful. When asking for approval ratings, respondents are asked to respond on a scale. Remember answering questions with strongly agree or somewhat disagree answers? They are designed to elicit a nuanced response. With a nuance, you interpret it in any way you want to. It is all spin.
What’s the answer? Take a poll with a large pinch of salt.