Right now, if you’re paying attention, you’re seeing a lot of polling going on in connection with the presidential campaign. Ever wondered what they mean by “margin of error”?

Well I did. So I set out to find my own little simplistic way of understanding what those pollsters are doing. I don’t mean completely understanding what is probably so complex that only Karl Rove could figure it out. I mean, fooling around with statistical functions in Excel to see what I can determine on the basis of the little I know about polling.

As I understand it, in the simplest case, a pollster asks a fixed number of people who they will vote for and tallies up the totals. There are many possible sources of error inherent in this process. First, people may lie to you. Or change their mind before the election. Or not bother to vote. Professional pollsters have ways of dealing with these and other issues, but I don’t know much about that, and even if I did, it would be beyond the scope of this entry. The only source of error I’m prepared to talk about is the probability that the randomly selected small “sample” of voters are not exactly representative of the whole state or country. It’s kind of like the probability of getting more or less than 5 heads in 10 flips of a perfect coin.

Excel has a formula for doing that. It’s based on the binomial distribution in math, which is related to Pascal’s triangle. This is a discrete formula for predicting the probability of getting a given number of “positive results” from a given number of “trials” when you know the probability of getting that result in one trial.

The form of the function for the coin flipping example is as follows: BINOMDIST (heads,flips,probability (i.e. 50%),logical parameter). A logical parameter of 1 means the formula gives the probability of getting less than or equal to that many heads in so many flips. Thus, the probability of getting 5 or fewer heads in 10 coin tosses is: BINOMDIST (5,10,0.5,1) = 0.623.

It turns out this function can be used to estimate the margin of error in simple polls. Say you ask 1000 people picked randomly from the population who they want to vote for and 500 of them say Obama. You can use the formula to find out the probability of getting more than 500 Obama results from 1000 people if the real answer (by asking everyone) is that 47 percent like Obama. BINOMDIST (500,1000,0.47,1) = 0.973. This is the probability of getting no more than 500 Obama votes. Thus, getting more than 500 would occur only about 2.7% of the time. Usually, statisticians pick 2.5% as a reasonable cut off point for an “improbable” event. So, if the “real” answer is that 47% of the people favor Obama, then it would be highly improbable that a sample of 1000 people would include 500 Obama supporters. Hence, the “margin of error” around the estimate of 50% given by the poll is about 3% (50-47). A similar probability can be determined for getting less than 500 if the real answer were 53 percent. So, in this example, Obama supporters would be in the range of 47-53 percent.

Sampling fewer than 1000 people would, naturally enough, result in a higher margin of error, but you can check that out for yourself.

DadExplanations04/23/08 7 comments


David • 05/02/08 8:24 PM:

Nope. I can’t say I’ve every wondered what they meant by “margin of error”. But I hope to some day read your post and maybe understand it. Till then, I’ll remain dumb.

Patrick • 05/04/08 10:40 AM:

You’re welcome, Dad. I fixed your link to Pascal’s Triangle (which was cutting off because of the apostrophe in the URL). I did it by converting the apostrophe to %27.

As for polling, couldn’t you also reasonably argue that people are not like gas molecules, that despite your best efforts to obtain a representative sample, you still might be quite a bit off? And because this possibility exists, you have to be even more wary of any polls that seem to advance a particular political view, etc. because your pollsters may not be entirely honest.

Dad • 05/06/08 1:49 PM:

Thankfully, people are not like gas molecules (although some are gasbags), but their yes/no answers on political decisions are like coin flips, and therefore subject to mathematical analysis. In fact, it is precisely because of the difficulty in obtaining a representative sample that the concept of “margin of error” was invented. It says nothing of the honesty of the pollster, the confusion of the pollees upon reading an ambiguous question, or the tendency for all of us to periodically change our minds. The margin of error I am describing is something more fundamental that relates to the use of a small sample to infer some property of the whole population. Even if the pollster is honest, the questions clear, the people unswerving in their views, you still have a good chance of getting the wrong answer. Hence the error bands. The real margin of error if you throw in all the other factors is much larger. That’s why we have to actually hold the election. (And hope that the votes are counted fairly.)

David • 05/10/08 3:08 PM:

Here’s a really good video on polling from Penn and Teller (those guys are actually pretty dang good, in my opinion - although there are some profanities in this segment of bull$*%#).

They basically point out that polling isn’t science, but, like most things, can be wildly manipulated. Funny comment on Hobbit movies!

Dad • 05/11/08 8:56 PM:

P&T got it right. Polling is BS. I basically hate polling. The only reason for polling is to get questions for Family Feud or Power of Ten. Politicians are trying to trick you into thinking you like them. Corporations are trying to find your weaknesses so they can trick you into buying more of their crap. Psychologists are trying to find out things that are none of their business. So everybody should lie when polled. (Not participating isn’t enough.)

David • 05/12/08 7:41 PM:

Take a chill. Pill. Pops, I love the fury! Here’s an NPR link about math and polls that is funny. I love the vampire part!

Dad • 05/12/08 10:01 PM:

I love the vampire logic. Just like the logic I used to “prove” to Patrick that we’re all descendants of Abraham.

Post a comment

Thanks for signing in, . Now you can comment. (sign out)

Please capitalize your name properly and use the same information each time you comment. We will not send you spam, and your email address will not be posted.

Remember me?


Related Entries
  1. Survey Says!
    Should small businesses be encouraged to grow and hire more workers?
  1. Site Error Message: Please Ignore
    Please ignore the error message when you post an entry or a comment.
  1. Comments Error
    I just got an Internal Server Error when I tried to post a comment on Dan’s entry. So I did…
  1. Custom 404 Error Page
    You know how when you follow a broken link you get a page that says “Error 404: File Not Found”?…
  1. The Luge
    Watching the Luge event, I was struck by the closeness of the race.