Margin of Error

[vakkotaur]

Come the next Tuesday night or Wednesday morning even if things go roughly as many predict or hope, there will likely be a significant difference between actual ballots and exit polls, and greater difference between actual ballots and pre-election polls. This has happened for at least the last couple elections and some have taken it as meaning there had to be tampering and dirty tricks. There might well be some of that, but ballot and poll mismatch is not evidence of it. No poll is perfect. Not every sample is truly representative. Polling is done hopefully big enough, hopefully often enough, and hopefully good enough to narrow the unknowable.

Iowahawk often posts parody of goings on in the world. One of his recent posts is not a parody but an analysis of statistics. It's not as dry that might sound and he takes care of all the math anyway. It's worth a read.

Comments

"Excuse Me, Sir..."

[michaelmink.livejournal.com]

"But we're taking a poll: on Wednesday, November 5th, would you like pollsters to be baked, boiled, fried or frozen?"

[vakkotaur.livejournal.com]

Yes.

[nefaria.livejournal.com]

The one that gets me is that pollsters think that they're getting a truly random sample. The large portion of the population that doesn't answer polls is excluded, and this may be people who are too busy working. It also excludes people who aren't at the phones when they call, which would be other people working. So they're getting a larger portion of people who don't work than who work.

Then there are loaded poll questions that encourage one answer over another. And two polls each claiming a 2% margin of error that come up with results 10% apart.

Oh, and don't get me started on pollsters who decide to massage the data after the poll. "Hmm, my sample had equal number of Democrat and Republican opinions, but I know this region is 75% Democrat, so I'll weight the Democrat results by 1.5 and the Republican results by 0.5."

[vakkotaur.livejournal.com]

I don't know that they think they're getting a random sample. They might wish people to think it, but as you (and Iowahwak) point out there is a lot of unsampled space - not the part a sample could have been taken from but wasn't, but the parts from which no sample can be taken. And this bit (http://www.zombietime.com/lefts_big_blunder/) the Clever Hans Effect and other things shows that even when a person makes every every effort present themselves as unbiased, the bias signals get through. But it's also hopeful (for McCain & the Republicans) as it also goes on about the Asch experiments and normative and informational conformity. The ending line is:

Yet it may very well be that an army of glum, dispirited and pessimistic conservatives will reluctantly trudge to the polls on November 4, each one imagining they are the only remaining person in the entire country voting for McCain, and lo and behold -- they'll turn out to be a silent majority after all.

[luke-jaywalker.livejournal.com]

I've spent quite a bit of time working in market research. There *are* specialized political polling firms, but a lot of the corporate-research firms do political contracts in election season.

You go where the people are. The more people you approach, the more people will take polls (the response rate is said to be MUCH higher in political than corporate, but still), so you don't go down Rural Route 25 in western Colorado where the houses are a mile and a half apart. You knock on suburban doors or a busy downtown street.

This is bias. Many market-research firms pay and train their interviewers, the people gathering information, no better than McDonalds trains its cashiers. This is *really* bad because (I have seen this more than once) you have interviewers who have no idea what the hell they're doing, how to ask an unbiased question, why bias is bad, or what the **** bias is in the first place.

These things are not conducive to a good survey. And frankly, anyone with a certain amount of marketing knowledge (knowledge that is NOT rare, both sides have MANY people who can do this) can word an ostensibly neutral survey that in fact biases the results up to 15-20% in a chosen direction.

Polling is overrated and ineffective. (OK, there are a FEW polling firms that do it right. A very, very few.)

[thecanuckguy.livejournal.com]

There must be *some* semblance of truth to the polls, else we'd probably notice after the first few elections that they were way off and we'd not pay attention to them any more.

However, no I don't tend to trust "expert opinions" on anything. I am reminded of last year when, due to the fact that the school my children attend (one in kindergarten that year, one in day care) was below 160 students, and at "40% capacity" or something like that. The school division then decided to put the school "on review for possible closure". They'd then have community meetings where people from the school division would trot out so-called "experts" saying "based on population projections in your area, attendance is just going to keep going down. We don't foresee an improvement in class size, so we are going to recommend that keeping the school open would not be feasible." No matter what the parents would say, they'd trot out these "experts" (who, BTW, live in BC), saying that their predictions haven't been wrong before (apparently, before the year began, they predicted how many kids would be attending that year, and they were 1 off.) Fortunately, the education minister (of the provincial NDP (a socialist party, just wanted to point that out ;) )) put a freeze on all school closures in the province (unless it had already been decided beforehand (the vote on our school was to be at the end of the year)). So, we never would have known what the division would have decided otherwise, but my bet is that they would have closed it.

The kicker is this year they had so many kindergarten students coming in (my daughter is now in grade 1) they had to make two kindergarten classes! Take *that*, experts! :P

This is the first time I've seen polling experts *that* far off, though.

[luke-jaywalker.livejournal.com]

I know some people working in a McCain headquarters here in Colorado. Apparently a *lot* of the people they call are saying things like "I'm voting Republican, but don't tell my neighbors."

They're not comfortable telling a pollster on the street what they think ("Don't tell my neighbors", and one of those neighbors could be with them or about to pass by from behind), but the voting booth is private and anonymous.

I think we're going to see an interesting discrepancy here. I hope we don't see riots coming from it. Some of my McCain friends are very concerned about this. ("The polls show that we were going to win, but the election result was different. The Republicans must have hacked the voting system! We was robbed! Time to break some windows!")

[vakkotaur.livejournal.com]

A couple weeks ago I was asked by a relative, "I see a lot of McCain/Palin signs around town and few Obama signs. How is he ahead in the polls?" Well, that wasn't a big city where the opposite is true. I wonder how many people don't have signs that might otherwise so that they can camouflage themselves in a sort of political anonymity.

As I've also seen foreign newspaper headlines like "Can America Overcome" I would expect a lot of articles saying how racist the U.S.A. is there isn't an Obama win. It would much like the tired Guardian headline "How Can $NUMBER People Be So Stupid?" And that would certainly be played up here as well. That we'd have our first female Vice President would somehow not matter, of course.