While Tuesdays election result is a global disaster, it is most immediately distressing for three groups: American Latinos, American Muslims, and American pollsters.
First of all, let us dispel with the idea (that I have heard some propound) that they weren’t wrong. Huge numbers of polls done independently in multiple states gave results that were consistently at variance in the same direction with the actual election results. I can see three kinds of explanations:
- The pollsters shared a mistaken idea or methodology for correcting their tiny unrepresentative samples for differential turnout.
- Subjects lied about their voting intentions.
- Subjects changed their minds between the last poll and the election.
3 seems unlikely to account for a lot, as it seems implausible to suppose that many people changed their minds so rapidly. 2 is plausible, but hard to check and difficult impossible to correct. 1 is a nice technical-sounding explanation, and certainly seems like there must be some truth to it. Except, probably not much. As evidence, I bring the failure of VoteCastr.
Slate magazine teamed up with the big-data firm VoteCastr to trial a system of estimating votes in real time. Ahead of time they did extensive polling to fit an extensive model to predict an individual’s vote (probabilistically) as a function of several publicly-available demographic variables. Then they track records of who actually voted, and update their totals for the number of votes for each candidate accordingly.
Sounds like a perfectly plausible scheme. And it bombed. For instance, their final projection for Florida was 4.9 million (actually, 4,225,249) for Clinton and 4.6 million for Trump, a lead of about 3% for Clinton. The real numbers were 4.5 million and 4.6 million, a lead of 1.3% for Trump. (The difference in the total seems to be mainly due to votes for other candidates, though the total number of Florida votes in VoteCastr is about 100,000 more than in the official tally, which I find suspicious.) They projected a big victory for Clinton in Wisconsin.
The thing is, this removes the uncertainty related to reason 1: They know exactly who came to vote, and they’re matched by age, sex, and party registration. Conclusion: Estimating turnout is not the main problem that undermined this year’s presidential election polls.
Common Infirmities 
The answer is very simple. Many polling/media organizations most of the time are not about discovering the truth, but convincing people that their “truth” (their favorite Leftist) is way ahead. You can’t combine real science and the mathematics with ideology. But that is what most try to do.
Typically they only converge anything approaching reality as the election gets close so as not to be too embarrassed. The IBD/TIPP is one of the few that almost got it right.
Sounds pretty conspiratorial and implausible to me. It’s not even clear to me what the motivation might be. If (as you seem to suppose) the poll results intentionally overstated Democratic voting in key states, it’s hard to see whose interests that would serve: The Republicans (by inducing overconfidence in their opponents and convincing Democrats that they don’t really need to vote) or the Democrats (by keeping their spirits up). It seems like a remarkably expensive way to accomplish these effects indirectly (unless you are of the full-conspiracy party who believe that they don’t actually conduct any polls, but just make up the numbers).
In any case, none of this would apply to VoteCastr (the main subject of my post), which is a start-up trying to get attention for its big-data nous by getting the results right — no bragging rights now — and is producing its results too late to influence anyone one way or another.
“Sounds pretty conspiratorial and implausible to me.”
You mean like Democratic National Committee chair (“Dirty”) Donna Brazile working with CNN to provide Hillary with the debate questions beforehand? Or media members asking the Democrats what questions they should be asking the Republican candidate?
Wikileaks shed a lot of light on that. It is very clear significant parts of the media are acting in collusion with Democrats.