: 
: 
: 
: 
: 
: 
: 
: 
: 
: 
This year some of the biggest noise around conservative blogs regards the belief that pollsters are oversampling Democratic voters and weighting Democrats too heavily in their calculations. This is a flip from the last presidential election in which it was liberal blogs claiming that Republicans were being weighted too heavily in polls. Polls are weighted on a variety of factors to reflect as near as is possible the demographic characteristics of the American electorate, and many pollsters include party ID as part of that equation. I had never given the issue much thought until the last couple of weeks, as I heard more and more from conservative sites about the “flawed polling” that was painting a far too optimistic picture of Barack Obama’s support. It turns out that much of this had already been put in perspective during the 2004 race by Mystery Pollster:
Party Identification is one of the longest tracked and closely examined questions in political polling. “Generally speaking, do you consider yourself a Republican, a Democrat, an independent or what,” has probably been asked on more surveys over more years more
consistently than any other question.The most important thing to remember is that Party ID is an attitude, not a demographic. People can change their views of political parties. They cannot change their age, gender, race, years of education and locale (unless they’ve moved).
<snip>
[R]ecent studies have shown evidence of significant short-term change in Party ID. The 2000 Annenberg National Election Study (NAES), like the 2004 study now underway, was a daily tracking survey that ultimately included more than 58,000 interviews over the course of the year, roughly 5000 interviews per month. NAES observed that the percentage of the electorate identified as Independent “was not stable over time.” In a chart on page 61 of Capturing Campaign Dynamics, Daniel Romer and his colleagues showed the percentage of Independents falling steadily from roughly 31% to 27% during the conventions, then spiking 8 points to 35% just after the Democratic convention in early September, then falling off again steadily back to roughly 28% on election day, then plummeting sharply to below 25% a few days later. No surprise that they concluded:
Surveys that are weighted by party identification may be operating under some misconceptions about party identification. Party identification may not be as stable as once thought and could be considered an indicator of the respondents’ attitudes toward candidates at a given moment of the campaign. (p. 61)
There is another good reason to be weary of automatically weighting survey results by Party ID. Most of the national polls ask Party ID toward the end of the survey. Campaign pollsters can tell you that Party ID can vary with the content of the questions that precede it.
Though I had never considered the issue before, this does make intuitive sense. I’ve always considered the number of self-proclaimed independents to be far higher than actual voting habits would suggest. It only makes sense that this would extend to party ID generally. Now, what weighting by party ID does do is make polls more stable over time. Not surprisingly, if you consistently weight your polls, for example, as 35% Democrat, 35% Republican, and 30% Independent, you won’t see massive shifts in opinion, because those self-identifying as Democrats are likely to hold the same opinion as did the self-identifying Democrats in previous polls. If their opinion has shift markedly, they will simply not identify as Democrats anymore. Same goes for Republicans and Independents.
Given the dramatic shifts party ID can take in a relatively short time frame, it would appear that weighting party ID in one election based on the exit polls of an election four years before is a good way to introduce bad assumptions into one’s numbers. A solution to this is known as “dynamic weighting,” by which pollsters weight for party ID not based on irrelevant exit poll data, but rather based on the party ID as given by respondents over a shorter time frame. If you get 40% of respondents identifying as Democrats over a six or eight week period, then it’s probably better to weight your polls based on that than on older exit polls.
At any rate, given the stabilizing effects of weighting by party ID, it’s easy to see its appeal to pollsters. The more a poll varies, the less likely it is to be considered reputable. Stability is a hallmark of the best known pollsters.
Ultimately, the validity of party ID is certainly open to interpretation, but what I will say is that there is no clear reason to believe that in either this election or 2004’s election that over or under-representing based on party ID created polls much off the mark of the actual results. As long as the demographics are realistic, then the party ID is unlikely to pose a significant problem. For instance, in 2004 despite the fact that many Democrats felt they were being under-represented in polls, the average of all polls as compiled by pollster.com was only about one point off the actual vote. In 2000, the margin was higher, but the average of all polls taken right before the election was still only two-and-a-half points off the actual vote. So, if Republicans are hoping to see a major difference between what the polls say the night before election day, and where the vote ends up, they are probably in for a disappointment.
TheZoo 
Just as important a question, is what constitutes a likely voter, to a poll analyst? Is the question simply asked, are you likely to vote in this election? No it is not. Is the method somewhat more concealed, as in asking if the respondent has voted in each of the last several elections, then basing the categorizing as a likely voter on a statistical interpretation of the answers? Partially, but it is much trickier, and varies from poll to poll. In a change election such as we are experiencing, new voter registration is much higher than typical, and new voters may be more motivated to turn out on election day, or earlier, as we are already seeing. Since most of the new registrants are signing up as Democrats, it would appear that the pollsters are trying to account for that factor in their weighting calculations. There has also been discussed the factoring in that younger voters may not have landline phones and are undersampled. The poor and non-caucasians also are less likely to have landline phones, and the polls are having to try to account for this as well.
The Republicans are fearful that too many of their less motivated members will be discouraged enough by the polls to stay home on election day, and the downticket races will suffer on their side.