The infamous Literary Digest poll of 1936 predicted Alf Landon would defeat President Roosevelt. Were telephones to blame?
Image may be NSFW.Clik here to view.
Kansas governor Alf Landon visits Roosevelt at the White House in 1943, seven years after the president defeated him in a landslide election. (AP)
If all goes smoothly, late on the night of November 6th we'll know whether President Obama has won reelection. But six weeks is a long time to wait, so we look to the polls -- eat, breathe, and sleep the polls -- until we can have the real thing.
For those following closely, no analyst is more dearly beloved than Nate Silver at The New York Times, and last night Silver pointed to a little discrepancy in the polling numbers that is, for the moment at least, complicating matters: President Obama is polling more strongly in the polls that call cell-phone numbers in addition to landlines.
"Why is this happening?" Will Oremus writes at Slate. "The most obvious answer is that Obama enjoys more support among minorities and the young, who are less likely to have landlines. A possible separate factor is that Obama seems to do better in live-interview polls, while Romney benefits from automated polls."
This question is basically one of how the uneven distribution of a technology -- cellphones, in this case -- correlate with votes come Election Day. This very same question underlies one of the most famous case studies of polling failure in history: the 1936 Literary Digest presidential poll, which took place at a time when roughly 40 percent of households had telephones, and which wrongly predicted Kansas Governor Alf Landon would carry the day. Was the distribution of telephones what skewed the results? That's long been the story, but new research says otherwise.
At the time the weekly magazine Literary Digest was the leader in electoral predictions -- the Nate Silver of its day, if you will. In advance of the 1936 election, Literary Digest mailed out some 10 million postcard "ballots" to a list assembled from rosters of different clubs and associations, voter registration rolls, city directories, and -- crucially -- telephone directories and lists of registered car owners. Those latter two groups made up the bulk of the list. Days before the election, the Digest predicted a 3-to-2 victory for Landon. When Election Day came, Roosevelt won one of the most decisive victories in the 20th century, carrying 46 of 48 states and winning 62 percent of the popular vote.
Was the telephone and automobile distribution at the heart of the error? That's the conventional story, "a staple of statistical literature," as Maurice C. Bryson called it in an The American Statistician in 1976. A 1974 statistics book by Robert Reichard, The Figure Finaglers, made that case:
The time was late 1935.* The opinion ostensibly being measured: the voter's choice for the president -- the incumbent, President Roosevelt, or the challenger, Senator Alfred M. Landon of Kansas. Everything was planned impeccably -- with a statistically significant number of voters to be called up from all sections of the country. But the planners forgot one basic fact: the use of the phone itself was introducing a bias into the sample. Remember, this was 1935, and the people who owned phones at that time did not representa cross section of the American public. Quite the contrary. Telephones were a luxury then -- and the people being sampled were the relatively affluent ones -- and hence the ones more likely to vote for the Republican candidate.
There's just one problem with this explanation and others that echoed it. As Bryson noted, "The are all wrong!" He wrote:
Good statisticians should have cauth the mistake, since the telephone-survey excuse is inherently implausible. ... Since voter participation tends to be highest among the well-to-do, the telephone owners shouldn't have been all that bad as a sample of the voting population. Furthermore, consider quantitatively the Digest's prediction that Landon would get about 60 percent of the vote. If he had obtained 60 percent of the votes of all those with telephones, then -- assuming two voters per phone -- he would have had a block of over 14 million votes, out of the mere 16 million he actually got. That would leave Roosevelt carrying the non-phone voting population by an incredible 27-to-2 million vote margin. If this were not already implausible enough, one could note that Landon won heavily only in the (non-Southern) rural areas of the country, where telephones were relatively scarce. In such a well-to-do area as Westchester County, New York, presumably well-populated by phone owners, the Landon margin was a modest 51 percent.
For its part, Literary Digest felt the same way. In its own assessment, released the week after the election, the error was not the result of the Digest's failure to "reach the lower strata." Its editors explained:
[But] the "have-nots" did not re-elect Mr. Roosevelt. ... As Dorothy Thompson remarked in the New York Herald Tribune, you could eliminate the straight labor vote, the relief vote, and the Negro vote, and still Mr. Roosevelt would have a majority. ... Besides -- we did reach these so-called 'have-not' strata. In the city of Chicago, for example, we polled every third registered voter. In the city of Scranton, Pennsylvania, likewise other cities, we polled every registered voter. ... The fact is that we were as badly off there as we were on the national total. ... All this conjecture about our 'not reaching certain strata' simply will not hold water.
More recent analyses agree. A study by Dominic Lusinchi published this past spring in Social Science History applies contemporary statistical techniques to the Literary Digest data, combining it -- powerfully -- with a dataset collected by Gallup the following year (earlier analyzed by Peverill Squire in a landmark 1988 paper on the case), which sought to examine the validity of mail-ballot polls. It asked respondents whom they had voted for, whether they had received the Literary Digest ballot, whether they had returned it, and whether they had changed their minds between filling it out and voting. "The Digest's original list was no impediment to predict [sic] the correct winner. ... Had everybody on the original Digest list returned his or her straw ballot, the magazine would have been in a position to forecast the correct winner of the election: Roosevelt," Lusinchi writes.
So, then, what is the culprit? Not technology distribution, not the vagaries of inequality, but a problem far more basic in statistics: nonresponse bias. Those 10 million ballots mailed? Only 2.3 million were returned. Landon supporters were far, far more likely to fill out the postcard and send it in. "Poll respondents and nonrespondents favored opposite candidates," Lusinchi says. "While three-fifths of nonrespondents voted for the incumbent, only two-fifths of respondents did."
Lusinchi provides a fascinating explanation for how the myth gained acceptance. At the time, new companies -- George Gallup's American Institute of Public Opinion, in particular -- wanted to challenge Literary Digest's primacy in the polling world. When the magazine failed, they seized on the error as evidence of their superiority. Pollster Archibald Crossley argued in 1937, "The Literary Digest method is outmoded." Gallup, Crossley, and a third pollster, Elmo Roper, argued that their methodologies were more scientific, and could yield samples far more representative of the electorate. Gallup, Lusinchi writes, "wrote the most and the longest about [the Digest] event. Throughout his career he mentioned [it] in books, articles, and interviews," always arguing that the automobile and telephone lists skewed the results away from the poor.
Where does all this leave Mr. Obama, Mr. Romney, and today's polling discrepancies? Statistical and polling methods have improved dramatically since 1936, making the chance of an error of this magnitude -- for reasons of technological penetration or any other -- highly unlikely. That's not to say that the cell-phone gap is insignificant. It is, in fact, quite revealing. But what it reveals is less about who will win come November, than it is about how cellphones and landlines are spread out across society, and how those fissures relate to our political ones.
* I am not sure why this book referred to 1935 since the poll was in 1936. The quote appears in the Bryson article.Image may be NSFW.
Clik here to view.
Image may be NSFW.
Clik here to view.
Clik here to view.
Clik here to view.