PollyVote


The PollyVote project uses the high-profile application of predicting U.S. presidential election results to demonstrate advances in forecasting research. The project is run by political science professors and forecasting experts, one of which is J. Scott Armstrong. All procedures, data, and results are fully disclosed and freely available online.
The project started in March 2004 to demonstrate the benefits of combining forecasts. In averaging forecasts within and across different forecasting methods, the combined PollyVote forecast provided highly accurate predictions of the two-party popular vote shares for the last three U.S. presidential elections.

History

The PollyVote was created in March 2004 by marketing and forecasting expert J. Scott Armstrong and political science professors and . The goal at that time was to apply the combination principle in forecasting to predict President Bush's share of the two-party popular vote in the 2004 presidential election.
Until Election Day in November of the same year, the researchers collected data from 268 polls, 10 quantitative models, and 246 daily market prices from the Iowa Electronic Markets vote-share market. In each of the last three months prior to the election, they also administered a survey with a panel of 17 experts on US politics, asking them for their predictions.
The forecasts were first combined within each component method by averaging recent polls, the IEM prediction market forecasts from the previous week, and averaging the predictions of the quantitative models. Then, the researchers averaged the forecasts across the four component methods. The resulting forecast was named the PollyVote. From March to November, the forecasts were initially updated weekly, and then, twice a week. The forecasts were published at the Political Forecasting Special Interest Group at forprin.com.
In 2007, joined the PollyVote team and helped to launch the PollyVote.com website prior to the 2008 U.S. presidential election. For predicting the 2008 election, the general structure of the PollyVote remained unchanged; the PollyVote combined forecasts within and across the same four component methods as in 2004. However, some changes were made at the level of the component methods. Instead of averaging recent polls themselves, the PollyVote team used the RCP poll average by RealClearPolitics as the polls component. In addition, the advantage of the leading candidate was discounted using the approach suggested by Jim Campbell. The first PollyVote forecast for the 2008 election was published in August 2007, 14 months prior to Election Day, and was updated daily.
For forecasting the 2012 election, a fifth component called "index models" was added to the PollyVote. This component captured information from quantitative models that use a different method and rely on different information than the traditional political economy models. In particular, the index models capture information about the campaign, such as the candidates' perceived issue-handling competence, their leadership skills, their biographies or the influence of other factors such as whether the incumbent government faced some scandal. The first forecast for the 2012 election was published on January 1, 2011, almost two years prior to Election Day. As in 2008, the forecasts were updated daily, or whenever new information became available.
In 2013, the PollyVote was launched in Germany to predict the German federal election of the same year.
For the presidential election in 2016, PollyVote augmented their reporting with computational writing for the campaign coverage , publishing articles for all of their predictions on their .

Method

The PollyVote demonstrates the benefits of combining forecasts by averaging predictions within and across several component methods. In its application for the U.S. presidential election, the PollyVote is currently based on five component methods: polls, prediction markets, expert judgment, political economy models, and index models. The PollyVote predicts the share of the popular two-party vote achieved by the candidate of the incumbent party.

Accuracy of the PollyVote

The PollyVote published forecasts prior to each of the three U.S. presidential elections, the 2006 U.S. House of Representatives election, and the 2013 German federal election. In addition, one analysis tested how the PollyVote would have performed for the three elections from 1992 to 2000. As expected, the application of the forecasting principles has led to accurate forecasts. Surprisingly, however, across the three U.S. Presidential elections, the forecast error was always lower than the error of each component methods. Comparisons have also been made with other methods. For example, forecasts of the 2012 election were also substantially more accurate than the closely watched forecasts from Nate Silver's model at FiveThirtyEight.com.

2004 U.S. presidential election

The 2004 PollyVote was launched in March 2004 and forecast a victory for President Bush over the 8 months that it was making forecasts. The final forecast published on the morning of the election predicted that President would receive 51.5% of the popular two-party vote, an error of 0.3 percentage points.

2008 U.S. presidential election

The 2008 PollyVote was launched in August 2007 and forecast a victory for Barack Obama over the 14 months that it was making daily forecasts. On Election Eve, it predicted that Obama would receive 53.0% of the popular two-party vote, an error of 0.7 percentage points.

2012 U.S. presidential election

The 2012 PollyVote was launched in January 2011 and forecast a victory for President Obama over the 22 months that it was making daily forecasts. On Election Eve, it predicted that Obama would receive 51.0% of the popular two-party vote, an error of 0.9 percentage points.

2016 U.S. presidential election

The 2016 PollyVote predicted that Hillary Clinton would win both popular vote and electoral vote.

2006 US House of Representatives election

PollyVote predicted the outcome of the 2006 U.S. House of Representatives Elections, forecasting that the Republicans would lose 23 seats, and thus, their majority in the House. The Republicans lost 30 seats and the House majority in those elections.

Perception

The results of the PollyVote project are regularly published in the academic community. Prior to the past elections, forecasts were published in Foresight and the New Scientist. Analyses of the accuracy of the PollyVote were published in the International Journal of Forecasting and PS: Political Science & Politics. In addition, scholars have referenced the PollyVote as a benchmark when assessing the validity of U.S. presidential election forecasts.
To date, the PollyVote predictions have been rarely cited in the popular press. In their IJF paper, the PollyVote team discusses several reasons why this might be the case: people have difficulties to understand the benefits of combining, people wrongly believe that they can identify the best forecast, and people think that the method of calculating averages is too simple. Another possible reason is that the PollyVote predictions are very stable and rarely change, whereas election observers and journalists are interested in excitement and newsworthiness.