Sharad Goel
Daniel M. Reeves
Duncan J. Watts
David M. Pennock
ABSTRACT
Citing recent successes in forecasting elections, movies, products, and other outcomes, prediction market advocates call for widespread use of market-based methods for government and corporate decision making. Though theoretical and empirical evidence suggests that markets do often outperform alternative mechanisms, less attention has been paid to the magnitude of improvement. Here we compare the performance of prediction markets to conventional methods of prediction, namely polls and statistical models. Examining thousands of sporting and movie events, we find that the relative advantage of prediction markets is surprisingly small, as measured by squared error, calibration, and discrimination. Moreover, these domains also exhibit remarkably steep diminishing returns to information, with nearly all the predictive power captured by only two or three parameters. As policy makers consider adoption of prediction markets, costs should be weighed against potentially modest benefits.
- K. J. Arrow, R. Forsythe, M. Gorham, R. Hahn, R. Hanson, J. O. Ledyard, S. Levmore, R. Litan, P. Milgrom, F. D. Nelson, G. R. Neumann, M. Ottaviani, T. C. Schelling, R. J. Shiller, V. L. Smith, E. Snowberg, C. R. Sunstein, P. C. Tetlock, P. E. Tetlock, H. R. Varian, J. Wolfers, and E. Zitzewitz. The promise of prediction markets. Science, 320(5878):877--878, 2008.Google Scholar
Cross Ref
- J. E. Berg, R. Forsythe, F. D. Nelson, and T. A. Rietz. Results from a dozen years of election futures markets research. In C. R. Plott and V. Smith, editors, Handbook of Experimental Economics Results, Volume 1, pages 742--751. North Holland, 2008.Google ScholarCross Ref
- J. E. Berg and T. A. Rietz. Prediction markets as decision support systems. Information Systems Frontiers, 5(1):79--93, 2003. Google ScholarDigital Library
- Center for Gaming Research, University of Nevada, Las Vegas. 2008 Nevada gaming statewide revenue breakdown.Google Scholar
- Y. Chen, C. Chu, T. Mullen, and D. Pennock. Information markets vs. opinion pools: An empirical comparison. In Proceedings of the 6th ACM conference on Electronic commerce, page 67. ACM, 2005. Google ScholarDigital Library
- Y. Chen and A. M. Kwasnica. Security design and information aggregation in markets, 2006.Google Scholar
- J. D. Christiansen. Prediction markets: Practical experiments in small markets and behaviours observed. Journal of Prediction Markets, 1(1), 2006.Google Scholar
- R. Clemen. Combining forecasts: A review and annotated bibliography. International Journal of Forecasting, 5(4):559--583, 1989.Google ScholarCross Ref
- V. Dani, O. Madani, D. Pennock, S. Sanghai, and B. Galebach. An empirical comparison of algorithms for aggregating expert predictions. In Proceedings of the Conference on Uncertainty in Artificial Intelligence (UAI). Citeseer, 2006.Google Scholar
- R. S. Erikson and C. Wlezien. Are political markets really superior to polls as election predictors? Public Opinion Quarterly, 72(2):190--215, 2008.Google ScholarCross Ref
- E. Fama. The behavior of stock-market prices. Journal of business, 38(1):34, 1965.Google ScholarCross Ref
- M. Ferrari and A. Rudd. Investing in movies. Journal of Asset Management, 9(1):22--40, 2008.Google ScholarCross Ref
- R. Forsythe and R. Lundholm. Information aggregation in an experimental market. Econometrica, 58(2):309--347, 1990.Google ScholarCross Ref
- R. Forsythe, F. D. Nelson, and G. R. Neumann. Anatomy of an experimental political stock market. American Economic Review, 82(5):1142--1161, 1992.Google Scholar
- R. Forsythe, T. A. Rietz, and T. W. Ross. Wishes, expectations, and actions: A survey on price formation in election stock markets. Journal of Economic Behavior & Organization, 39:83--110, 1999.Google ScholarCross Ref
- A. Gelman, J. Carlin, H. Stern, and D. Rubin. Bayesian data analysis. Chapman & Hall, 2003.Google Scholar
- S. Goel, J. Hofman, S. Lahaie, D. M. Pennock, and D. J. Watts. What can search predict? Technical Report.Google Scholar
- A. Graefe and J. S. Armstrong. Predicting elections from the most important issue facing the country, 2009.Google Scholar
- R. W. Hahn and P. C. Tetlock, editors. Information Markets: A New Way of Making Decisions. AEI-Brookings Press, 2006.Google Scholar
- R. Hanson. Decision markets. IEEE Intelligent Systems, 14(3):16--19, 1999.Google ScholarDigital Library
- R. Hanson and R. Oprea. Manipulators increase information market accuracy, 2004.Google Scholar
- R. Hanson, R. Oprea, and D. Porter. Information aggregation and manipulation in an experimental market. Journal of Economic Behavior & Organization, 60(4):449--459, 2006.Google ScholarCross Ref
- F. A. Hayek. The use of knowledge in society. American Economic Review, 35(4):519--530, 1945.Google Scholar
- P. Healy, J. Ledyard, S. Linardi, and R. J. Lowery. Prediction market alternatives for complex environments. In Conference on Auctions, Market Mechanisms and Their Applications, 2009.Google Scholar
- J. Howe. Crowdsourcing: Why the Power of the Crowd Is Driving the Future of Business. Crown Business, New York, 2008. Google ScholarDigital Library
- J. C. Jackwerth and M. Rubenstein. Recovering probability distributions from options prices. Journal of Finance, 51(5):1611--1631, 1996.Google ScholarCross Ref
- F. Kleeman, G. G. Voss, and K. Rieder. Un(der)paid innovators: The commercial utilization of consumer work through crowdsourcing. Science, Technology & Innovation Studies, 4(1):5--26, 2008.Google Scholar
- J. Ledyard, R. Hanson, and T. Ishikida. An experimental test of combinatorial information markets. Journal of Economic Behavior & Organization, 69(2):182--189, 2009.Google ScholarCross Ref
- S. Makridakis and M. Hibon. The M3-Competition: results, conclusions and implications. International Journal of Forecasting, 16:451--476, 2000.Google ScholarCross Ref
- S. Makridakis, M. Hibon, and C. Moser. Accuracy of forecasting: An empirical investigation. Journal of the Royal Statistical Society. Series A, 142(2):97--145, 1979.Google ScholarCross Ref
- S. Makridakis, R. M. Hogarth, and A. Gaba. Forecasting and uncertainty in the economic and business world. International Journal of Forecasting, In press, 2009.Google Scholar
- J. Muth. Rational expectations and the theory of price movements. Econometrica, 29(3):315--335, 1961.Google ScholarCross Ref
- K. Oliven and T. A. Rietz. Suckers are born, but markets are made: Individual rationality, arbitrage and market efficiency on an electronic futures market. Management Science, 50(3):336--351, 2004. Google ScholarDigital Library
- D. M. Pennock, S. Debnath, E. J. Glover, and C. L. Giles. Modeling information incorporation in markets, with application to detecting and explaining events. In Proceedings of the Eighteenth Conference on Uncertainty in Artificial Intelligence, pages 405--413, Edmonton, CA, 2002. Association for Uncertainty in Artificial Intelligence. Google ScholarDigital Library
- C. R. Plott and S. Sunder. Efficiency of experimental security markets with insider information: An application of rational-expectations models. Journal of Political Economy, 90(4):663--698, 1982.Google ScholarCross Ref
- C. R. Plott and S. Sunder. Rational expectations and the aggregation of diverse information in laboratory security markets. Econometrica, 56(5):1085--1118, 1988.Google ScholarCross Ref
- C. Polk, R. Hanson, J. Ledyard, and T. Ishikida. Policy analysis market: An electronic commerce application of a combinatorial information market., 2003. Google ScholarDigital Library
- P. W. Rhode and K. S. Strumpf. Manipulating political stock markets: A field experiment and a century of observational data, 2006.Google Scholar
- R. Roll. Orange juice and weather. American Economic Review, 74(5):861--880, 1984.Google Scholar
- R. N. Rosett. Gambling and rationality. Journal of Political Economy, 73(6):595--607, 1965.Google ScholarCross Ref
- P. Samuelson. Proof that properly anticipated prices fluctuate randomly. Management Review, 6(2), 1965.Google Scholar
- L. Savage. Elicitation of personal probabilities and expectations. Journal of the American Statistical Association, 66(336):783--801, 1971.Google ScholarCross Ref
- C. Schmidt and A. Werwatz. How accurately do markets predict the outcome of an event? The Euro 2000 soccer championships experiment, 2002.Google Scholar
- E. Servan-Schreiber, J. Wolfers, D. Pennock, and B. Galebach. Prediction markets: does money matter? Electronic Markets, 14(3):243--251, 2004.Google ScholarCross Ref
- A. Serwer. Making a market in (almost) anything. Fortune, Monday, July 25, 2005.Google Scholar
- B. J. Sherrick, P. Garcia, and V. Tirupattur. Recovering probabilistic information from options markets: Tests of distributional assumptions. Journal of Futures Markets, 16(5):545--560, 1996.Google ScholarCross Ref
- W. W. Snyder. Horse racing: Testing the efficient markets model. Journal of Finance, 33(4):1109--1118, 1978.Google ScholarCross Ref
- S. Sunder. Experimental asset markets. In J. H. Kagel and A. E. Roth, editors, The Handbook of Experimental Economics, pages 445--500. Princeton University Press, Princeton, NJ, 1995.Google Scholar
- C. R. Sunstein. Group judgements: Statistical means, deliberation, and information markets. New York Law Review, 80(3):962--1049, 2005.Google Scholar
- P. C. Tetlock. Does liquidity affect securities market efficiency?, 2006.Google Scholar
- P. C. Tetlock. How efficient are information markets? evidence from an online exchange, 2006.Google Scholar
- P. E. Tetlock. Expert Political Judgment: How Good Is It? How Can We Know? Princeton University Press, Princeton, NJ, 2005.Google Scholar
- R. H. Thaler and W. T. Ziemba. Anomalies: Parimutuel betting markets: Racetracks and lotteries. Journal of Economic Perspectives, 2(2):161--174, 1988.Google ScholarCross Ref
- G. Tziralis and I. Tatsiopoulos. Prediction markets: An extended literature review. Journal of Prediction Markets, 1(1), 2006.Google Scholar
- M. Weitzman. Utility analysis and group behavior: An empirical study. Journal of Political Economy, 73(1):18--26, 1965.Google ScholarCross Ref
- J. Wolfers and E. Zitzewitz. Prediction markets. The Journal of Economic Perspectives, 18(2):107--126, 2004.Google ScholarCross Ref
- J. Wolfers and E. Zitzewitz. Using markets to inform policy: The case of the Iraq war, 2006.Google Scholar
Index Terms
- Prediction without markets
Recommendations
Arbitrage Opportunities and Market-Making Traders in Prediction Markets
CECANDEEE '08: Proceedings of the 2008 10th IEEE Conference on E-Commerce Technology and the Fifth IEEE Conference on Enterprise Computing, E-Commerce and E-ServicesPrediction markets are a promising approach for predicting uncertain future events and developments. These markets will work well if they are efficient and in efficient markets, one does not expect arbitrage opportunities to be persistent. This paper ...
Corporate prediction markets: evidence from google, ford, and firm X
EC '14: Proceedings of the fifteenth ACM conference on Economics and computationDespite the popularity of prediction markets among economists, businesses and policymakers have been slow to adopt them in decision making. Most studies of prediction markets outside the lab are from public markets with large trading populations. ...
Modeling volatility in prediction markets
EC '09: Proceedings of the 10th ACM conference on Electronic commerceThere is significant experimental evidence that prediction markets are efficient mechanisms for aggregating information and are more accurate in forecasting events than traditional forecasting methods, such as polls. Interpretation of prediction market ...
Comments