Yiling Chen
Chao-Hsien Chu
David M. Pennock
ABSTRACT
In this paper, we examine the relative forecast accuracy of information markets versus expert aggregation. We leverage a unique data source of almost 2000 people's subjective probability judgments on 2003 US National Football League games and compare with the "market probabilities" given by two different information markets on exactly the same events. We combine assessments of multiple experts via linear and logarithmic aggregation functions to form pooled predictions. Prices in information markets are used to derive market predictions. Our results show that, at the same time point ahead of the game, information markets provide as accurate predictions as pooled expert assessments. In screening pooled expert predictions, we find that arithmetic average is a robust and efficient pooling function; weighting expert assessments according to their past performance does not improve accuracy of pooled predictions; and logarithmic aggregation functions offer bolder predictions than linear aggregation functions. The results provide insights into the predictive performance of information markets, and the relative merits of selecting among various opinion pooling methods.
- http://us.newsfutures.comGoogle Scholar
- http://www.biz.uiowa.edu/iem/Google Scholar
- http://www.hsx.com/Google Scholar
- http://www.ideosphere.com/fx/Google Scholar
- http://www.probabilityfootball.com/Google Scholar
- http://www.probabilitysports.com/Google Scholar
- http://www.tradesports.com/Google Scholar
- A. H. Ashton and R. H. Ashton. Aggregating subjective forecasts: Some empirical results. Management Science, 31:1499--1508, 1985.Google ScholarDigital Library
- R. P. Batchelor and P. Dua. Forecaster diversity and the benefits of combining forecasts. Management Science, 41:68--75, 1995. Google ScholarDigital Library
- N. Cesa-Bianchi, Y. Freund, D. Haussler, D. P. Helmbold, R. E. Schapire, and M. K. Warmuth. How to use expert advice. Journal of the ACM, 44(3):427--485, 1997. Google ScholarDigital Library
- K. Chen, L. Fine, and B. Huberman. Predicting the future. Information System Frontier, 5(1):47--61, 2003. Google ScholarDigital Library
- R. T. Clemen and R. L. Winkler. Combining probability distributions from experts in risk analysis. Risk Analysis, 19(2):187--203, 1999.Google ScholarCross Ref
- R. M. Cook. Experts in Uncertainty: Opinion and Subjective Probability in Science. Oxford University Press, New York, 1991.Google Scholar
- A. L. Delbecq, A. H. Van de Ven, and D. H. Gustafson. Group Techniques for Program Planners: A Guide to Nominal Group and Delphi Processes. Scott Foresman and Company, Glenview, IL, 1975.Google Scholar
- E. F. Fama. Efficient capital market: A review of theory and empirical work. Journal of Finance, 25:383--417, 1970.Google ScholarCross Ref
- R. Forsythe and F. Lundholm. Information aggregation in an experimental market. Econometrica, 58:309--47, 1990.Google ScholarCross Ref
- R. Forsythe, F. Nelson, G. R. Neumann, and J. Wright. Forecasting elections: A market alternative to polls. In T. R. Palfrey, editor, Contemporary Laboratory Experiments in Political Economy, pages 69--111. University of Michigan Press, Ann Arbor,MI, 1991.Google Scholar
- R. Forsythe, F. Nelson, G. R. Neumann, and J. Wright. Anatomy of an experimental political stock market. American Economic Review, 82(5):1142--1161, 1992.Google Scholar
- R. Forsythe, T. A. Rietz, and T. W. Ross. Wishes, expectations, and actions: A survey on price formation in election stock markets. Journal of Economic Behavior and Organization, 39:83--110, 1999.Google ScholarCross Ref
- S. French. Group consensus probability distributions: a critical survey. Bayesian Statistics, 2:183--202, 1985.Google Scholar
- C. Genest. A conflict between two axioms for combining subjective distributions. Journal of the Royal Statistical Society, 46(3):403--405, 1984.Google Scholar
- C. Genest. Pooling operators with the marginalization property. Canadian Journal of Statistics, 12(2):153--163, 1984.Google ScholarCross Ref
- C. Genest, K. J. McConway, and M. J. Schervish. Characterization of externally Bayesian pooling operators. Annals of Statistics, 14(2):487--501, 1986.Google ScholarCross Ref
- C. Genest and J. V. Zidek. Combining probability distributions: A critique and an annotated bibliography. Statistical Science, 1(1):114--148, 1986.Google ScholarCross Ref
- S. J. Grossman. An introduction to the theory of rational expectations under asymmetric information. Review of Economic Studies, 48(4):541--559, 1981.Google ScholarCross Ref
- F. A. Hayek. The use of knowledge in society. American Economic Review, 35(4):519--530, 1945.Google Scholar
- J. C. Jackwerth and M. Rubinstein. Recovering probability distribution from options prices. Journal of Finance, 51(5):1611--1631, 1996.Google ScholarCross Ref
- H. A. Linstone and M. Turoff. The Delphi Method: Techniques and Applications. Addison-Wesley, Reading, MA, 1975.Google Scholar
- P. A. Morris. Decision analysis expert use. Management Science, 20(9):1233--1241, 1974.Google ScholarCross Ref
- P. A. Morris. Combining expert judgments: A bayesian approach. Management Science, 23(7):679--693, 1977.Google ScholarDigital Library
- P. A. Morris. An axiomatic approach to expert resolution. Management Science, 29(1):24--32, 1983.Google ScholarDigital Library
- E. W. Noreen. Computer-Intensive Methods for Testing Hypotheses: An Introduction. Wiley and Sons, Inc., New York, 1989.Google Scholar
- D. M. Pennock, S. Lawrence, C. L. Giles, and F. A. Nielsen. The real power of artificial markets. Science, 291:987--988, February 2002.Google ScholarCross Ref
- D. M. Pennock, S. Lawrence, F. A. Nielsen, and C. L. Giles. Extracting collective probabilistic forecasts from web games. In Proceedings of the 7th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pages 174--183, San Francisco, CA, 2001. Google ScholarDigital Library
- C. Plott and S. Sunder. Rational expectations and the aggregation of diverse information in laboratory security markets. Econometrica, 56:1085--118, 1988.Google ScholarCross Ref
- E. Servan-Schreiber, J. Wolfers, D. M. Pennock, and B. Galebach. Prediction markets: Does money matter? Electronic Markets, 14(3):243--251, 2004.Google ScholarCross Ref
- M. Spann and B. Skiera. Internet-based virtual stock markets for business forecasting. Management Science, 49(10):1310--1326, 2003. Google ScholarDigital Library
- M. West. Bayesian aggregation. Journal of the Royal Statistical Society. Series A. General, 147(4):600--607, 1984.Google ScholarCross Ref
- R. L. Winkler. The consensus of subjective probability distributions. Management Science, 15(2):B61--B75, 1968.Google ScholarDigital Library
- J. Wolfers and E. Zitzewitz. Prediction markets. Journal of Economic Perspectives, 18(2):107--126, 2004.Google ScholarCross Ref
Index Terms
- Information markets vs. opinion pools: an empirical comparison
Recommendations
Information transparency in prediction markets
Prediction markets are designed and conducted for the primary purpose of aggregating information so that market prices forecast future events. In such markets, a group of traders buy and sell contracts and the payoff depends on unknown future events. ...
Forecasting stock returns based on information transmission across global markets using support vector machines
This paper provides evidence that forecasts based on global stock returns transmission yield better returns in day trading, for both developed and emerging stock markets. The study investigates the performance of global stock market price transmission ...
Modeling volatility in prediction markets
EC '09: Proceedings of the 10th ACM conference on Electronic commerceThere is significant experimental evidence that prediction markets are efficient mechanisms for aggregating information and are more accurate in forecasting events than traditional forecasting methods, such as polls. Interpretation of prediction market ...
Comments