Skip to main content
Log in

Frequentist model averaging estimation: a review

  • Published:
Journal of Systems Science and Complexity Aims and scope Submit manuscript

Abstract

In applications, the traditional estimation procedure generally begins with model selection. Once a specific model is selected, subsequent estimation is conducted under the selected model without consideration of the uncertainty from the selection process. This often leads to the underreporting of variability and too optimistic confidence sets. Model averaging estimation is an alternative to this procedure, which incorporates model uncertainty into the estimation process. In recent years, there has been a rising interest in model averaging from the frequentist perspective, and some important progresses have been made. In this paper, the theory and methods on frequentist model averaging estimation are surveyed. Some future research topics are also discussed.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. J. M. Bates and C. M. J. Granger, The combination of forecasts, Operations Research Quarterly, 1969, 20: 451–468.

    Article  Google Scholar 

  2. D. A. Bessler and J. A. Brandt, Forecasting livestock prices with individual and composite methods, Applied Economics, 1981, 13: 513–522.

    Article  Google Scholar 

  3. R. T. Clemen and R. L. Winkler, Combining economic forecasts, Journal of Business and Economic Statistics, 1986, 4: 39–46.

    Article  Google Scholar 

  4. P. Newbold and C. W. J. Granger, Experience with forecasting univariate time series and the combination of forecasts, Journal of the Royal Statistical Society, Series A, 1974, 2: 131–165.

    Article  MathSciNet  Google Scholar 

  5. R. F. Phillips, Composite forecasting: An integrated approach and optimality reconsidered, Journal of Business & Economic Statistics, 1987, 5: 389–395.

    Article  Google Scholar 

  6. M. A. Clyde and E. George, Model uncertainty, Statistical Science, 2004, 19: 81–94.

    Article  MATH  MathSciNet  Google Scholar 

  7. D. Draper, Assessment and propagation of model uncertainty, Journal of the Royal Statistical Society: Series B, 1995, 57: 45–70.

    MATH  MathSciNet  Google Scholar 

  8. J. A. Hoeting, D. Madigan, A. E. Raftery, and C. T. Volinsky, Bayesian model averaging: A tutorial, Statistical Science, 1999, 14: 382–417.

    Article  MATH  MathSciNet  Google Scholar 

  9. J. R. Magnus, O. Powell, and P. Prüfer, A comparison of two averaging techniques with an application to growth empirics, Journal of Econometrics, 2009, in press, doi:10.1016/j.jeconom.2009.07.004.

    Google Scholar 

  10. A. E. Reftery, D. Madigen, and J. A. Hoeting, Bayesian model averaging for regression models, Journal of the American Statistical Association, 1997, 92: 179–191.

    Article  MathSciNet  Google Scholar 

  11. N. L. Hjort and G. Claeskens, Frequestist model average estimators, Journal of the American Statistical Association, 2003, 98: 879–899.

    Article  MATH  MathSciNet  Google Scholar 

  12. S. T. Buckland, K. P. Burnham, and N. H. Augustin, Model selection: An integral part of inference, Biometrics, 1997, 53: 603–618.

    Article  MATH  Google Scholar 

  13. J. R. Magnus and J. Durbin, Estimation of regression coefficients of interest when other regression coefficients are of no interest, Econometrica, 1999, 67: 639–643.

    Article  MATH  MathSciNet  Google Scholar 

  14. Y. Yang, Adaptive regression by mixing, Journal of the American Statistical Association, 2001, 96: 574–586.

    Article  MATH  MathSciNet  Google Scholar 

  15. K. P. Burnham and D. R. Anderson, Model Selection and Multimodel Inference: A Practical Infromation-Theoretic Approach, Springer, New York, 2002.

    Google Scholar 

  16. G. Leung and A. R. Barron, Infromation theory and mixing least-squares regressions, Information Theory, IEEE Transactions, 2006, 52: 3396–3410.

    Article  MathSciNet  Google Scholar 

  17. B. E. Hansen, Least squares model averaging, Econometrica, 2007, 75: 1175–1189.

    Article  MATH  MathSciNet  Google Scholar 

  18. G. Claeskens, C. Croux, and J. ven Kerckhoven, Variable selection for logit regression using a prediction-focused information criterion, Biometrics, 2006, 62: 972–979.

    Article  MATH  MathSciNet  Google Scholar 

  19. G. Kapetanios, V. Labhard, and S. Price, Forecasting using Bayesian and information-theoretic model averaging, Journal of Business and Economic Statistics, 2008, 26: 33–41.

    Article  MathSciNet  Google Scholar 

  20. M. H. Pesaran, C. Schleicher, and P. Zaffaroni, Model averaging in risk management with an application to futures markets, Journal of Empirical Finance, 2009, 16: 280–305.

    Article  Google Scholar 

  21. A. T. K. Wan and X. Zhang, On the use of model averaging in tourism research, Annals of Tourism Research, 2009, 36: 525–532.

    Article  Google Scholar 

  22. G. Claeskens and N. L. Hjort, Model Selection and Model Averaging, Cambridge University Press, New York, 2008.

    MATH  Google Scholar 

  23. G. G. Judge and M. E. Bock, The Statistical Implications of Pre-test and Stein-rule Estimators in Econometrics, North-Holland, Amsterdam, 1978.

    MATH  Google Scholar 

  24. N. L. Hjort and G. Claeskens, Rejoinder, Journal of the American Statistical Association, 2003, 98: 938–945.

    Article  Google Scholar 

  25. N. L. Hjort and G. Claeskens, Focused information criteria and model averaging for the Cox hazard regression model, Journal of the American Statistical Association, 2006, 110: 1449–1464.

    Article  MathSciNet  Google Scholar 

  26. H. Akaike, Maximum likelihood identification of Gaussian autoregression moving average models, Biometrika, 1973, 60: 255–265.

    Article  MATH  MathSciNet  Google Scholar 

  27. G. Claeskens and R. J. Carroll, An asymptotic theory for model selection inference in general semiparametric problems, Biometrika, 2007, 94: 249-265.

    Article  MATH  MathSciNet  Google Scholar 

  28. H. Wang, Frequentist model averaging estimation, Master Thesis, Academy of Mathematics and Systems Science, Chinese Academy of Sciences, Beijing, 2009.

  29. D. Danilov and J. R. Magnus, On the harm that ignoring pretesting can cause, Journal of Econometrics, 2004, 122: 27–46.

    Article  MathSciNet  Google Scholar 

  30. D. Danilov and J. R. Magnus, Forecast accuracy after pretesting with an application to the stock market, Journal of Forecasting, 2004, 23: 251–274.

    Article  Google Scholar 

  31. G. H. Zou, A. T. K. Wan, X. Wu, and T. Chen, Estimation of regression coefficients of interest when other regression coefficient are of no interest: The case of non-normal errors, Statistics & Probability Letters, 2007, 77: 803–810.

    Article  MATH  MathSciNet  Google Scholar 

  32. K. P. Burnham and D. R. Anderson, Multimodel inferenc uderstanding AIC and BIC in model selection, Sociological Methods & Research, 2004, 33: 261–304.

    Article  MathSciNet  Google Scholar 

  33. F. E. Turheimer, R. Hinz, and V. J. Cunningham, On the undecidability among kinetic models: From model selection to model averaging, Journal of Cerbral Blood Flow & Metabolism, 2003, 23: 490–498.

    Article  Google Scholar 

  34. E. J. Wagenmakes and S. Farrell, AIC model selecion using Akaike weights, Psychonomic Bulletin & Review, 2004, 11: 192–196.

    Google Scholar 

  35. C. L. Mallows, Some comments on C p , Technometrics, 1973, 15: 661–675.

    Article  MATH  Google Scholar 

  36. B. E. Hansen, Challenges for econometric model selection, Econometric Theory, 2005, 21: 60–68.

    MATH  MathSciNet  Google Scholar 

  37. P. Kabaila, On variable selection in linear regression, Econometric Theory, 2002, 18: 913–925.

    Article  MATH  MathSciNet  Google Scholar 

  38. B. E. Hansen, Least squares forecast averaging, Journal of Econometrics, 2008, 146: 342–350.

    Article  MathSciNet  Google Scholar 

  39. B. E. Hansen, Averaging estimators for autoregressions with a near unit root, Journal of Econometrics, 2009, forthcoming.

  40. A. T. K. Wan, X. Zhang, and G. Zou, Least squares model combining by Mallows criterion, Technical Report, Department of Management Sciences, City University of Hong Kong, 2009.

  41. B. E. Hansen and J. S. Racine, Jacknife model averaging, Technical Report, Department of Economics, University of Wisconsin-Madison, 2009.

  42. H. Liang, G. Zou, and X. Zhang, Choice of weights for frequentist model average estimators, Technical Report, Department of Biostatistics and Computational Biology, University of Rochester, 2009.

  43. T. H. Kim and H. White, James-Stein type estimators in large samples with application to the least absolute deviations estimator, Journal of the American Statistical Association, 2001, 96: 697–705.

    Article  MATH  MathSciNet  Google Scholar 

  44. H. Liang, G. Zou, A. T. K. Wan, and X. Zhang, On optimal weight choice in a frequentist model average estimator, Technical Report, Department of Biostatistics and Computational Biology, University of Rochester, 2009.

  45. Y. Yang, Regression with multiple candidate models: Selecting or mixing? Statistica Sinica, 2003, 13: 783–809.

    MATH  MathSciNet  Google Scholar 

  46. Z. Yuan and Y. Yang, Combining linear regression models: When and how? Journal of the American Statistical Association, 2005, 100: 1202–1204.

    Article  MATH  MathSciNet  Google Scholar 

  47. M. Schomaker, A. T. K. Wan, and C. Heumann, Frequentist model averaging with missing observations, Computational Statistics and Data Analysis, 2009, in press, doi:10.1016/j.csda.2009.07.023.

    Google Scholar 

  48. J. Fan and R. Li, Variable selection via nonconcave penalized likelihood and its oracle properties, Journal of the American Statistical Association, 2001, 96: 1348–1360.

    Article  MATH  MathSciNet  Google Scholar 

  49. H. Zou, The adaptive Lasso and its oracle properties, Journal of the American Statistical Association, 2006, 101: 1418–1429.

    Article  MATH  MathSciNet  Google Scholar 

Download references

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Guohua Zou.

Additional information

This research is supported by the National Natural Science Foundation of China under Grant Nos. 70625004, 10721101, and 70221001.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Wang, H., Zhang, X. & Zou, G. Frequentist model averaging estimation: a review. J Syst Sci Complex 22, 732–748 (2009). https://doi.org/10.1007/s11424-009-9198-y

Download citation

  • Received:

  • Revised:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11424-009-9198-y

Key words

Navigation