Skip to main content
SpringerLink
Log in

On a method for constructing ensembles of regression models

  • Topical Issue
  • Published:
Automation and Remote Control Aims and scope Submit manuscript

Abstract

We propose a method for constructing ensembles of regression models and perform its theoretical and experimental analysis.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Similar content being viewed by others

References

  1. Breiman, L., Bagging Predictors, Mach. Learn., 1996, vol. 24, no. 2, pp. 123–140.

    MathSciNet  MATH  Google Scholar 

  2. Duffy, N. and Helmbold, D., Boosting Methods for Regression, Mach. Learn., 2002, vol. 47, no. 2, pp. 153–200.

    Article  MATH  Google Scholar 

  3. Ueda, N. and Nakano, R., Generalization Error of Ensemble Estimators, Proc. Int. Conf. on Neural Networks, 1996, pp. 90–95.

    Google Scholar 

  4. Bruhlmann, P. and Yu, B., Explaining Bagging, Research Report no. 92, Statistics Department, University of California at Berkeley, 2000.

    Google Scholar 

  5. Friedman, J., Greedy Function Approximation: A Gradient Boosting Machine, Ann. Statist., 2001, vol. 29, no. 5, pp. 1189–1232.

    Article  MathSciNet  MATH  Google Scholar 

  6. Shrestha, D.L. and Solomatine, D.P., Experiments with AdaBoosting.RT, an Improved Boosting Scheme for Regression, Neur. Computat., 2006, vol. 18, no. 7, pp. 1678–1710.

    Article  MATH  Google Scholar 

  7. Vapnik, V.N. and Chervonenkis, A.Ja., Ordered Risk Minimization. I, II, Autom. Remote Control, 1974, vol. 35, no. 8, part 1, pp. 1226–1235; no. 9, part 1, pp. 1403–1412.

    MathSciNet  MATH  Google Scholar 

  8. Vapnik, V.N. and Mikhal’skii, A.I., The Search for Dependencies by the Method of Ordered Minimization of Risk, Autom. Remote Control, 1974, vol. 35, no. 10, part 1, pp. 1615–1624.

    Google Scholar 

  9. Anthony, M. and Bartlett, P.L., Neural Network Learning: Theoretical Foundations. Cambridge: Cambridge Univ. Press, 2002.

    Google Scholar 

  10. Burnaev, E., Belyaev, M., and Prihodko, P., About Hybrid Algorithm for Tuning of Parameters in Approximation Based on Linear Expansions in Parametric Functions, Proc. 8th Int. Conf. “Intelligent Information Processing,” Republic of Cyprus, Paphos, October 17–24, 2010, pp. 200–205.

    Google Scholar 

  11. Hastie, T., Tibshirani, R., and Friedman, J., The Elements of Statistical Learning: Data Mining, Inference and Prediction, New York: Springer, 2008.

    Google Scholar 

  12. Pinkus, A., Approximation Theory of the MLP Model in Neural Networks, Acta Numerica, 1999, vol. 8, pp. 143–195.

    Article  MathSciNet  Google Scholar 

  13. Zhou, Z., Wu, J., and Tang, W., Ensembling Neural Networks: Many Could be Better Than All, Artif. Intell., 2002, vol. 137, no. 1, pp. 239–263.

    Article  MathSciNet  MATH  Google Scholar 

  14. Freund, Y. and Schapire, R., A Decision-Theoretic Generalization of On-line Learning and an Application to Boosting, Proc. Second Eur. Conf. Comput. Learning Theory, 1995, pp. 23–37.

    Chapter  Google Scholar 

  15. Zhou, Z.H., Ensemble Methods: Foundations and Algorithms, Boca Raton: Chapman & Hall/CRC, 2012.

    Google Scholar 

  16. Marquardt, D., An Algorithm for Least-Squares Estimation of Nonlinear Parameters, SIAM J. Appl. Math., 1963, vol. 11, no. 2, pp. 431–441.

    Article  MathSciNet  MATH  Google Scholar 

  17. Khaikin, C., Neironnye seti. Polnyi kurs (Neural Networks. A Complete Course), Moscow: Vil’yams, 2006.

    Google Scholar 

  18. Neural Network Toolbox Documentation, The MathWorks, Inc. (http://www.mathworks.com/help/nnet/index.html).

  19. Belyaev, M.G. and Lyubin, A.D., Characteristic Features of an Optimization Problem Arising in Constructing an Approximation for a Multidimensional Dependence, Tr. konf. “Informatsionnye tekhnologii i sistemy” (ITiS’11) (Proc. Conf. “Informational Technologies and Systems”), 2011, pp. 415–422.

    Google Scholar 

  20. Global Optimization Test Problems (http://www-optima.amp.i.kyoto-u.ac.jp/member/student/hedar/Hedar-files/TestGO-files/Page364.htm).

  21. Drago, G.P. and Ridella, S., Statistically Controlled Activation Weight Initialization (SCAWI), IEEE Trans. Neur. Networks, 1992, vol. 3, no. 4, pp. 627–631.

    Article  Google Scholar 

  22. Dolan, E.D. and More, J., Benchmarking Optimization Software with Performance Profiles, Math. Program. Manuscript, 2002, vol. 91, no. 2, pp. 201–213.

    Article  MathSciNet  MATH  Google Scholar 

  23. Friedman, J., Hastie, T., and Tibshirani, R., Additive Logistic Regression: A Statistical View of Boosting, Ann. Statist., 2000, vol. 28, no. 2, pp. 337–407.

    Article  MathSciNet  MATH  Google Scholar 

  24. Friedman, J., Greedy Function Approximation: A Gradient Boosting Machine, Ann. Statist., 2001, vol. 29, no. 5, pp. 1189–1232.

    Article  MathSciNet  MATH  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. SJC “Datadvance”, Moscow, Russia

    E. V. Burnaev & P. V. Prikhod’ko

  2. Institute for Information Transmission Problems (Kharkevich Institute), Russian Academy of Sciences, Moscow, Russia

    E. V. Burnaev & P. V. Prikhod’ko

  3. Moscow Institute of Physics and Technology (State University), Dolgoprudny, Russia

    E. V. Burnaev & P. V. Prikhod’ko

Authors
  1. E. V. Burnaev
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. P. V. Prikhod’ko
    View author publications

    You can also search for this author in PubMed Google Scholar

Additional information

Original Russian Text © E.V. Burnaev, P.V. Prikhod’ko, 2013, published in Avtomatika i Telemekhanika, 2013, No. 10, pp. 36–54.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Burnaev, E.V., Prikhod’ko, P.V. On a method for constructing ensembles of regression models. Autom Remote Control 74, 1630–1644 (2013). https://doi.org/10.1134/S0005117913100044

Download citation

  • Received:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1134/S0005117913100044

Keywords

Search

Navigation