Abstract
The technique bag-of-little bootstrap provides statistical estimates equivalent to the ones of bootstrap in a tiny fraction of the time required by bootstrap. In this work, we propose to combine bag-of-little bootstrap into an ensemble of classifiers composed of random trees. We show that using this bootstrapping procedure, instead of standard bootstrap samples, as the ones used in random forest, can dramatically reduce the training time of ensembles of classifiers. In addition, the experiments carried out illustrate that, for a wide range of training times, the proposed ensemble method achieves a generalization error smaller than that achieved by random forest.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsReferences
Bache, K., Lichman, M.: UCI machine learning repository (2013). http://archive.ics.uci.edu/ml
Breiman, L.: Bagging predictors. Mach. Learn. 26, 123–140 (1996)
Breiman, L.: Random forests. Mach. Learn. 45(1), 5–32 (2001)
Breiman, L., Friedman, J.H., Olshen, R.A., Stone, C.J.: Classification and Regression Trees. Chapman & Hall, New York (1984)
Chen, T., Guestrin, C.: XGBoost: a scalable tree boosting system. In: Proceedings of the 22Nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, KDD 2016, pp. 785–794. ACM, New York (2016)
Efron, B.: Bootstrap methods: another look at the jackknife. Ann. Stat. 7(1), 1–26 (1979)
Fernández-Delgado, M., Cernadas, E., Barro, S., Amorim, D.: Do we need hundreds of classifiers to solve real world classification problems? J. Mach. Learn. Res. 15, 3133–3181 (2014)
Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. J. Comput. Syst. Sci. 55(1), 119–139 (1997)
Hall, P., Samworth, R.J.: Properties of bagged nearest neighbour classifiers. J. Roy. Stat. Soc. Ser. B 67(3), 363–379 (2005)
Kleiner, A., Talwalkar, A., Sarkar, P., Jordan, M.: A scalable bootstrap for massive data. J. Roy. Stat. Soc. Ser. B 76, 795–816 (2014)
Kleiner A., Talwalkar A., Sarkar, P., Jordan, M.: The big data bootstrap. In: ICML (2012)
Martínez-Muñoz, G., Suárez, A.: Out-of-bag estimation of the optimal sample size in bagging. Pattern Recognit. 43(1), 143–152 (2010)
Mason, L., Baxter, J., Bartlett, P., Frean, M.: Boosting algorithms as gradient descent. In: Advances in Neural Information Processing Systems, vol. 12, pp. 512–518. MIT Press (2000)
Zaman, F., Hirose, H.: Effect of subsampling rate on subbagging and related ensembles of stable classifiers. In: Chaudhury, S., Mitra, S., Murthy, C.A., Sastry, P.S., Pal, S.K. (eds.) PReMI 2009. LNCS, vol. 5909, pp. 44–49. Springer, Heidelberg (2009). https://doi.org/10.1007/978-3-642-11164-8_8
Acknowledgments
The research has been supported by the Spanish Ministry of Economy, Industry, and Competitiveness project TIN2016-76406-P, and Comunidad de Madrid, project CASI-CAM-CM (S2013/ICE-2845).
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2018 Springer Nature Switzerland AG
About this paper
Cite this paper
de Viña, P., Martínez-Muñoz, G. (2018). Using Bag-of-Little Bootstraps for Efficient Ensemble Learning. In: Kůrková, V., Manolopoulos, Y., Hammer, B., Iliadis, L., Maglogiannis, I. (eds) Artificial Neural Networks and Machine Learning – ICANN 2018. ICANN 2018. Lecture Notes in Computer Science(), vol 11139. Springer, Cham. https://doi.org/10.1007/978-3-030-01418-6_53
Download citation
DOI: https://doi.org/10.1007/978-3-030-01418-6_53
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-01417-9
Online ISBN: 978-3-030-01418-6
eBook Packages: Computer ScienceComputer Science (R0)