Abstract
To reduce error in generalization, a great number of work is carried out on the classifiers aggregation methods in order to improve generally, by voting techniques, the performance of a single classifier. Among these methods of aggregation, we find the Boosting which is most practical thanks to the adaptive update of the distribution of the examples aiming at increasing in an exponential way the weight of the badly classified examples. However, this method is blamed because of overfitting, and the convergence speed especially with noise. In this study, we propose a new approach and modifications carried out on the algorithm of AdaBoost. We will demonstrate that it is possible to improve the performance of the Boosting, by exploiting assumptions generated with the former iterations to correct the weights of the examples. An experimental study shows the interest of this new approach, called hybrid approach.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Vezhnevets, A., Vezhnevets, V.: Modest adaboost: Teaching adaboost to generalize better, Moscow State University (2002)
Bauer, E., Kohavi, R.: An empirical comparison of voting classification algorithms: Bagging, boosting, and variants. Machine Learning 24, 173–202 (1999)
Breiman, L.: Bagging predictors. Machine Learning 26, 123–140 (1996)
Brodley, C.E., Friedl, M.A.: Identifying and eliminating mislabeled training instances. In: AAAI/IAAI, vol. 1, pp. 799–805 (1996)
Dharmarajan, R.: An effecient boosting algorithm for combining preferences. Technical report, MIT (September 1999)
Dietterich, T.G.: An experimental comparison of three methods for constructing ensembles of decision trees: bagging, boosting, and randomization. Machine Learning, 1–22 (1999)
Dietterich, T.G.: Ensemble methodes in machine learning. In: First International Workshop on Multiple ClassifierSystems, pp. 1–15 (2000)
Domingo, C., Watanabe, O.: Madaboost: A modification of adaboost. In: Proc. 13th Annu. Conference on Comput. Learning Theory, pp. 180–189. Morgan Kaufmann, San Francisco (2000)
Friedman, J., Hastie, T., Tibshirani, R.: Additive logistic regression: a statistical view of boosting. Dept. of Statistics, Stanford University Technical Report (1998)
Friedman, J.H., Popescu, B.E.: Predictive learning via rule ensembles (technical report). Stanford University (7) (2005)
Kohavi, R.: A study of cross-validation and bootstrap for accuracy estimation and model selection. In: International Joint Conference on Artificial Intelligence (IJCAI) (1995)
Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. In: Information and computation, vol. 24, pp. 212–261 (1994)
Maclin, R.: Boosting classifiers regionally. In: AAAI/IAAI, pp. 700–705 (1998)
McDonald, R., Hand, D., Eckley, I.: An empirical comparison of three boosting algorithms on real data sets with artificial class noise. In: Fourth International Workshop on Multiple Classifier Systems, pp. 35–44 (2003)
Meir, R., El-Yaniv, R., Ben-David, S.: Localized boosting. In: Proc. 13th Annu. Conference on Comput. Learning Theory, pp. 190–199. Morgan Kaufmann, San Francisco (2000)
Rtsch, G.: Ensemble learning methods for classification. Master’s thesis, Dep. of computer science, University of Potsdam (April 1998)
Rätsch, G., Onoda, T., Müller, K.-R.: Soft margins for adaboost. Mach. Learn. 42(3), 287–320 (2001)
Schapire, R.E., Singer, Y.: Improved boosting algorithms using confedence rated predictions. Machine Learning 37(3), 297–336 (1999)
Sebban, M., Suchier, H.M.: tude sur amlioration du boosting: rduction de l’erreur et acclration de la convergence. Journal lectronique d’intelligence artificielle, 200–214 (2003)
Servedio, R.A.: Smooth boosting and learning with malicious noise. In: Helmbold, D.P., Williamson, B. (eds.) COLT 2001 and EuroCOLT 2001. LNCS (LNAI), vol. 2111, pp. 473–489. Springer, Heidelberg (2001)
Shapire, R.: The strength of weak learnability. Machine Learning 5, 197–227 (1990)
Kwek, S., Nguyen, C.: iboost: Boosting using an instance-based exponential weighting scheme. In: Thirteenth European Conference on Machine Learning, pp. 245–257 (2002)
Stolfo, S.J., Fan, W., Lee, W., Prodromidis, A., Chan, P.K.: Cost-based modeling and evaluation for data mining with application to fraud and intrusion detection (1999)
Torre, F.: Globoost: Boosting de moindres gnraliss. Technical report, GRAppA - Universit Charles de Gaulle - Lille 3 (September 2004)
Wilson, D.R., Martinez, T.R.: Reduction techniques for instance-based learning algorithms. Machine Learning 38(3), 257–286 (2000)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2009 Springer-Verlag Berlin Heidelberg
About this chapter
Cite this chapter
Bahri, E., Lallich, S., Nicoloyannis, N., Mondher, M. (2009). A Hybrid Approach of Boosting Against Noisy Data. In: Zighed, D.A., Tsumoto, S., Ras, Z.W., Hacid, H. (eds) Mining Complex Data. Studies in Computational Intelligence, vol 165. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-540-88067-7_3
Download citation
DOI: https://doi.org/10.1007/978-3-540-88067-7_3
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-88066-0
Online ISBN: 978-3-540-88067-7
eBook Packages: EngineeringEngineering (R0)