Abstract
The performance of ensemble, including AdaBoost, is determined by accuracy and generalization ability. However, the currently available AdaBoost’s weak learners mostly show high accuracy but rather low generalization ability. In this paper, we introduce three requirements that weak learners must satisfy in order to improve generalization ability of AdaBoost. Then, we propose w-delta learning rule based neural network(NN) as a weak learner that satisfies those requirements. Through experiments, we show that the proposed method improves generalization ability while maintaining the high accuracy.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsPreview
Unable to display preview. Download preview PDF.
References
Freund, Y., Schapire, R.E.: A decision-theoretic generalization of online learning and an application to boosting. Journal of Computer and System Sciences 55, 119–139 (1997)
Levin, E., Tishby, N., Solla, S.A.: A statistical approach to learning and generalization in layered neural networks. Proc. of IEEE 78(10), 1568–1574 (1990)
Hansen, L., Salamon, P.: Neural network ensembles. IEEE Trans. PAMI 12(10), 993–1001 (1990)
Wickramaratna, J., Holden, S., Buxton, B.: Performance degradation in boosting. In: Kittler, J., Roli, F. (eds.) MCS 2001. LNCS, vol. 2096, pp. 11–21. Springer, Heidelberg (2001)
Duda, R.O., Hart, P.E., Stork, D.G.: Pattern Classification, 2nd ed. Wiley (2000)
Quinlan, J.R.: C4.5 programs for machine learning. Morgan Kaufmann Publishers (1993)
Vapnik, V.: Statistical Learning Theory. John Wiley (1998)
Evgeniou, T., Perez-Breva, L., Pontil, M., Poggio, T.: Bound on the generalization performance of kernel machine ensembles. In: Proc. of ICML, pp. 271–278 (2000)
Bishop, C.M.: Neural Networks for Pattern Recognition. Clarendon Press (1995)
Hagan, M.T., Demuth, H.B., Beale, M.: Neural Network Design. PWS (1996)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2014 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Won, J., Kim, M. (2014). A Neural Network Based Simple Weak Learner for Improving Generalization Ability for AdaBoost. In: Park, J., Adeli, H., Park, N., Woungang, I. (eds) Mobile, Ubiquitous, and Intelligent Computing. Lecture Notes in Electrical Engineering, vol 274. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-40675-1_70
Download citation
DOI: https://doi.org/10.1007/978-3-642-40675-1_70
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-40674-4
Online ISBN: 978-3-642-40675-1
eBook Packages: EngineeringEngineering (R0)