Abstract
The paper considers applying a boosting strategy to optimise the generalisation bound obtained recently by Shawe-Taylor and Cristianini [7] in terms of the two norm of the slack variables. The formulation performs gradient descent over the quadratic loss function which is insensitive to points with a large margin. A novel feature of this algorithm is a principled adaptation of the size of the target margin. Experiments with text and UCI data shows that the new algorithm improves the accuracy of boosting. DMarginBoost generally achieves significant improvements over Adaboost.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Bennett, K. P., Demiriz, A., Shawe-Taylor, J.: Column generation aproaches to boosting. In International Conference on Machine Learning (ICML), 2000.
Friedman, J.: Greedy Function approximation: a gradient boosting machine. Technical report, Stanford University, 1999.
Mason, L., Baxter, J., Bartlett, P., Frean, M.: Boosting algorithms as gradient descent in function space. Technical report, RSISE., Australian National University, 1999.
Schapire, R., Singer, Y., Singhal, A.: Boosting and rocchio applied to text filtering. In Proceedings of the 21st Annual International SIGIR conference on Research and Development in Information Retrieval, SIGIR’98, 1998.
Cortes, C., and Vapnik, V.: Support-vector networks. Machine Learning, 20:273–297, 1995.
Ratsch, G., Onoda, T., and Muller, K.-R.: Regularizing adaboost. In Advances in Neural Information Processing Systems 11, pp. 564–570. MIT Press.
Shawe-Taylor, J., Cristianini, N.: Further results on the margin distribution. In Proceedings of the Conference on Computational Learning Theory, COLT 99, 1999.
Zhang, T.: Analysis of regularised linear functions for classiffication problem. Technical Report RC-21572, IBM, October 1999.
Joachima, T.: Text Categorization with support vector machines: Learning with many relevant fetures. In European Conference on Machine Learning (ECML), 1998.
Dumais, T. S., Platt, J., Heckerman, D., and Sahami, M.: Inductive learning algorithms and representations for text categorization. In Proceedings of ACM-CIKM98.
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2000 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Lodhi, H., Karakoulas, G., Shawe-Taylor, J. (2000). Boosting the Margin Distribution. In: Leung, K.S., Chan, LW., Meng, H. (eds) Intelligent Data Engineering and Automated Learning — IDEAL 2000. Data Mining, Financial Engineering, and Intelligent Agents. IDEAL 2000. Lecture Notes in Computer Science, vol 1983. Springer, Berlin, Heidelberg. https://doi.org/10.1007/3-540-44491-2_9
Download citation
DOI: https://doi.org/10.1007/3-540-44491-2_9
Published:
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-41450-6
Online ISBN: 978-3-540-44491-6
eBook Packages: Springer Book Archive