Abstract
In this paper, we propose an adaptive algorithm for bipartite ranking and prove its statistical performance in a stronger sense than the AUC criterion. Our procedure builds on and significantly improves the RankOver algorithm proposed in [1]. The algorithm outputs a piecewise constant scoring rule which is obtained by overlaying a finite collection of classifiers. Here, each of these classifiers is the empirical solution of a specific minimum-volume set (MV-set) estimation problem. The major novelty arises from the fact that the levels of the MV-sets to recover are chosen adaptively from the data to adjust to the variability of the target curve. The ROC curve of the estimated scoring rule may be interpreted as an adaptive spline approximant of the optimal ROC curve. Error bounds for the estimate of the optimal ROC curve in terms of the L ∞ -distance are also provided.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Clémençon, S., Vayatis, N.: Overlaying classifiers: a practical approach for optimal ranking. In: NIPS 2008: Proceedings of the 2008 conference on Advances in neural information processing systems, Vancouver, Canada, pp. 313–320 (2009)
Flach, P.: Tutorial on “the many faces of roc analysis in machine learning”. In: ICML 2004 (2004)
Freund, Y., Iyer, R.D., Schapire, R.E., Singer, Y.: An efficient boosting algorithm for combining preferences. Journal of Machine Learning Research 4, 933–969 (2003)
Agarwal, S., Graepel, T., Herbrich, R., Har-Peled, S., Roth, D.: Generalization bounds for the area under the ROC curve. Journal of Machine Learning Research 6, 393–425 (2005)
Clémençon, S., Lugosi, G., Vayatis, N.: Ranking and empirical risk minimization of U-statistics. The Annals of Statistics 36(2), 844–874 (2008)
Clémençon, S., Vayatis, N.: Overlaying classifiers: a practical approach to optimal scoring. To appear in Constructive Approximation (hal-00341246) (2009)
Scott, C., Nowak, R.: Learning minimum volume sets. Journal of Machine Learning Research 7, 665–704 (2006)
Clémençon, S., Vayatis, N.: Tree-structured ranking rules and approximation of the optimal ROC curve. Technical Report hal-00268068, HAL (2008)
Clémençon, S., Vayatis, N.: Tree-structured ranking rules and approximation of the optimal ROC curve. In: ALT 2008: Proceedings of the 2008 conference on Algorithmic Learning Theory (2008)
Devore, R., Lorentz, G.: Constructive Approximation. Springer, Heidelberg (1993)
Scott, C., Nowak, R.: A Neyman-Pearson approach to statistical learning. IEEE Transactions on Information Theory 51(11), 3806–3819 (2005)
Boucheron, S., Bousquet, O., Lugosi, G.: Theory of Classification: A Survey of Some Recent Advances. ESAIM: Probability and Statistics 9, 323–375 (2005)
Tsybakov, A.: Optimal aggregation of classifiers in statistical learning. Annals of Statistics 32(1), 135–166 (2004)
Devore, R.: A note on adaptive approximation. Approx. Theory Appl. 3, 74–78 (1987)
Bennett, C., Sharpley, R.: Interpolation of Operators. Academic Press, London (1988)
Devore, R.: Nonlinear approximation. Acta Numerica, 51–150 (1998)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2009 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Clémençon, S., Vayatis, N. (2009). Adaptive Estimation of the Optimal ROC Curve and a Bipartite Ranking Algorithm. In: Gavaldà, R., Lugosi, G., Zeugmann, T., Zilles, S. (eds) Algorithmic Learning Theory. ALT 2009. Lecture Notes in Computer Science(), vol 5809. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-04414-4_20
Download citation
DOI: https://doi.org/10.1007/978-3-642-04414-4_20
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-04413-7
Online ISBN: 978-3-642-04414-4
eBook Packages: Computer ScienceComputer Science (R0)