Abstract
Supervised learning is characterized by a broad spectrum of learning problems, often involving structured types of prediction, including classification, ranking-based predictions (label and instance ranking), and (ordinal) regression in its various forms. All these different learning problems are typically addressed by specific algorithmic solutions.
In this chapter, we propose ageneral preference learning model (GPLM), which gives an easy way to translate any supervised learning problem and the associated cost functions into sets of preferences to learn from. A large margin principled approach to solve this problem is also proposed.
Examples of how the proposed framework has been effectively used by us to address non-standard real-world applications are reported showing the flexibility and effectiveness of the approach.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Notes
- 1.
Note that this task and the two that follow are conceptually different from the task to decide about the membership of an instance. Here, supervision only gives qualitative information about the fact that some classes are more relevant than others.
- 2.
Note that the same employee can play the role of negative example in several events. Moreover, it might also be a positive example.
- 3.
References
F.Aiolli, Large margin multiclass learning: models and algorithms. Ph.D. thesis, Department of Computer Science, University of Pisa, 2004. http://www.di.unipi.it/phd/tesi/tesi_2004/PhDthesisAiolli.ps.gz
F.Aiolli, A preference model for structured supervised learning tasks, in Proceedings of the IEEE International Conference on Data Mining (ICDM) (2005), pp. 557–560
F.Aiolli, R.Cardin, F.Sebastiani, A.Sperduti, Preferential text classification: Learning algorithms and evaluation measures. Inf. Retr. 12(5), 559–580 (2009)
F.Aiolli, M.De Filippo, A.Sperduti, Application of the preference learning model to a human resources selection task, in Proceedings of the IEEE Symposium on Computational Intelligence and Data Mining (CIDM) (Amsterdam, NL, 2009), pp. 203–210
F.Aiolli, A.Sperduti, Learning preferences for multiclass problems, in Advances in Neural Information Processing Systems (MIT, Cambridge, MA, 2005) pp. 17–24
C.J.C. Burges, T.Shaked, E.Renshaw, A.Lazier, M.Deeds, N.Hamilton, G.N. Hullender, Learning to rank using gradient descent, in Proceedings of the International Conference on Machine Learning (ICML) (2005), pp. 89–96
W.Chu, S.Sathiya Keerthi, Support vector ordinal regression. Neural Comput. 19(3), 792–815 (2007)
W.W. Cohen, R.E. Schapire, Y.Singer, Learning to order things. J. Artif. Intell. Res. 10 243–270 (1999)
K.Crammer, Y.Singer, Pranking with ranking, in Advances in Neural Information Processing Systems (NIPS) (2002), pp. 641–647
K.Crammer, Y.Singer, A family of additive online algorithms for category ranking. J. Mach. Learn. Res. 3, 1025–1058 (2003)
O.Dekel, C.D. Manning, Y.Singer, Log-linear models for label ranking, in Advances in Neural Information Processing Systems (2003)
T.Evgeniou, M.Pontil, T.Poggio, Regularization networks and support vector machines. Adv. Comput. Math. 13, 1–50 (2000)
C.J. Fall, A.Törcsvári, K.Benzineb, G.Karetka, Automated categorization in the International Patent Classification. SIGIR Forum 37(1), 10–25 (2003)
Y.Freund, R.D. Iyer, R.E. Schapire, Y.Singer, An efficient boosting algorithm for combining preferences. J. Mach. Learn. Res. 4, 933–969 (2003)
T.T. Friess, N.Cristianini, C.Campbell, The kernel adatron algorithm: a fast and simple learning procedure for support vector machines, in Proceedings of International Conference of Machine Learning (ICML) (1998), pp. 188–196
T.T. Friess, N.Cristianini, C.Campbell, Subset ranking using regression, in Proceedings of the International Conference on Learning Theory (COLT) (Springer Berlin/Heidelberg, 2006), pp. 605–619
J.Fürnkranz, E.Hüllermeier, E.Mencía, K.Brinker, Multilabel classification via calibrated label ranking. Mach. Learn. 73(2), 133–153 (2008)
S.Har-Peled, D.Roth, D.Zimak, Constraint classification for multiclass classification and ranking, in Advances in Neural Information Processing Systems (2002), pp. 785–792
R.Herbrich, T.Graepel, P.Bollmann-Sdorra, K.Ober-mayer, Learning a preference relation for information retrieval, in Proceedings of the AAAI Workshop Text Categorization and Machine Learning (1998)
R.Herbrich, T.Graepel, K.Obermayer, Large margin rank boundaries for ordinal regression, in Advances in Large Margin Classifiers (MIT, 2000), pp. 115–132
E.Hüllermeier, J.Fürnkranz, W.Cheng, K.Brinker, Label ranking by learning pairwise preferences. Artif. Intell. 172(16–17), 1897–1916 (2008)
T.Joachims, Making large-scale svm learning practical, in Advances in Kernel Methods - Support Vector Learning ed. by B.Schlkopf, C.Burges, A.Smola (MIT, 1999)
T.Joachims, Optimizing search engines using clickthrough data, in Proceedings of the Conference on Knowledge Discovery and Data Mining (KDD) (2002) pp. 133–142
Q.Le, A.Smola, Direct optimization of ranking measures. Technical report, NICTA, Canberra, Australia, 2007
P.Li, C.Burges, Q.Wu, Mcrank: Learning to rank using multiple classification and gradient boosting, in Advances in Neural Information Processing Systems (NIPS) (MIT, 2008), pp.897–904
P.McCullagh, J.A. Nelder, Generalized Linear Models (Chapman & Hall, 1983)
R.Nallapati, Discriminative models for information retrieval, in Proceedings of the Conference on Research and Development in Information Retrieval (SIGIR) (ACM, 2004), pp. 64–71
J.C. Platt, N.Cristianini, J. Shawe-Taylor, Large margin DAGs for multiclass classification, in Advances in Neural Information Processing Systems (NIPS) (1999), pp. 547–533
A.Shashua, A.Levin, Ranking with large margin principle: Two approaches, in Advances in Neural Information Processing Systems (NIPS) (2002), pp. 937–944
I.Tsochantaridis, T.Hofmann, T.Joachims, Y.Altun, Support vector machine learning for interdependent and structured output spaces, in Proceedings of the International Conference on Machine learning (ICML) (2004), pp. 1453–1484
H.Wu, H.Lu, S.Ma, A practical svm-based algorithm for ordinal regression in image retrieval, in Proceedings of the ACM international conference on Multimedia (2003), pp. 612–621
F.Xia, T.Liu, J.Wang, W.Zhang, H.Li, Listwise approach to learning to rank: theory and algorithm, in Proceedings of the International Conference on Machine Learning (ICML) (2008), pp. 1192–1199
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2010 Springer-Verlag Berlin Heidelberg
About this chapter
Cite this chapter
Aiolli, F., Sperduti, A. (2010). A Preference Optimization Based Unifying Framework for Supervised Learning Problems. In: Fürnkranz, J., Hüllermeier, E. (eds) Preference Learning. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-14125-6_2
Download citation
DOI: https://doi.org/10.1007/978-3-642-14125-6_2
Published:
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-14124-9
Online ISBN: 978-3-642-14125-6
eBook Packages: Computer ScienceComputer Science (R0)