Abstract
We investigate the problem of data-driven ordinal regression—the problem of learning to rank order new data items based on information inherent in existing data items. Ordinal regression shares common features with multi-category classification and metric regression. However, it requires new, tailor-made methodologies to reduce prediction error. The approach has application in various domains, including information retrieval, collaborative filtering and social sciences. We propose a new distribution-independent methodology for ordinal regression based on pairwise preferences employing information about strength of dependency between two data instances, which we refer to as continuous preferences. Our hypothesis is that additional information about strength of preference as well as its direction can improve algorithmic performance. We also provide a novel technique for deriving ordinal regression labels from pairwise information. Experimental results on real-world ordinal and metric regression data sets confirm usefulness of the methodology compared with other state-of-the-art approaches.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
Notes
Weka configuration for SMOreg: weka.classifiers.functions.SMOreg -C 1.0 -N 0 -I “weka.classifiers.functions.supportVector.RegSMOImproved -L 0.0010 -W 1 -P 1.0E-12 -T 0.0010 -V” -K “weka.classifiers.functions.supportVector.RBFKernel -C 250007 -G 0.01” -t.
Data sets available at: http://www.cs.waikato.ac.nz/ml/weka and http://www.liaad.up.pt/ltorgo/Regression/DataSets.html.
Authors’ descriptions from the original data sets.
References
Burges C, Shaked T, Renshaw E, Lazier A, Deeds M, Hamilton N, Hullender G (2005) Learning to rank using gradient descent. In: ICML’05: Proceedings of the 22nd international conference on machine learning. ACM, New York, NY, USA, pp 89–96
Chu W, Keerthi SS (2007) Support vector ordinal regression. Neural Comput 19(3):792–815
Cohen WW, Schapire RE, Singer Y (1998) Learning to order things. In: NIPS’97: Proceedings of the 1997 conference on advances in neural information processing systems 10. MIT Press, Cambridge, MA, USA, pp 451–457
Crammer K, Singer Y (2001) Pranking with ranking. In: NIPS, pp 641–647
Frank E, Hall M (2001) A simple approach to ordinal classification. In: EMCL’01: Proceedings of the 12th European conference on machine learning. Springer, London, UK, pp 145–156
Freund Y, Iyer R, Schapire RE, Singer Y (2003) An efficient boosting algorithm for combining preferences. J Mach Learn Res 4:933–969
Herbrich R, Graepel T, Obermayer K (1999) Support vector learning for ordinal regression. In: In international conference on artificial neural networks, pp 97–102
Herbrich R, Graepel T, Obermayer K (2000) Large margin rank boundaries for ordinal regression. MIT Press, Cambridge, MA, pp 115–132. http://citeseer.ist.psu.edu/contextsummary/1891774/0
Huhn J, Hullermeier E (2009) Is an ordinal class structure useful in classifier learning?. Int J Data Min Model Manag 1(1):45–67
Jiao T, Peng J, Terlaky T (2009) A confidence voting process for ranking problems based on support vector machines. Ann Oper Res 166(1):23–38. http://dx.doi.org/10.1007/s10479-008-0410-6
Rennie JD, Srebro N (2005) Loss functions for preference levels: regression with discrete ordered labels. In: Proceedings of the IJCAI multidisciplinary workshop on advances in preference handling, pp 180–186
Shashua A, Levin A (2002) Ranking with large margin principle: two approaches. In: NIPS, pp 937–944
Shevade SK, Chu W (2006) Minimum enclosing spheres formulations for support vector ordinal regression. In: ICDM’06: Proceedings of the sixth international conference on data mining. IEEE Computer Society, Washington, DC, USA, pp 1054–1058
Torra V, Domingo-Ferrer J, Mateo-Sanz JM, Ng M (2006) Regression for ordinal variables without underlying continuous variables. Inf Sci 176(4):465–474
Waegeman W, Baets BD, Boullart L (2008) Roc analysis in ordinal regression learning. Pattern Recogn Lett 29(1):1–9
Witten IH, Frank E (2005) Data Mining: practical machine learning tools and techniques. Morgan Kaufmann, San Fransisco
Zhao B, Wang F, Zhang C (2009) Block-quantized support vector ordinal regression. IEEE Trans Neural Netw 20(5):882–90
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Dobrska, M., Wang, H. & Blackburn, W. Ordinal regression with continuous pairwise preferences. Int. J. Mach. Learn. & Cyber. 3, 59–70 (2012). https://doi.org/10.1007/s13042-011-0036-x
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s13042-011-0036-x