Abstract
The classification of patterns into naturally ordered labels is referred to as ordinal regression. This paper explores the notion of kernel trick and empirical feature space in order to reformulate the most widely used linear ordinal classification algorithm (the Proportional Odds Model or POM) to perform nonlinear decision regions. The proposed method seems to be competitive with other state-of-the-art algorithms and significantly improves the original POM algorithm when using 8 ordinal datasets. Specifically, the capability of the methodology to handle nonlinear decision regions has been proven by the use of a non-linearly separable toy dataset.
This work has been partially subsidized by the TIN2011-22794 project of the Spanish Ministerial Commission of Science and Technology (MICYT), FEDER funds and the P2011-TIC-7508 project of the “Junta de Andalucía” (Spain).
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Agresti, A.: Categorical Data Analysis, 2nd edn. Wiley Series in Probability and Statistics. Wiley-Interscience (2002)
McCullagh, P.: Regression models for ordinal data. Journal of the Royal Statistical Society 42(2), 109–142 (1980)
Schölkopf, B., Mika, S., Burges, C.J.C., Knirsch, P., Müller, K.R., Rätsch, G., Smola, A.J.: Input space versus feature space in kernel-based methods. IEEE Transactions on Neural Networks 10, 1000–1017 (1999)
Xiong, H., Swamy, M.N.S., Ahmad, M.O.: Optimizing the kernel in the empirical feature space. IEEE Transactions on Neural Networks 16(2), 460–474 (2005)
Abe, S., Onishi, K.: Sparse least squares support vector regressors trained in the reduced empirical feature space. In: de Sá, J.M., Alexandre, L.A., Duch, W., Mandic, D.P. (eds.) ICANN 2007. LNCS, vol. 4669, pp. 527–536. Springer, Heidelberg (2007)
Xiong, H.: A unified framework for kernelization: The empirical kernel feature space. In: Chinese Conference on Pattern Recognition (CCPR), pp. 1–5 (November 2009)
Verwaeren, J., Waegeman, W., De Baets, B.: Learning partial ordinal class memberships with kernel-based proportional odds models. Comput. Stat. Data Anal. 56(4), 928–942 (2012)
Schölkopf, B., Smola, A., Müller, K.R.: Nonlinear component analysis as a kernel eigenvalue problem. Neural Computation 10(5), 460–474 (1998)
Asuncion, A., Newman, D.: UCI machine learning repository (2007)
Chu, W., Ghahramani, Z.: Gaussian processes for ordinal regression. Journal of Machine Learning Research 6, 1019–1041 (2005)
Sun, B.Y., Li, J., Wu, D.D., Zhang, X.M., Li, W.B.: Kernel discriminant learning for ordinal regression. IEEE Transactions on Knowledge and Data Engineering 22, 906–910 (2010)
Chu, W., Keerthi, S.S.: Support vector ordinal regression. Neural Computation 19(3), 792–815 (2007)
Gutiérrez, P.A., Pérez-Ortiz, M., Fernández-Navarro, F., Sánchez-Monedero, J., Hervás-Martínez, C.: An Experimental Study of Different Ordinal Regression Methods and Measures. In: Corchado, E., Snášel, V., Abraham, A., Woźniak, M., Graña, M., Cho, S.-B. (eds.) HAIS 2012, Part II. LNCS, vol. 7209, pp. 296–307. Springer, Heidelberg (2012)
Baccianella, S., Esuli, A., Sebastiani, F.: Evaluation measures for ordinal regression. In: Proceedings of the Ninth International Conference on Intelligent Systems Design and Applications (ISDA 2009), Pisa, Italy (2009)
Demsar, J.: Statistical comparisons of classifiers over multiple data sets. Journal of Machine Learning Research 7, 1–30 (2006)
Drineas, P., Mahoney, M.W.: On the nyström method for approximating a gram matrix for improved kernel-based learning. J. Mach. Learn. Res. 6 (2005)
Braun, M.L., Buhmann, J.M., Müller, K.R.: On relevant dimensions in kernel feature spaces. J. Mach. Learn. Res. 9, 1875–(1908)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2013 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Pérez-Ortiz, M., Gutiérrez, P.A., Cruz-Ramírez, M., Sánchez-Monedero, J., Hervás-Martínez, C. (2013). Kernelizing the Proportional Odds Model through the Empirical Kernel Mapping. In: Rojas, I., Joya, G., Gabestany, J. (eds) Advances in Computational Intelligence. IWANN 2013. Lecture Notes in Computer Science, vol 7902. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-38679-4_26
Download citation
DOI: https://doi.org/10.1007/978-3-642-38679-4_26
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-38678-7
Online ISBN: 978-3-642-38679-4
eBook Packages: Computer ScienceComputer Science (R0)