Abstract
Most existing ordinal regression methods are adapted from traditional supervised learning algorithms (e.g., support vector machines and neural networks) which have shown to work well mostly on dense data. However, the use of existing ordinal regression methods on sparse data has received less scrutiny. This paper proposes to address the sparsity issue arose in many real-world ordinal regression applications by leveraging the feature interaction modeling techniques. Following the popular threshold methodology in ordinal regression studies, we extend Factorization Machines, an effective solution to modeling pairwise feature interactions in sparse feature space, to ordinal regression. The proposed model, namely Factorization Machines for Ordinal Regression (FMOR), combines the ability of threshold methodology in predicting targets of ordinal scale with the advantages of factorization models in handling high-dimensional sparse data. Through extensive experimental studies, we show that the proposed FMOR is both effective and efficient against state-of-the-art baselines.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Alp, A.: Structural shifts in credit rating standards. J. Financ. 68(6), 2435–2470 (2013)
Beckham, C., Pal, C.: Unimodal probability distributions for deep ordinal classification, pp. 411–419 (2017)
Chu, W., Keerthi, S.S.: New approaches to support vector ordinal regression. In: Proceedings of the 22nd International Conference on Machine Learning, pp. 145–152 (2005)
Goh, C.K., Liu, Y., Kong, A.W.: A constrained deep neural network for ordinal regression. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 831–839 (2018)
Gutierrez, P.A., Perez-Ortiz, M., Sanchez-Monedero, J., Fernandez-Navarro, F., Hervas-Martinez, C.: Ordinal regression methods: survey and experimental study. IEEE Trans. Knowl. Data Eng. 28(1), 127–146 (2016)
Gutiérrez, P.A., Tiňo, P., Hervás-Martínez, C.: Ordinal regression neural networks based on concentric hyperspheres. Neural Netw. 59, 51–60 (2014)
Huang, X., Zhang, L., Wang, B., Zhang, Z., Li, F.: Feature weight estimation based on dynamic representation and neighbor sparse reconstruction. Pattern Recogn. 81, 388–403 (2018)
Kingma, D.P., Ba, J.: Adam: a method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014)
Koren, Y., Sill, J.: Collaborative filtering on ordinal user feedback. In: Proceedings of the 23th International Joint Conference on Artificial Intelligence (2013)
Li, L., Lin, H.T.: Ordinal regression by extended binary classification. In: Advances in Neural Information Processing Systems, pp. 865–872 (2007)
Lin, H.T., Li, L.: Large-margin thresholded ensembles for ordinal regression: theory and practice. In: Proceedings of the International Conference on Algorithmic Learning Theory, pp. 319–333 (2006)
Lin, H.T., Li, L.: Reduction from cost-sensitive ordinal ranking to weighted binary classification. Neural Comput. 24(5), 1329–1367 (2012)
Liu, X., Zou, Y., Song, Y., Yang, C., You, J., Kumar, B.V.K.V.: Ordinal regression with neuron stick-breaking for medical diagnosis. In: Leal-Taixé, L., Roth, S. (eds.) ECCV 2018. LNCS, vol. 11134, pp. 335–344. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-11024-6_23
Liu, Y., Kong, A.W.K., Goh, C.K.: Deep ordinal regression based on data relationship for small datasets. In: Proceedings of the 26th International Joint Conferences on Artificial Intelligence, pp. 2372–2378 (2017)
McCullagh, P.: Regression models for ordinal data. J. Roy. Stat. Soc.: Ser. B (Methodol.) 42(2), 109–127 (1980)
Ni, W., Liu, T., Zeng, Q., Zhang, X., Duan, H., Xie, N.: Robust factorization machines for credit default prediction. In: Geng, X., Kang, B.-H. (eds.) PRICAI 2018. LNCS (LNAI), vol. 11012, pp. 941–953. Springer, Cham (2018). https://doi.org/10.1007/978-3-319-97304-3_72
Pan, Z., Chen, E., Liu, Q., Xu, T., Ma, H., Lin, H.: Sparse factorization machines for click-through rate prediction. In: Proceedings of the IEEE 16th International Conference on Data Mining, pp. 400–409 (2016)
Qiang, R., Liang, F., Yang, J.: Exploiting ranking factorization machines for microblog retrieval. In: Proceedings of the 22nd ACM International Conference on Conference on Information & Knowledge Management, pp. 1783–1788 (2013)
Rendle, S.: Factorization machines with libFM. ACM Trans. Intell. Syst. Technol. 3(3), 57 (2012)
Shashua, A., Levin, A.: Ranking with large margin principle: two approaches. In: Advances in Neural Information Processing Systems, pp. 961–968 (2003)
Sun, B.Y., Li, J., Wu, D.D., Zhang, X.M., Li, W.B.: Kernel discriminant learning for ordinal regression. IEEE Trans. Knowl. Data Eng. 22(6), 906–910 (2010)
Tian, Q., Zhang, W., Wang, L., Chen, S., Yin, H.: Robust ordinal regression induced by lp-centroid. Neurocomputing 313, 184–195 (2018)
Tran, T., Phung, D., Luo, W., Venkatesh, S.: Stabilized sparse ordinal regression for medical risk stratification. Knowl. Inf. Syst. 43(3), 555–582 (2015)
Wang, H., Shi, Y., Niu, L., Tian, Y.: Nonparallel support vector ordinal regression. IEEE Trans. Cybern. 47(10), 3306–3317 (2017)
Zhu, M., Aggarwal, C.C., Ma, S., Zhang, H., Huai, J.: Outlier detection in sparse data with factorization machines. In: Proceedings of the 2017 ACM Conference on Information and Knowledge Management, pp. 817–826 (2017)
Acknowledgement
This work is partially supported by Natural Science Foundation of China (61602278, 71704096 and 31671588), Sci. & Tech. Development Fund of Shandong Province of China (ZR2017MF027), the Humanities and Social Science Research Project of the Ministry of Education (18YJAZH017), the Taishan Scholar Climbing Program of Shandong Province, and SDUST Research Fund (2015TDJH102).
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2019 Springer Nature Switzerland AG
About this paper
Cite this paper
Ni, W., Liu, T., Zeng, Q. (2019). Sparse Ordinal Regression via Factorization Machines. In: Nayak, A., Sharma, A. (eds) PRICAI 2019: Trends in Artificial Intelligence. PRICAI 2019. Lecture Notes in Computer Science(), vol 11671. Springer, Cham. https://doi.org/10.1007/978-3-030-29911-8_13
Download citation
DOI: https://doi.org/10.1007/978-3-030-29911-8_13
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-29910-1
Online ISBN: 978-3-030-29911-8
eBook Packages: Computer ScienceComputer Science (R0)