Abstract
This paper proposes a nonlinear ordinal logistic regression method based on the hybridization of a linear model and radial basis function (RBF) neural network models for ordinal regression. The process for obtaining the coefficients is carried out in several steps. In the first step we use an evolutionary algorithm to determine the structure of the RBF neural network model, in a second step we transform the initial feature space (covariate space) adding the nonlinear transformations of the input variables given by the RBFs of the best individual in the final generation of the evolutionary algorithm. Finally, we apply an ordinal logistic regression in the new feature space. This methodology is tested using 8 benchmark problems from the UCI repository. The hybrid model outperforms both the linear and the nonlinear part obtaining a good compromise between them and better results in terms of accuracy and ordinal classification error.
This work has been partially subsidised by the TIN2011-22794 project of the Spanish Ministry of Economy and Competitiveness (MINECO), FEDER funds and the P2011-TIC-7508 project of the “Junta de Andalucía” (Spain).
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Agresti, A.: Analysis of ordinal categorical data, vol. 656. John Wiley & Sons (2010)
Angeline, P.J., Sauders, G.M., Pollack, J.B.: An evolutionary algorithm that constructs recurrent neural networks. IEEE Transactions on Neural Networks 5(1), 54–65 (1994)
Asuncion, A., Newman, D.: UCI machine learning repository (2007). http://www.ics.uci.edu/mlearn/MLRepository.html
Baccianella, S., Esuli, A., Sebastiani, F.: Evaluation measures for ordinal regression. In: Proceedings of the Ninth International Conference on Intelligent Systems Design and Applications (ISDA 2009), Pisa, Italy, December 2009
Buchtala, O., Klimek, M., Sick, B.: Evolutionary optimization of radial basis function classifiers for data mining applications. IEEE Transactions on Neural Networks Part B 35(5), 928–947 (2005)
Chu, W., Ghahramani, Z.: Gaussian processes for ordinal regression. Journal of Machine Learning Research 6, 1019–1041 (2005)
Dorado-Moreno, M., Gutiérrez, P.A., Hervás-Martínez, C.: Ordinal Classification Using Hybrid Artificial Neural Networks with Projection and Kernel Basis Functions. In: Corchado, E., Snášel, V., Abraham, A., Woźniak, M., Graña, M., Cho, S.-B. (eds.) HAIS 2012, Part II. LNCS, vol. 7209, pp. 319–330. Springer, Heidelberg (2012)
Gutiérrez, P.A., Hervás-Martínez, C., Martínez-Estudillo, F.J.: Logistic regression by means of evolutionary radial basis function neural networks. IEEE Transactions on Neural Networks 22(2), 246–263 (2011)
Hastie, T., Tibshirani, R.: Generalized additive models. Chapman & Hall, London (1990)
Igel, C., Hsken, M.: Empirical evaluation of the improved rprop learning algorithms. Neurocomputing 50(6), 105–123 (2003)
Lee, S.H., Hou, C.L.: An art-based construction of RBF networks. IEEE Transactions on Neural Networks 13(6), 1308–1321 (2002)
Lippmann, R.P.: Pattern classification using neural networks. IEEE Transactions on Neural Networks 27, 47–64 (1989)
Maniezzo, V.: Genetic evolution of the topology and weight distribution of neural networks. IEEE Transactions on Neural Networks 5, 39–53 (1994)
McCullagh, P.: Regression models for ordinal data (with discussion). Journal of the Royal Statistical Society 42(2), 109–142 (1980)
Odri, S.V., Petrovacki, D.P., Krstonosic, G.A.: Evolutional development of a multilevel neural network. Neural Networks 6, 583–595 (1993)
PASCAL: Pascal (pattern analysis, statistical modelling and computational learning) machine learning benchmarks repository (2011). http://mldata.org/
van Rooij, A.J.F., Jain, L.C., Johnson, R.P.: Neural Networks Training Using Genetic Algorithms, Series in Machine Perception and Artificial Intelligence, vol. 26. World Scientific, Singapore (1996)
Yao, X., Liu, Y.: A new evolutionary system for evolving artificial neural networks. IEEE Transactions on Neural Networks 8, 694–713 (1997)
Yao, X.: Evolving artificial neural networks. Proceedings of the IEEE 87(9) (1999)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2015 Springer International Publishing Switzerland
About this paper
Cite this paper
Dorado-Moreno, M., Gutiérrez, P.A., Sánchez-Monedero, J., Hervás-Martínez, C. (2015). Nonlinear Ordinal Logistic Regression Using Covariates Obtained by Radial Basis Function Neural Networks Models. In: Rojas, I., Joya, G., Catala, A. (eds) Advances in Computational Intelligence. IWANN 2015. Lecture Notes in Computer Science(), vol 9095. Springer, Cham. https://doi.org/10.1007/978-3-319-19222-2_7
Download citation
DOI: https://doi.org/10.1007/978-3-319-19222-2_7
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-19221-5
Online ISBN: 978-3-319-19222-2
eBook Packages: Computer ScienceComputer Science (R0)