Abstract
Metalearning is a methodology aiming at recommending the most suitable algorithm (or method) from several alternatives for a particular dataset. Its classification rule is learned over an available training database of datasets. It gradually penetrates to various applications in computer science and has also the potential to recommend the most suitable statistical estimator for a given dataset. We consider the nonlinear regression model. While there are some robust alternatives to the traditional (and very non-robust) nonlinear least squares available, it is not theoretically known which estimator performs the best for a particular dataset. In this work, we perform a metalearning study performed over 721 datasets predicting the best nonlinear regression estimator for an independent dataset. The estimators considered here include standard nonlinear least squares as well as its robust alternatives with a high breakdown point. On the whole, the presented study brings new arguments in favor of the nonlinear least weighted squares estimator, which is based on the idea to assign implicit weights to individual observations based on outlyingness of their residuals.
The work is supported by the projects 19-05704S (J. Kalina) and 18-23827S (P. Vidnerová) of the Czech Science Foundation.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Arel-Bundock, V.: Website of datasets. https://vincentarelbundock.github.io/ Rdatasets/datasets.html. Accessed 10 Sep 2019
Baldauf, M., Silva, J.M.C.S.: On the use of robust regression in econometrics. Econ. Let. 114, 124–127 (2012)
Brazdil, P., Giraud-Carrier, C., Soares, C., Vilalta, E.: Metalearning: Applications to Data Mining. Springer, Berlin (2009)
Čížek, P.: Semiparametrically weighted robust estimation of regression models. Comput. Stat. Data An. 55, 774–788 (2011)
Guo, Y., Hastie, T., Tibshirani, R.: Regularized discriminant analysis and its application in microarrays. Biostatistics 8, 86–100 (2007)
Hastie, T., Tibshirani, R., Friedman, J.: The Elements of Statistical Learning. Springer, New York (2001)
Jurečková, J., Picek, J., Schindler, M.: Robust Statistical Methods with R, 2nd edn. CRC Press, Boca Raton (2019)
Jurečková, J., Sen, P.K., Picek, J.: Methodology in Robust and Nonparametric Statistics. CRC Press, Boca Raton (2013)
Kalina, J.: Highly robust methods in data mining. Serb. J. Manag. 8, 9–24 (2013)
Kalina, J.: A robust pre-processing of BeadChip microarray images. Biocybern. Biomed. Eng. 38, 556–563 (2018)
Kalina, J.: Metalearning for robust regression. https://github.com/jankalinaUI/ Metalearning-for-robust-regression. Accessed 20 Oct 2019
Kalina, J., Peštová, B.: Robust regression estimators: a comparison of prediction performance. In: Proceedings of the 35th International Conference Mathematical Methods in Economics MME 2017, pp. 307–312. University of Hradec Králové, Hradec Králové (2017)
Kalina, J., Schlenker, A.: A robust supervised variable selection for noisy high-dimensional data. BioMed Res. Int. 2015, 1–10 (2015). Article 320385
Koenker, R.: Quantile Regression. Cabridge University Press, Cambridge (2005)
Koenker, R., Park, B.J.: An interior point algorithm for nonlinear quantile regression. J. Econometrics 71, 265–283 (1996)
Mount, D.M., Netanyahu, N.S., Piatko, C.D., Silverman, R., Wu, A.Y.: On the least trimmed squares estimator. Algorithmica 69, 148–183 (2014)
Riazoshams, H., Midi, H.B., Sharipov, O.S.: The performance of robust two-stage estimator in nonlinear regression with autocorrelated error. Commun. Stat. Simulat. 39, 1251–1268 (2010)
Rousseeuw, P.J., van Driessen, K.: Computing LTS regression for large datasets. Data Min. Knowl. Disc. 12, 29–45 (2006)
Seber, G.A.F., Wild, C.J.: Nonlinear Regression. Wiley, New York (2003)
Smith-Miles, K., Baatar, D., Wreford, B., Lewis, R.: Towards objective measures of algorithm performance across instance space. Comput. Oper. Res. 45, 12–24 (2014)
Stromberg, A.J., Ruppert, D.: Breakdown in nonlinear regression. J. Am. Stat. Assoc. 87, 991–997 (1992)
Víšek, J.Á.: Robust error-term-scale estimate. In: Antoch, J., Hušková, M., Sen, P.K. (eds.) Nonparametrics and Robustness in Modern Statistical Inference and Time Series Analysis: A Festschrift in Honor of Professor Jana Jurečková. IMS Collections, 7, pp. 254–267 (2010)
Víšek, J.Á.: Consistency of the least weighted squares under heteroscedasticity. Kybernetika 47, 179–206 (2011)
Acknowledgment
The authors would like to thank Aleš Neoral for help.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2020 The Editor(s) (if applicable) and The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Kalina, J., Vidnerová, P. (2020). A Metalearning Study for Robust Nonlinear Regression. In: Iliadis, L., Angelov, P., Jayne, C., Pimenidis, E. (eds) Proceedings of the 21st EANN (Engineering Applications of Neural Networks) 2020 Conference. EANN 2020. Proceedings of the International Neural Networks Society, vol 2. Springer, Cham. https://doi.org/10.1007/978-3-030-48791-1_39
Download citation
DOI: https://doi.org/10.1007/978-3-030-48791-1_39
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-48790-4
Online ISBN: 978-3-030-48791-1
eBook Packages: Computer ScienceComputer Science (R0)