Abstract
Least Squares Support Vector Regression (LS-SVR) is a powerful kernel-based learning tool for regression problems. However, since it is based on the ordinary least squares (OLS) approach for parameter estimation, the standard LS-SVR model is very sensitive to outliers. Robust variants of the LS-SVR model, such as the WLS-SVR and IRLS-SVR models, have been developed aiming at adding robustness to the parameter estimation process, but they still rely on OLS solutions. In this paper we propose a totally different approach to robustify the LS-SVR. Unlike previous models, we maintain the original LS-SVR loss function, while the solution of the resulting linear system for parameter estimation is obtained by means of the Recursive Least M-estimate (RLM) algorithm. We evaluate the proposed approach in nonlinear system identification tasks, using artificial and real-world datasets contaminated with outliers. The obtained results for infinite-steps-ahead prediction shows that proposed model consistently outperforms the WLS-SVR and IRLS-SVR models for all studied scenarios.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Notes
- 1.
IQR stands for InterQuantile Range, which is the difference between the 75th percentile and 25th percentile.
- 2.
References
Suykens, J.A.K., De Brabanter, J., Lukas, L., Vandewalle, J.: Weighted least squares support vector machines: robustness and sparse approximation. Neurocomputing 48(1), 85–105 (2002)
De Brabanter, K., Pelckmans, K., De Brabanter, J., Debruyne, M., Suykens, J.A.K., Hubert, M., De Moor, B.: Robustness of Kernel based regression: a comparison of iterative weighting schemes. In: Alippi, C., Polycarpou, M., Panayiotou, C., Ellinas, G. (eds.) ICANN 2009, Part I. LNCS, vol. 5768, pp. 100–110. Springer, Heidelberg (2009)
Zou, Y., Chan, S., Ng, T.: A recursive least \(m\)-estimate (RLM) adaptive filter for robust filtering in impulse noise. IEEE Signal Proccess. Lett. 7(11), 324–326 (2000)
Saunders, C., Gammerman, A., Vovk, V.: Ridge regression learning algorithm in dual variables. In:Proceedings of the 15th International Conference on Machine Learning (ICML 1998), pp. 515–521. Morgan Kaufmann (1998)
Suykens, J.A.K., Van Gestel, T., De Brabanter, J., De Moor, B., Vandewalle, J.: Least Squares Support Vector Machines. World Scientific, Singapore (2002)
Van Gestel, T., Suykens, J.A., Baestaens, D.E., Lambrechts, A., Lanckriet, G., Vandaele, B., De Moor, B., Vandewalle, J.: Financial time series prediction using least squares support vector machines within the evidence framework. IEEE Trans. Neural Netw. 12(4), 809–821 (2001)
Khalil, H.M., El-Bardini, M.: Implementation of speed controller for rotary hydraulic motor based on LS-SVM. Expert Syst. Appl. 38(11), 14249–14256 (2011)
Falck, T., Suykens, J.A., De Moor, B.: Robustness analysis for least squares kernel based regression: an optimization approach. In: Proceedings of the 48th IEEE Conference on Decision and Control (CDC 2009), pp. 6774–6779 (2009)
Huber, P.J., et al.: Robust estimation of a location parameter. Ann. Math. Stat. 35(1), 73–101 (1964)
Kocijan, J., Girard, A., Banko, B., Murray-Smith, R.: Dynamic systems identification with gaussian processes. Math. Comput. Model. Dyn. Syst. 11(4), 411–424 (2005)
Narendra, K.S., Parthasarathy, K.: Identification and control of dynamical systems using neural networks. IEEE Trans. Neural Netw. 1(1), 4–27 (1990)
Acknowledgments
The authors thank the financial support of IFCE, NUTEC and CNPq (grant no. 309841/2012-7).
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2015 Springer International Publishing Switzerland
About this paper
Cite this paper
Santos, J.D.A., Barreto, G.A. (2015). A Novel Recursive Solution to LS-SVR for Robust Identification of Dynamical Systems. In: Jackowski, K., Burduk, R., Walkowiak, K., Wozniak, M., Yin, H. (eds) Intelligent Data Engineering and Automated Learning – IDEAL 2015. IDEAL 2015. Lecture Notes in Computer Science(), vol 9375. Springer, Cham. https://doi.org/10.1007/978-3-319-24834-9_23
Download citation
DOI: https://doi.org/10.1007/978-3-319-24834-9_23
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-24833-2
Online ISBN: 978-3-319-24834-9
eBook Packages: Computer ScienceComputer Science (R0)