Abstract
To achieve robust estimation for noisy data set, a recursive outlier elimination-based least squares support vector machine (ROELS-SVM) algorithm is proposed in this paper. In this algorithm, statistical information from the error variables of least squares support vector machine is recursively learned and a criterion derived from robust linear regression is employed for outlier elimination. Besides, decremental learning technique is implemented in the recursive training–eliminating stage, which ensures that the outliers are eliminated with low computational cost. The proposed algorithm is compared with re-weighted least squares support vector machine on multiple data sets and the results demonstrate the remarkably robust performance of the ROELS-SVM.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Brabanter JD (2004) LS-SVM regression modelling and its applications. Ph.D. thesis. ftp://ftp.esat.kuleuven.ac.be/pub/SISTA//debrabant
Burges CJC (1998) A tutorial on support vector machines for pattern recognition. Data Min Knowl Discov 2(2):955–974
Cao LJ, Tay FEH (2003) Support vector machine with adaptive parameters in financial time series forecasting. IEEE Trans Neural Netw 14(6):1506–1518
Cawley GC, Talbot NLC (2004) Fast exact leave-one-out cross-validation of sparse least-squares support vector machines. Neural Netw 17:1467–1475
Chuang CC, Su SF, Jeng JT, Hsiao CC (2002) Robust support vector regression networks for function approximation with outliers. IEEE Trans Neural Netw 13(6):1322–1330
Cortes C (1993) Prediction of generalization ability in learning machines. Ph.D. thesis. http://homepage.mac.com/corinnacortes/
Demsar J (2006) Statistical comparisons of classifiers over multiple data sets. J Mach Learn Res 7:1–30
Evgeniou T, Pontil M, Poggio T (2000) Regularization networks and support vector machines. Adv Comput Math 13:1–50
Grubbs FE (1969) Procedures for detecting outlying observations in samples. Technometrics 11(1):1–21
Jiang JQ, Song CY, Wu CG, Maurizio M, Liang YC (2006) Support vector machine regression algorithm based on chunking incremental learning. In: Proceedings of ICCS’06. Lecture notes in computer science, vol 3991. Springer, Berlin, pp 547–554
Kvalseth TO (1985) Cautionary Note about R 2. Am Stat 39(4):279–285
Mangasarian OL, Musicant DR (2000) Robust linear and support vector regression. IEEE Trans Pattern Anal Mach Intell 22(9):950–955
Rousseeuw PJ (1984) Least median of squares regression. J Am Stat Assoc 79:871–880
Rousseeuw PJ, Driessen KV (2006) Computing LTS Regression for large data sets. Data Min Knowl Discov 12:29–45
Rousseeuw PJ, Leroy A (1987) Robust regression and outlier detection. Wiley, New York, pp 9–11
Scholkopf B, Sung KK, Burges CJC, Girosi F, Niyogi P, Poggio T, Vapnik V (1997) Comparing support vector machines with Gaussian kernels to radial basis function classifiers. IEEE Trans Signal Process 45(11):2758–2765
Smola AJ, Scholkopf B (1998) A tutorial on support vector regression. NeuroCOLT2 Technical Report NC2-TR-1998-030
Suykens JAK, Vandewalle J (1999) Least squares support vector machine classifiers. Neural Process Lett 9:293–300
Suykens JAK, Brabanter JD, Lukas L, Vandewalle J (2002) Weighted least squares support vector machines: robustness and sparse approximation. Neurocomputing 48:85–105
Tian SF, Huang HK (2002) A simplification algorithm to support vector machines for regression. J Softw 13(6):1169–1172
Vapnik V (1995) The nature of statistical learning theory. Wiley, New York
Wen W, Hao ZF, Yang XW (2008) A heuristic weight-setting strategy and iteratively updating algorithm for weighted least-squares support vector regression. Neurocomputing 71(16–18):3096–3103
Wu CH (2004) Travel-time prediction with support vector regression. IEEE Trans Intell Transp Syst 5(4):276–281
Zhang JS, Gao G (2005) Reweighted robust support vector regression method. Chin J Comput Sci 28(7):1171–1177
Zhao Y, Keong KC (2004) Fast Leave-one-out Evaluation and improvement on inference for LS-SVMs. In: Proceedings of ICPR’04, vol 3, pp 494–497
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Wen, W., Hao, Z. & Yang, X. Robust least squares support vector machine based on recursive outlier elimination. Soft Comput 14, 1241–1251 (2010). https://doi.org/10.1007/s00500-009-0535-9
Published:
Issue Date:
DOI: https://doi.org/10.1007/s00500-009-0535-9