Abstract
It is known in statistics that the linear estimators using the rank-based Wilcoxon approach in linear regression problems are usually insensitive to outliers. Outliers are the data points that differ greatly from the pattern set by the bulk of the data. Inspired by this, Hsieh et al introduced the Wilcoxon approach into the area of machine learning. They investigated four new learning machines, such as Wilcoxon neural network (WNN) etc., and developed four descent gradient based backpropagation algorithms to train these learning machines. The performances of these machines are better than the ordinary nonrobust neural networks. However, it is hard to balance the learning speed and the stability of these algorithms which is inherently the drawback of gradient descent based algorithms. In this paper, a new algorithm is used to train the output weights of single-layer feedforward neural networks (SLFN) with its input weights and biases being randomly chosen. This algorithm is called Wilcoxon-norm based robust extreme learning machine or WRELM for short.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsPreview
Unable to display preview. Download preview PDF.
References
Kohonen, T.: Self-organizing formation of topologically correct feature maps. Biological Cybernetics 43 (1982)
Powell, M.J.D.: Radial basis functions for multivariable interpolation: a review. In: Mason, J.C., Cox, M.G. (eds.) Algorithms for Approximation, pp. 143–167. Clarendon Press, Oxford (1987)
Cortes, C., Vapnik, V.: Support vector networks. Machine Learning 20, 273–297 (1995)
Werbos, P.: Beyond regression: new tools for prediction and analysis in the behavioral sciences. Ph.D. dissertation, Harvard Univ., Cambridge, MA (1974)
Gibb, J.: Back propagation family album. Technical Report C/TR96-05, Macquarie University (August 1996)
Riedmiller, M., Braun, H.: A direct adaptive method for faster backpropagation learning: the RPROP algorithm. In: Proc. of the IEEE Int. Conf. on Neural Netw., San Francisco, CA (April 1993)
Leshno, M., Lin, V.Y., Pinkus, A., Schocken, S.: Multilayer feedfor-ward networks with a nonpolynomial activation function can approximate any function. Neural Netw. 6, 861–867 (1993)
Huang, G.B.: Extreme learning machine: a new learning scheme of feedforward neural networks. In: Proc. Int. Joint Conf. Neural Netw. (IJCNN 2004), Budapest, Hungary, July 25–29, vol. 2, pp. 985–990 (2004)
Huang, G.B., Zhou, H.M., Ding, X.J., Zhang, R.: Extreme learning machine for regression and multiclass classification. IEEE Transactions on Systems, Man, and Cybernetics, Part B: Cybernetics 42(2), 513–529 (2012)
Liang, N.Y., Huang, G.B., Saratchandran, P., Sundararajan, N.: A fast and accurate online sequential learning algorithm for feedforward network. IEEE Trans. on Neural Netw., 17(6) (November 2006)
Hawkins, D.M.: Identification of Outliers. Chapman & Hall, London (1980)
Jureckova, J.: Asymptotic linearity of a rank statistic in regression parameter. Ann. Math. Statist. 40, 1889–1900 (1969)
Jaeckel, L.A.: Estimating regression coefficients by minimizing the dispersion of the residuals. Ann. Math. Statist. 43, 1449–1458 (1972)
Hettmansperger, T.P.: Statistical inference based on ranks. Wiley, New York (1984)
Hettmansperger, T.P., McKean, J.W.: Robust non-parametric statistics. Wiley, New York (1998)
Hettmansperger, T.P., McKean, J.W.: Robust nonparametric statistical methods. Arnold, London (1998)
Scuster, E.: On the rate of convergence of an estimate of a functional of a probability density. Scandinavian Actuarial Journal 1, 103–107 (1974)
Choi, Y.H., Ozturk, O.: A new class of score generating functions for regression models. Statistics & Probability Letters 57, 205–214 (2002)
Asuncion, A., Newman, D.J.: UCI Machine Learning Repository (2007). http://www.ics.uci.edu/~mlearn/MLRepository.html
Hsieh, J.G., Lin, Y.L., Jeng, J.H.: Preliminary study on Wilcoxon learning machines. IEEE Trans. on Neural Netw. 19(2), 201–211 (2008)
Rusiecki, A.: Robust LTS backpropagation learning algorithm. In: Sandoval, F., Prieto, A.G., Cabestany, J., Graña, M. (eds.) IWANN 2007. LNCS, vol. 4507, pp. 102–109. Springer, Heidelberg (2007)
Qing, C.Y., Annpey, P., Biao, X.: Rank regression in stability analysis. Journal of Biophamaceutical Statistics 13, 463–479 (2003)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2014 Springer International Publishing Switzerland
About this paper
Cite this paper
Xie, XL., Bian, GB., Hou, ZG., Feng, ZQ., Hao, JL. (2014). Wilcoxon-Norm-Based Robust Extreme Learning Machine. In: Zeng, Z., Li, Y., King, I. (eds) Advances in Neural Networks – ISNN 2014. ISNN 2014. Lecture Notes in Computer Science(), vol 8866. Springer, Cham. https://doi.org/10.1007/978-3-319-12436-0_23
Download citation
DOI: https://doi.org/10.1007/978-3-319-12436-0_23
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-12435-3
Online ISBN: 978-3-319-12436-0
eBook Packages: Computer ScienceComputer Science (R0)