Abstract
This paper proposes an improved orthogonal least square algorithm based on Singular Value Decomposition for spare basis selection of the linear-in-the-weights regression models. The improved algorithm is based on the idea of reducing meaningless calculation of the selection process through the improvement of orthogonal least square by using the Singular Value Decomposition. This is achieved by dividing the original candidate bases into several parts to avoid comparing among poor candidate regressors. The computation is further simplified by utilizing the Singular Value Decomposition to each sub-block and replacing every sub-candidate bases with the obtained left singular matrix, which is a unitary matrix with lower dimension. It can avoid the computation burden of the repeated orthogonalisation process before each optimal regressor is determined. This algorithm is applied to the linear-in-the-weights regression models with the predicted residual sums of squares (PRESS) statistic and minimizes it in an incremental manner. For several real and benchmark examples, the present results indicate that the proposed algorithm can relieve the load of the heave calculation and achieve a spare model with good performance.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Chen, S.M., Hwang, J.R.: Temperature prediction using fuzzy time series. IEEE Transactions on Systems, Man and Cybernetics -Part B 30(2), 263–275 (2000)
Coulibaly, P., Anctil, F., Bobee, B.: Multivariate reservoir inflow forecasting using temporal neural networks. Journal of Hydrologic Engineering 6(5), 367–376 (2001)
Han, M., Wang, Y.J.: Analysis and modeling of multivariate chaotic time series based on neural network. Expert System with Applications 36(2), 1280–1290 (2009)
Feng, G., Huang, G.B., Lin, Q., Gay, R.: Error Minimized Extreme Learning Machine with Growth of Hidden Nodes and Incremental Learning. IEEE Transactions on Neural Networks 20(8), 1352–1357 (2009)
Huang, G.B., Chen, L., Siew, C.K.: Universal Approximation Using Incremental Constructive Feedforward Networks with Random Hidden Nodes. IEEE Transactions on Neural Networks 17(4), 879–892 (2006)
Salmerón, M., Ortega, J., Puntonet, C.G., Prieto, A.: Improved RAN sequential prediction using orthogonal techniques. Neurocomputing 41, 153–172 (2001)
Rojas, I., Pomares, H., Bernier, J.L., Ortega, J., Pino, B., Pelayo, F.J., Prieto, A.: Time series analysis using normalized PG-RBF network with regression weights. Neurocomputing 42, 267–285 (2002)
Billings, S.A., Wei, H.L.: A New Class of Wavelet Networks for Nonlinear System Identification. IEEE Transactions on Neural Networks 16(4), 862–874 (2005)
Hong, X., Chen, S.: M-estimator and D-optimality model construction using orthogonal forward regression. IEEE Transactions on Systems, Man and Cybernetics, Part: B 35(1), 155–162 (2005)
Chen, S., Hong, X., Harris, C.J., Sharkey, P.M.: Sparse Modeling Using Orthogonal Forward Regression with PRESS Statistic and Regularization. IEEE Transactions on Systems, Man and Cybernetics, Part: B 34(2), 898–911 (2004)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2010 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Han, M., Li, Dc. (2010). Orthogonal Least Squares Based on Singular Value Decomposition for Spare Basis Selection. In: Zhang, L., Lu, BL., Kwok, J. (eds) Advances in Neural Networks - ISNN 2010. ISNN 2010. Lecture Notes in Computer Science, vol 6063. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-13278-0_53
Download citation
DOI: https://doi.org/10.1007/978-3-642-13278-0_53
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-13277-3
Online ISBN: 978-3-642-13278-0
eBook Packages: Computer ScienceComputer Science (R0)