Abstract
Estimating the non-flat function which comprises both the steep variations and the smooth variations is a hard problem. The existing kernel methods with a single common variance for all the regressors can not achieve satisfying results. In this paper, a novel multi-scale model is constructed to tackle the problem by orthogonal least squares regression (OLSR) with wavelet kernel. The scheme tunes the dilation and translation of each wavelet kernel regressor by incrementally minimizing the training mean square error using a guided random search algorithm. In order to prevent the possible over-fitting, a practical method to select termination threshold is used. The experimental results show that, for non-flat function estimation problem, OLSR outperforms traditional methods in terms of precision and sparseness. And OLSR with wavelet kernel has a faster convergence rate as compared to that with conventional Gaussian kernel.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Smola, A.: Regression Estimation with Support Vector Learning Machines. Master’s Thesis, Technische University München (1996), Available at http://www.kernel-machines.org
Suykens, J.A.K., Vandewalle, J.: Least Squares Support Vector Machine Classifiers. Neural Process. Lett. 9, 293–300 (1999)
Smola, A., Schölkopf, B., Rätsch, G.: Linear Programs for Automatic Accuracy Control in Regression. In: Proceedings of the Ninth International Conference on Artificial Neural Networks, London, pp. 575–580 (1999)
Zheng, D., Wang, J., Zhao, Y.: Non-flat Function Estimation with A Multi-scale Support Vector Regression. Neurocomputing (in press)
Zheng, D., Wang, J., Zhao, Y.: Training Sparse MS-SVR with an Expectation-Maximization Algorithm. Neurocomputing 69, 1659–1664 (2006)
Guigue, V., Rakotomamonjy, A., Canu, S.: Kernel Basis Pursuit. In: Gama, J., Camacho, R., Brazdil, P.B., Jorge, A.M., Torgo, L. (eds.) ECML 2005. LNCS (LNAI), vol. 3720, pp. 146–157. Springer, Heidelberg (2005)
Chen, S., Billings, S.A., Luo, W.: Orthogonal Least Squares Methods and Their Application to Non-linear System Identification. Int. J. Control 50, 1873–1896 (1989)
Chen, S., Cowan, C.F.N., Grant, P.M.: Orthogonal Least Squares Learning Algorithm for Radial Basis Function Networks. IEEE Trans. Neural Networks 2, 302–309 (1991)
Chen, S., Wang, X.X., Brown, D.J.: Orthogonal Least Squares Regression with Tunable Kernels. Electronics Letters 41(8) (2005)
Chen, S.Y., Wu, Y., Luk, B.L.: Combined Genetic Algorithm Optimization and Regularized Orthogonal Least Squares Learning for Radial Basis Function Networks. IEEE Trans. Neural Networks 10(5), 1239–1243 (1999)
Chen, S., Wang, X.X., Harris, C.J.: Experiments with Repeating Weighted Boosting Search for Optimization in Signal Processing Applications. IEEE Trans. Syst. Man Cybern. B, Cybern. 35(4), 682–693 (2005)
Mallat, S.: A Wavelet Tour of Signal Processing. Academic Press, London (1999)
Daubechies, I.: Ten Lectures on Wavelets. CBMS, vol. 61. SIAM, Philadelphia (1992)
Zhang, L., Zhou, W., Jiao, L.: Wavelet Support Vector Machine. IEEE Trans. on System, Man and Cybernetics-Part B: Cybernetics 34, 34–39 (2004)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2007 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Zhang, M., Fu, L., He, T., Wang, G. (2007). Non-flat Function Estimation Using Orthogonal Least Squares Regression with Multi-scale Wavelet Kernel. In: Liu, D., Fei, S., Hou, ZG., Zhang, H., Sun, C. (eds) Advances in Neural Networks – ISNN 2007. ISNN 2007. Lecture Notes in Computer Science, vol 4491. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-540-72383-7_75
Download citation
DOI: https://doi.org/10.1007/978-3-540-72383-7_75
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-72382-0
Online ISBN: 978-3-540-72383-7
eBook Packages: Computer ScienceComputer Science (R0)