Abstract
In the Neural Network universe, the Backpropagation and the Levenberg-Marquardt are the most used algorithms, being almost consensual that the latter is the most effective one. Unfortunately for this algorithm it has not been possible to develop a true iterative version for on-line use due to the necessity to implement the Hessian matrix and compute the trust region. To overcome the difficulties in implementing the iterative version, a batch sliding window with Early Stopping is proposed, which uses a hybrid Direct/Specialized evaluation procedure. The final solution is tested with a real system.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Levenberg, K.: A method for the solution of certain problems in least squares. Quart. Appl. Math. 2, 164–168 (1944)
Marquardt, D.: An algorithm for least -squares estimation of nonlinear parameters. SIAM J. Appl. Math. 11, 431–441 (1963)
Nørgaard, M., Ravn, O., Poulsen, N.K., Hansen, L.K.: Neural Networks for Modelling and Control of Dynamic Systems. Springer, Heidelberg (2000)
Ngia, L.S.H.: System Modeling Using Basis Functions and Application to Echo Cancelation, Ph.D. thesis, Department of Signals and Systems School of Electrical and Computer Engineering, Chalmers University of Technology (2000)
Ferreira, P., Faria, E., Ruano, A.: Neural network models in greenhouse air temperature prediction. Neurocomputing 43(1-4), 51–75 (2002)
Dias, F.M., Antunes, A., Vieira, J., Mota, A.M.: Implementing the levenberg-marquardt algorithm on-line: A sliding window approach with early stopping. In: 2nd IFAC Workshop on Advanced Fuzzy/Neural Control (2004)
Morgan, N., Bourlard, H.: Generalization and parameter estimation in feedforward nets: Some experiments. In: Touretzsky, D. (ed.) Advances in Neural Information Processing Systems, pp. 630–637. Morgan Kaufmann, San Francisco (1990)
Sjöberg, J.: Non-Linear System Identification with Neural Networks, Ph.D. thesis, Dept. of Electrical Engineering, Linköping University, Suécia (1995)
Dias, F.M., Antunes, A., Mota, A.: A new hybrid direct/specialized approach for generating inverse neural models. WSEAS Transactions on Systems 3(4), 1521–1529 (2004)
Nørgaard, M.: Neural network based system identification toolbox for use with matlab, version 1.1, technical report, Tech. Rep., Technical University of Denmark (1996)
Nørgaard, M.: Neural network based control system design toolkit for use with matlab, version 1.1, technical report, Tech. Rep., Technical University of Denmark (1996)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2005 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Dias, F.M., Antunes, A., Vieira, J., Mota, A.M. (2005). On-line Training of Neural Networks: A Sliding Window Approach for the Levenberg-Marquardt Algorithm. In: Mira, J., Álvarez, J.R. (eds) Artificial Intelligence and Knowledge Engineering Applications: A Bioinspired Approach. IWINAC 2005. Lecture Notes in Computer Science, vol 3562. Springer, Berlin, Heidelberg. https://doi.org/10.1007/11499305_59
Download citation
DOI: https://doi.org/10.1007/11499305_59
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-26319-7
Online ISBN: 978-3-540-31673-2
eBook Packages: Computer ScienceComputer Science (R0)