Skip to main content
SpringerLink
Log in

A Hybrid Training Algorithm for Feedforward Neural Networks

  • Published:
Neural Processing Letters Aims and scope Submit manuscript

Abstract

Feedforward neural networks (FNN) have been proposed to solve complex problems in pattern recognition, classification and function approximation. Despite the general success of learning methods for FNN, such as the backpropagation (BP) algorithm, second-order algorithms, long learning time for convergence remains a problem to be overcome. In this paper, we propose a new hybrid algorithm for a FNN that combines unsupervised training for the hidden neurons (Kohonen algorithm) and supervised training for the output neurons (gradient descent method). Simulation results show the effectiveness of the proposed algorithm compared with other well-known learning methods.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Rumelhart D.E., Hinton G.E. Williams R.J. (1986) Learning internal representations by error propagation, Parallel Distributed Processing: Explorations in the Microstructures of Cognition, Vol 1. MIT Press, Cambridge MA, pp. 318–362

    Google Scholar 

  2. Lera G., Pinzolas M., (2002) Neighborhood Based Levenberg-Marquardt Algorithm for Neural Network Training. IEEE Transactions on Neural Networks 13(5): 1200–1203

    Article  Google Scholar 

  3. Silva F., Almeida L., (1990) Speeding-Up Backpropagation. In: Eckmiller R. (ed), Advanced Neural Computers. North-Holland, Amsterdam, pp. 151–156

    Google Scholar 

  4. Scalero R.S., Tepedelenlioglu N., (1992), A fast algorithm for training feedforward neural networks. IEEE Transactions on Signal Processing 40(1): 202–210

    Article  Google Scholar 

  5. Fahlman S.E.: Faster learning variations on back propagation, An Empirical Study, In: Proceedings of the 1988 Connectionist Models Summer School, pp. 38–51, Carnegie Mellon University.

  6. Rojas R., (1996) Neural Networks : A Systematic Introduction. Springer-Verlag, Berlin

    Google Scholar 

  7. Jacobs R.A. (1988), Increased rates of convergence through learning rate adaptation. Neural Networks 1: 295–307

    Article  Google Scholar 

  8. Najim K.,, Chtourou M., Thibault J. (1992), Neural network synthesis using learning automata. Journal of Systems Engineering 2(4): 192–197

    Google Scholar 

  9. Poznyak A.S., Najim K., Chtourou M. (1993), Use of recursive stochastic algorithm for neural network synthesis. Applied Math Modelling 17: 444–448

    Article  Google Scholar 

  10. Nickolai S.R. (2000), The layer-wise method and the backpropagation hybrid approch to learning a feedforward neural network. IEEE Transactions on Neural Networks 11(2): 295–305

    Article  Google Scholar 

  11. Abid S., Fnaiech F., Najim M. (2001), A fast feedforward training algorithm using a modified form of the standard backpropagation algorithm. IEEE Transactions on Neural Networks 12(2): 424–430

    Article  Google Scholar 

  12. Oscar F.R., Deniz E., Jose C.P.: Accelerating the convergence speed of neural networks learning methods using least squares, In: Proceedings of the ESANN’2003, pp. 255–260, Belgium, 2003

  13. Bilski J. (2005), The UD RLS algorithm for training feedforward neural networks, Internat. Journal of Applied Mathematical Computer Science 15(1): 115–123

    MATH  Google Scholar 

  14. Liu P., Li H. (2004), Efficient learning algorithms for three-layer regular feedforward fuzzy neural networks. IEEE Transactions on Neural Networks 15(3): 545–558

    Article  MATH  Google Scholar 

  15. Bortoletti A., Di FIore C., Fanelli S., Zellini P. (2003), A new class of Quasi-Newtonian methods for optimal learning in MLP-networks. IEEE Transactions on Neural Networks 14(2): 263–273

    Article  Google Scholar 

  16. Hornik K., Stinchcombe M., White H. (1989), Multilayer feedforward networks are universal approximators. Neural Networks 2: 359–366

    Article  Google Scholar 

  17. Kohonen T. (1990), The self-organizing map. Proceedings of the IEEE 78(9): 1464–1480

    Article  Google Scholar 

  18. Azimi-Sadjadi I.M.R., Liou R.J. (1992), Fast learning process of multilayer neural nets using recursive least squares method. IEEE Transactions on Signal Processing 40(2): 443–446

    Article  Google Scholar 

  19. Alexander G.P., Benito F., Amir F.A., Jayakumar M., Wei K.T. (1994), An accelerated learning algorithm for multilayer networks. IEEE Transactions on Neural Networks 5(3): 493–497

    Article  Google Scholar 

  20. Mackey M.C., Class L. (1977), Oscillation and chaos in physiological control system. Sience 197: 287–289

    ADS  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Electrical Engineering, Research Unit on Intelligent Control, Design and Optimization of Complex Systems (ICOS), ENIS, BPW, 3038, Sfax, Tunisia

    Mounir Ben Nasr & Mohamed Chtourou

Authors
  1. Mounir Ben Nasr
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Mohamed Chtourou
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Mohamed Chtourou.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Nasr, M.B., Chtourou, M. A Hybrid Training Algorithm for Feedforward Neural Networks. Neural Process Lett 24, 107–117 (2006). https://doi.org/10.1007/s11063-006-9013-x

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11063-006-9013-x

Keywords

Search

Navigation