Skip to main content
Log in

Nonlinear optimisation using directional step lengths based on RPROP

  • Original Paper
  • Published:
Optimization Letters Aims and scope Submit manuscript

Abstract

A search method based on the backpropagation rule commonly used for training neural networks is proposed here for the optimisation of smooth nonlinear functions. The use of the Resilient backPROPagation (RPROP) heuristic rule for local minimisation is described. The details of employing the directional step length determined by RPROP along with a simple restarting scheme are provided. In the approach proposed here direct use of the directional step determined by the heuristic without using any line search conditions takes place. The overall algorithm has been tested on a number of benchmark functions found in the literature with very positive results. The test problems’ dimension ranges from 100 to 50,000. The results obtained show that the suggested search direction method results to a highly efficient algorithm suitable for large scale optimisation.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1

Similar content being viewed by others

References

  1. Anastasiadis, A., Magoulas, G., Vrahatis, M.: New globally convergent training scheme based on the resilient propagation algorithm. Neurocomputing 64, 253–270 (2005)

    Article  Google Scholar 

  2. Andrei, N.: An unconstrained optimization test functions collection. Adv. Model. Optim. 10(1), 147–161 (2008)

    MATH  MathSciNet  Google Scholar 

  3. Fletcher, R.: Pract. Methods Optim., 2nd edn. Wiley-Interscience, New York, NY, USA (1987)

    Google Scholar 

  4. Hecht-Nielsen, R.: Theory of the backpropagation neural network. In: International Joint Conference on Neural Networks, vol. 1, pp. 593–605. Washington, DC, USA (1989)

  5. Igel, C., Hüsken, M.: Empirical evaluation of the improved RPROP learning algorithms. Neurocomputing 50, 105–123 (2003)

    Article  MATH  Google Scholar 

  6. Kotsialos, A.: Non-Smooth Optimization Based on Resilient Backpropagation Search for Unconstrained and Simply Bounded Problems. Optimization Methods and Software (available on line) (2012)

  7. Kotsialos, A., Papageorgiou, M.: Nonlinear optimal control applied to coordinated ramp metering. IEEE Trans. Control Syst. Technol. 10(6), 920–933 (2004)

    Article  Google Scholar 

  8. Lukšan, L., Vlček, J.: Test Problems for Unconsgtrained Optimization. Institute of Computing Science, Academy of Sciences of the Czech Republic, Technical Report (2003)

  9. Mastorocostas, P., Hilas, C.: A computational intelligence forecasting system for telecommunications time series. Eng. Appl. Artif. Intell 25(1), 200–206 (2012)

    Article  Google Scholar 

  10. Nelder, J., Mead, R.: A simplex method for function minimization. Comput. J. 7, 308–313 (1965)

    Article  MATH  Google Scholar 

  11. Ng, S., Leung, S., Luk, A., Wu, Y.: Convergence analysis of generalized back-propagation algorithm with modified gradient function. In: Proceedings of IEEE International Joint Conference on Neural Networks, pp. 3672–3678. Vancouver, BC, Canada (2006)

  12. Nocedal, J., Wright, S.: Numer. Optim. Springer, New York (1999)

    Book  Google Scholar 

  13. Prechelt, L.: PROBEN1—a set of neural network benchmark problems and benchmarking rules. Faculty of Informatics, University of Karlsruhe, Technical Report (1994)

  14. Riedmiller, M., Braun, H.: A direct adaptive method for faster backpropagation learning: The RPROP algorithm. In: IEEE International Conference On, Neural Networks, pp. 586–591 (1993)

  15. Zhan, Z.H., Zhang, J., Li, Y., Chung, H.H.: Adaptive particle swarm optimization. IEEE Trans. Syst. Man Cybern. Part B Cybern. 39(6), 1362–1381 (2009)

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Apostolos Kotsialos.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Kotsialos, A. Nonlinear optimisation using directional step lengths based on RPROP. Optim Lett 8, 1401–1415 (2014). https://doi.org/10.1007/s11590-013-0668-8

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11590-013-0668-8

Keywords

Navigation