Skip to main content
SpringerLink
Log in

A novel hybrid trust region algorithm based on nonmonotone and LOOCV techniques

  • Published:
Computational Optimization and Applications Aims and scope Submit manuscript

Abstract

In this paper, a novel hybrid trust-region algorithm using radial basis function (RBF) interpolations is proposed. The new algorithm is an improved version of ORBIT algorithm based on two novel ideas. Because the accuracy and stability of RBF interpolation depends on a shape parameter, so it is more appropriate to select this parameter according to the optimization problem. In the new algorithm, the appropriate shape parameter value is determined according to the optimization problem based on an effective statistical approach, while the ORBIT algorithm in all problems uses a fixed shape parameter value. In addition, the new algorithm is equipped with a new intelligent nonmonotone strategy which improves the speed of convergence, while the monotonicity of the sequence of objective function values in the ORBIT may decrease the rate of convergence, especially when an iteration is trapped near a narrow curved valley. The global convergence of the new hybrid algorithm is analyzed under some mild assumptions. The numerical results significantly indicate the superiority of the new algorithm compared with the original version.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4

Similar content being viewed by others

References

  1. Ahookhosh, M., Amini, K., Peyghami, M.: A nonmonotone trust-region line search method for large-scale unconstrained optimization. Appl. Math. Modell. 36, 478–487 (2012)

    Article  MathSciNet  MATH  Google Scholar 

  2. Allen, D.M.: The relationship between variable selection and data agumentation and a method for prediction. Technometrics 16(1), 125–127 (1974)

    Article  MathSciNet  MATH  Google Scholar 

  3. Alkhamis, T.M., Hasan, M., Ahmed, M.A.: Simulated annealing for the unconstrained quadratic pseudo-Boolean function. Eur. J. Oper. Res. 108(3), 641–652 (1998)

    Article  MATH  Google Scholar 

  4. Andrei, N.: An unconstrained optimization test functions collection. Adv. Model. Optim. 10(1), 147–161 (2008)

    MathSciNet  MATH  Google Scholar 

  5. Bozzini, M., Lenarduzzi, L., Schaback, R.: Adaptive interpolation by scaled multiquadrics. Adv. Comput. Math. 16, 375–387 (2002)

    Article  MathSciNet  MATH  Google Scholar 

  6. Buhmann, M.D.: Radial Basis Functions. Cambridge University Press, New York (2003)

    Book  MATH  Google Scholar 

  7. Conn, A.R., Scheinberg, K., Vicente, L.N.: Global convergence of general derivative-free trust-region algorithms to first and second order critical points. SIAM J. Optim. 20, 387–415 (2009)

    Article  MathSciNet  MATH  Google Scholar 

  8. Dolan, E.D., Moré, J.J.: Benchmarking optimization software with performance profiles. Math. Progr. 91(2), 201–213 (2002)

    Article  MathSciNet  MATH  Google Scholar 

  9. Deng, N., Xiao, Y., Zhou, F.: Nonmonotonic trust region algorithm. J. Optim. Theory Appl. 76(2), 259–285 (1993)

    Article  MathSciNet  MATH  Google Scholar 

  10. Elster, C., Neumaier, A.: A grid algorithm for bound constrained optimization of noisy function. IMA J. Numer. Anal. 15, 585–608 (1995)

    Article  MathSciNet  MATH  Google Scholar 

  11. Esmaeilbeigi, M., Hosseini, M.M.: Dynamic node adaptive strategy for nearly singular problems on large domains. Eng. Anal. Bound. Elem. 36(9), 1311–1321 (2012)

    Article  MathSciNet  MATH  Google Scholar 

  12. Esmaeili, H., Kimiaei, M.: A new adaptive trust-region method for system of nonlinear equations. Appl. Math. Modell. 38, 3003–3015 (2014)

    Article  MathSciNet  MATH  Google Scholar 

  13. Fowler, K.R., Reese, J.P., Kees Jr., C.E., Dennis, J.E., Kelley, C.T., Miller, C.T., Audet, C., Booker, A.J., Couture, G., Darwin, R.W., Farthing, M.W., Finkel, D.E., Gablonsky, J.M., Gray, G., Kolda, T.G.: A comparison of derivative-free optimization methods for water supply and hydraulic capture community problems. Adv. Water Res. 31, 743–757 (2008)

    Article  Google Scholar 

  14. Franke, R.: Scattered data interpolation: tests of some method. Math. Comput. 38(157), 181–200 (1982)

    MathSciNet  MATH  Google Scholar 

  15. Fasshauer, G.E., McCourt, M.: Kernel-based approximation methods using MATLAB. In: Interdisciplinary Mathematical Sciences, vol. 19. World Scientific, Singapore (2015)

  16. Fasshauer, G.E., Zhang, J.G.: On choosing optimal shape parameters for RBF approximation. Numer. Algorithms 45(1–4), 345–368 (2007)

    Article  MathSciNet  MATH  Google Scholar 

  17. Fletcher, R.: Practical Method of Optimization, Unconstrained Optimization. Wiley, New York (2000)

    Book  Google Scholar 

  18. Fornberg, B., Zuev, J.: The Runge phenomenon and spatially variable shape parameters in RBF interpolation. Comput. Math. Appl. 54, 379–398 (2007)

    Article  MathSciNet  MATH  Google Scholar 

  19. Fu, J., Sun, W.: Nonmonotone adaptive trust-region method for unconstrained optimization problems. Appl. Math. Comput. 163(1), 489–504 (2005)

    MathSciNet  MATH  Google Scholar 

  20. Gould, N., Orban, D., Toint, P.L.: CUTEst: a constrained and unconstrained testing environment with safe threads for mathematical optimization. Comput. Optim. Appl. 60(3), 545–557 (2015)

    Article  MathSciNet  MATH  Google Scholar 

  21. Grippo, L., Lampariello, F., Lucidi, S.: A nonmonotone line search technique for Newton’s method. SIAM J. Numer. Anal. 23(4), 707716 (1986)

    Article  MathSciNet  MATH  Google Scholar 

  22. Grippo, L., Lampariello, F., Lucidi, S.: A truncated Newtone method with nonmonotone line search for unconstrained optimization. J. Optim. Theory Appl. 60(3), 401–419 (1998)

    Article  MATH  Google Scholar 

  23. Gray, G.A., Kolda, T.G.: Algorithm 856: APPSPACK 4.0: asynchronous parallel pattern search for derivative-free optimization. ACM Trans. Math. Softw. 32, 485–507 (2006)

    Article  MathSciNet  MATH  Google Scholar 

  24. Hamrani, A., Belaidi, I., Monteiro, E., Lorong, Ph: On the factors affecting the accuracy and robustness of smoothed-radial point interpolation method. Adv. Appl. Math. Mech. 9(1), 43–72 (2017)

    Article  MathSciNet  Google Scholar 

  25. Hardy, R.L.: Multiquadric equations of topography and other irregular surfaces. J. Geophys. Res. 76(8), 1905–1915 (1971)

    Article  Google Scholar 

  26. Hickernell, F.J., Hon, Y.C.: Radial basis function approximations as smoothing splines. Appl. Math. Comput. 102(1), 1–24 (1999)

    Article  MathSciNet  MATH  Google Scholar 

  27. Liu, J., Xu, X., Cui, X.: An accelerated nonmonotone trust region method with adaptive trust region for unconstrained optimization. Comput. Optim. Appl. 69(1), 77–97 (2018)

    Article  MathSciNet  MATH  Google Scholar 

  28. Moré, J., Wild, S.M.: Benchmarking derivative-free optimization algorithms. SIAM J. Optim. 20(1), 172–191 (2009)

    Article  MathSciNet  MATH  Google Scholar 

  29. Narcowich, F.J., Ward, J.D.: Norm estimates for the inverses of a general class of scattered-data radial-function interpolation matrices. J. Approx. Theory 69, 84–109 (1992)

    Article  MathSciNet  MATH  Google Scholar 

  30. Oeuvray, R.: Trust-region methods based on radial basis functions with application to biomedical imaging. Ph.D. Thesis, EPFL, Lausanne (2005)

  31. Oeuvray, R., Bierlaire, M.: A new derivative-free algorithm for the medical image registration problem. Int. J. Model. Simul. 27, 115–124 (2007)

    Article  Google Scholar 

  32. Powell, M.J.D.: A new algorithm for unconstrained optimization. In: Rosen, J.B., Mangassarian, O.L., Ritter, K. (eds.) Nonlinear Programming, pp. 31–66. Academic, New York (1970)

    Chapter  Google Scholar 

  33. Ren, Y.F., Wu, Y.: An efficient algorithm for high-dimensional function optimization. Soft. Comput. 17(6), 995–1004 (2013)

    Article  Google Scholar 

  34. Rippa, S.: An algorithm for selecting a good value for the parameter c in radial basis function interpolation. Adv. Comput. Math. 11(2–3), 193–210 (1999)

    Article  MathSciNet  MATH  Google Scholar 

  35. Schaback, R.: Error estimates and condition numbers for radial basis function interpolation. Adv. Comput. Math. 3, 251–264 (1995)

    Article  MathSciNet  MATH  Google Scholar 

  36. Scheuerer, M.: An alternative procedure for selecting a good value for the parameter c in RBF-interpolation. Adv. Comput. Math. 34(1), 105–126 (2011)

    Article  MathSciNet  MATH  Google Scholar 

  37. Wahba, G.: Spline Models for Observational Data. SIAM, Philadelphia (1990)

    Book  MATH  Google Scholar 

  38. Wendland, H.: Scattered Data Approximation, Cambridge Monographs on Applied and Computational Mathematics, vol. 17. Cambridge University Press, Cambridge (2005)

    Google Scholar 

  39. Wild, S.M., Regis, R.G., Shoemaker, C.A.: Optimization by radial basis function interpolation in trust-region. SIAM J. Sci. Comput. 30(6), 3197–3219 (2008)

    Article  MathSciNet  MATH  Google Scholar 

  40. Wild, S.M., Shoemaker, C.A.: Global convergence of radial basis function trust region derivative-free algorithms. SIAM J. Optim. 21(3), 761–781 (2011)

    Article  MathSciNet  MATH  Google Scholar 

  41. Wright, G.B.: Radial basis function interpolation: numerical and analytical developments. Ph.D. Thesis (2003)

  42. Zhang, H.C., Hager, W.W.: A nonmonotone line search technique and its application to unconstrained optimization. SIAM J. Optim. 14(4), 1043–1056 (2004)

    Article  MathSciNet  MATH  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Mathematics, Hamedan Branch, Islamic Azad University, Hamedan, Iran

    M. Ahmadvand & F. M. Yaghoobi

  2. Faculty of Mathematical Sciences and Statistics, Malayer University, Malayer, Iran

    M. Esmaeilbeigi

  3. Faculty of Mathematical Sciences, University of Science and Technology of Mazandaran, Behshahr, Iran

    A. Kamandi

Authors
  1. M. Ahmadvand
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. M. Esmaeilbeigi
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. A. Kamandi
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. F. M. Yaghoobi
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to M. Esmaeilbeigi.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Ahmadvand, M., Esmaeilbeigi, M., Kamandi, A. et al. A novel hybrid trust region algorithm based on nonmonotone and LOOCV techniques. Comput Optim Appl 72, 499–524 (2019). https://doi.org/10.1007/s10589-018-0051-x

Download citation

  • Received:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10589-018-0051-x

Keywords

Mathematics Subject Classification

Search

Navigation