Skip to main content

Advertisement

SpringerLink
Log in

Hybrid spectral gradient method for the unconstrained minimization problem

  • Published:
Journal of Global Optimization Aims and scope Submit manuscript

Abstract

We present a hybrid algorithm that combines a genetic algorithm with the Barzilai–Borwein gradient method. Under specific assumptions the new method guarantees the convergence to a stationary point of a continuously differentiable function, from any arbitrary initial point. Our preliminary numerical results indicate that the new methodology finds efficiently and frequently the global minimum, in comparison with the globalized Barzilai–Borwein method and the genetic algorithm of the Toolbox of Genetic Algorithms of MatLab.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Azofeifa D., Clark N., Vargas W.: Optical and electrical properties of terbium films as a function of hydrogen concentration. Phys. Stat. B – Basic Solid State Phys. 242, 2005–2009 (2005)

    Article  Google Scholar 

  2. Barzilai J., Borwein J.M.: Two-point step size gradient methods. IMA J. Numer. Anal. 8, 141–148 (1988)

    Article  Google Scholar 

  3. Bellavia S., Morini B.: A globally convergent Newton-GMRES subspace method for systems of nonlinear equations. SIAM J. Sci. Comput. 23, 940–960 (2001)

    Article  Google Scholar 

  4. Bello L., Raydan M.: Convex constrained optimization for the seismic reflection tomography problem. J. Appl. Geophys. 62, 158–166 (2007)

    Article  Google Scholar 

  5. Bielschowsky R.H., Friedlander A., Gomes F.A., Martínez J.M., Raydan M.: An adaptive algorithm for bound constrained quadratic minimization. Invest. Oper. 7, 67–102 (1997)

    Google Scholar 

  6. Birgin E.G., Chambouleyron I., Martínez J.M.: Estimation of the optical constants and the thickness of thin films using unconstrained optimization. J. Comput. Phys. 151, 862–880 (1999)

    Article  Google Scholar 

  7. Birgin E.G., Chambouleyron I., Martínez J.M.: Optimization problems in the estimation of parameters of thin films and the elimination of the influence of the substrate. J. Comput. Appl. Math. 152, 35–50 (2003)

    Article  Google Scholar 

  8. Birgin E.G., Evtushenko Y.G.: Automatic differentiation and spectral projected gradient methods for optimal control problems. Optim. Methods Softw. 10, 125–146 (1998)

    Article  Google Scholar 

  9. Birgin E.G., Martínez J.M., Mascarenhas W.F., Ronconi D.P.: Method of sentinels for packing items within arbitrary convex regions. J. Oper. Res. Soc. 57, 735–746 (2006)

    Article  Google Scholar 

  10. Birgin E.G., Martínez J.M., Nishihara F.H., Ronconi D.P.: Orthogonal packing of rectangular items within arbitrary convex regions by nonlinear optimization. Comput. Oper. Res. 33, 3535–3548 (2006)

    Article  Google Scholar 

  11. Birgin E.G., Martínez J.M., Raydan M.: Nonmonotone spectral projected gradient methods on convex sets. SIAM J. Optim. 10, 1196–1211 (2000)

    Article  Google Scholar 

  12. Birgin E.G., Martínez J.M., Raydan M.: Algorithm 813: SPG—software for convex-constrained optimization. ACM Trans. Math. Softw. 27, 340–349 (2001)

    Article  Google Scholar 

  13. Birgin E.G., Martínez J.M., Raydan M.: Inexact spectral projected gradient methods on convex sets. IMA J. Numer. Anal. 23, 539–559 (2003)

    Article  Google Scholar 

  14. Birgin E.G., Martínez J.M., Ronconi D.P.: Minimization subproblems and heuristics for an applied clustering problem. Eur. J. Oper. Res. 146, 19–34 (2003)

    Article  Google Scholar 

  15. Birgin E.G., Martínez J.M., Ronconi D.P.: Optimizing the packing of cylinders into a rectangular container: a nonlinear approach. Eur. J. Oper. Res. 160, 19–33 (2005)

    Article  Google Scholar 

  16. Brown P.N., Saad Y.: Hybrid Krylov methods for nonlinear systems of equations. SIAM J. Sci. Comput. 11, 450–481 (1990)

    Article  Google Scholar 

  17. Brown P.N., Saad Y.: Convergence theory of nonlinear Newton–Krylov algorithms. SIAM J. Optim. 4, 297–330 (1994)

    Article  Google Scholar 

  18. Castillo Z., Cores D., Raydan M.: Low cost optimization techniques for solving the nonlinear seismic reflection tomography problem. Optim. Eng. 1, 155–169 (2000)

    Article  Google Scholar 

  19. Coleman T.F., Li Y.: On the convergence of interior-reflective Newton methods for nonlinear minimization subject to bounds. Math. Program. 67, 189–224 (1994)

    Article  Google Scholar 

  20. Coleman T.F., Li Y.: An interior trust region approach for nonlinear minimization subject to bounds. SIAM J. Optim. 6, 418–445 (1996)

    Article  Google Scholar 

  21. Cores D., Fung G.M., Michelena R.J.: A fast and global tow point low storage optimization technique for tracing rays in 2D and 3D isotropic media. J. Appl. Geophys. 45, 273–287 (2000)

    Article  Google Scholar 

  22. Cores D., Loreto M.: A generalized two point ellipsoidal anisotropic ray tracing for converted waves. Optim. Eng. 8, 373–396 (2007)

    Article  Google Scholar 

  23. Curiel F., Vargas W.E., Barrera R.G.: Visible spectral dependence of the scattering and absorption coefficients of pigmented coatings from inversion of diffuse reflectance spectral. Appl. Opt. 41, 5969–5978 (2002)

    Article  Google Scholar 

  24. Dai Y.H.: On the nonmonotone line search. J. Optim. Theory Appl. 112, 315–330 (2002)

    Article  Google Scholar 

  25. Dai Y.H.: Alternate step gradient method. Optimization 52, 395–415 (2003)

    Article  Google Scholar 

  26. Dai Y.H., Fletcher R.: On the asymptotic behaviour of same new gradient methods. Math. Program. 103, 541–559 (2005)

    Article  Google Scholar 

  27. Dai Y.H., Fletcher R.: Project Barzilai–Borwein methods for large-scale box-constrained quadratic programming. Numerische Mathematik 100, 21–47 (2005)

    Article  Google Scholar 

  28. Dai Y.H., Hager W., Schittkowski K., Zhang H.C.: The cyclic Barzilai–Borwein method for unconstrained optimization. IMA J. Numer. Anal. 26, 604–627 (2006)

    Article  Google Scholar 

  29. Dai Y.H., Liao L.Z.: R-linear convergence of the Barzilai and Borwein gradient method. IMA J. Numer. Anal. 22, 1–10 (2002)

    Article  Google Scholar 

  30. Dai Y.H., Yuan J.Y., Yuan Y.X.: Modified two-point stepsize gradient methods for unconstrained optimization. Comput. Optim. Appl. 22, 103–109 (2002)

    Article  Google Scholar 

  31. Dai Y.H., Zhang H.C.: Adaptive two-point stepsize gradient algorithm. Numer. Algorithms 27, 377–385 (2001)

    Article  Google Scholar 

  32. Deidda G.P., Bonomi E., Manzi C.: Inversion of electrical conductivity data with tikhonov regularization approach: some considerations. Ann. Geophys. 46, 549–558 (2003)

    Google Scholar 

  33. Dennis J.E., Schnabel R.B.: Numerical Methods for Unconstrained Optimization and Nonlinear Equations. Prentice-Hall, Englewoog Cliffs (1983)

    Google Scholar 

  34. Davis L.: The Handbook of Genetic Algorithms. Van Nostrand Reingold, New York (1991)

    Google Scholar 

  35. Fletcher R.: Low storage methods for unconstrained optimization. Lectures in Applied Mathematics (AMS) 26, 165–179 (1990)

    Google Scholar 

  36. Fletcher R.: On the Barzilai–Borwein method. Appl. Optim. 96, 235–236 (2006)

    Article  Google Scholar 

  37. Francisco J.B., Martínez J.M., Martínez L.: Density-based globally convergent trust-region methods for self-consistent field electronic structure calculations. J. Math. Chem. 40, 349–377 (2006)

    Article  Google Scholar 

  38. Friedlander A., Martínez J.M., Molina B., Raydan M.: Gradient method with retards and generalizations. SIAM J. Numer. Anal. 36, 275–289 (1999)

    Article  Google Scholar 

  39. Friedlander A., Martínez J.M., Raydan M.: A new method for large-scale constrained convex quadratic minimization problems. Optim. Methods Softw. 5, 57–74 (1995)

    Article  Google Scholar 

  40. Glunt W., Hayden T.L., Raydan M.: Molecular conformation from distance matrices. J. Comput. Chem. 14, 114–120 (1993)

    Article  Google Scholar 

  41. Goldberg D.: Genetic Algorithms in Search, Optimization and Machine Learning. Addison-Wesley, Readomg (1989)

    Google Scholar 

  42. Hendrickson B.A.: The molecule problem: exploiting structure in global optimization. SIAM J. Optim. 5, 835–857 (1995)

    Article  Google Scholar 

  43. Holland J.: Adaptation in Natural and Artificial Systems. The University of Michigan Press, Ann Arbor (1975)

    Google Scholar 

  44. James C.S.: Introduction to Stochastics Search and Optimization. Wiley-Interscience, New Jersey (2003)

    Google Scholar 

  45. La Cruz, W.: Derivative-free residual algorithm for solving weakly nonlinear equations. IMA J. Numer. Anal. (2007), Submitted

  46. La Cruz W., Martínez J.M., Raydan M.: Spectral residual method without gradient information for solving large-scale nonlinear systems of equations. Math. Comput. 75, 1449–1466 (2006)

    Article  Google Scholar 

  47. La Cruz W., Raydan M.: Nonmonotone spectral methods for large-scale nonlinear systems. Optim. Methods Softw. 18, 583–599 (2003)

    Article  Google Scholar 

  48. Meza J.C., Martinez M.L.: On the use of direct search methods for the molecular conformation problem. J. Comput. Chem. 15, 627–632 (1994)

    Article  Google Scholar 

  49. Michalewicz Z.: Genetic Algorithms + Data Structures = Evolution Programs. AI Series. Springer- Verlag, New York (1994)

    Google Scholar 

  50. Molina B., Raydan M.: Preconditioned Barzilai–Borwein method for the numerical solution of partial differential equations. Numer. Algorithms 13, 45–60 (1996)

    Article  Google Scholar 

  51. Nelder J.A., Mead R.: A simplex method for function minimization. Comput. J. 7, 308–313 (1965)

    Google Scholar 

  52. Ramirez-Porras A., Vargas-Castro E.: Transmission of visible light through oxidized copper films: feasibility of using a spectral projected gradient method. Appl. Opt. 43, 1508–1514 (2004)

    Article  Google Scholar 

  53. Raydan M.: On the Barzilai and Borwein choice of steplength for the gradient method. IMA J. Numer. Anal. 13, 321–326 (1993)

    Article  Google Scholar 

  54. Raydan M.: The Barzilai and Borwein gradient method for the large scale unconstrained minimization problem. SIAM J. Optim. 7, 26–33 (1997)

    Article  Google Scholar 

  55. Serafini T., Zanghirati G., Zanni T.: Gradient projection methods for quadratic programs and applications in training support vector machines. Optim. Methods Softw. 20, 353–378 (2005)

    Article  Google Scholar 

  56. Vargas W.E.: Inversion methods from Kubelka–Munk analysis. J. Opt. A - Pure Appl. Opt. 4, 452–456 (2002)

    Article  Google Scholar 

  57. Vargas W.E., Azofeifa D.E., Clark N.: Retrieved optical properties of thin films on absorbing substrates from transmittance measurements by application of a spectral projected gradient method. Thin Solid Films 425, 1–8 (2003)

    Article  Google Scholar 

  58. Wells C., Glunt W., Hayden T.L.: Searching conformational space with the spectral distance geometry algorithm. J. Mol. Struct. (Theochem) 308, 263–271 (1994)

    Article  Google Scholar 

  59. Wright A.H.: Genetic algorithms for real parameter optimization. In: Rawlins J.E. (ed.) Foundations of Genetic Algorithms, pp. 205–218. Morgan Kaufmann (1991)

  60. Zeev N., Savasta O., Cores D.: Non-monotone spectral projected gradient method applied to full waveform inversion. Geophys. Prospect. 54, 525–534 (2006)

    Google Scholar 

  61. Zhina J.: Applications of conditional nonlinear optimal perturbation to the study of the stability and sensitivity of the Jovian atmosphere. Adv. Atmos. Sci. 23, 775–783 (2006)

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Departamento de Electrónica, Computación y Control, Facultad de Ingeniería, Universidad Central de Venezuela, Caracas, 1051-DF, Venezuela

    William La Cruz

  2. Área de Matemática, Universidad Nacional Abierta, Caracas, Venezuela

    Gilberto Noguera

Authors
  1. William La Cruz
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Gilberto Noguera
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to William La Cruz.

Rights and permissions

Reprints and permissions

About this article

Cite this article

La Cruz, W., Noguera, G. Hybrid spectral gradient method for the unconstrained minimization problem. J Glob Optim 44, 193–212 (2009). https://doi.org/10.1007/s10898-008-9318-6

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10898-008-9318-6

Keywords

Mathematics Subject Classification (2000)

Search

Navigation