Skip to main content
SpringerLink
Log in

Relaxed Steepest Descent and Cauchy-Barzilai-Borwein Method

  • Published:
Computational Optimization and Applications Aims and scope Submit manuscript

Abstract

The negative gradient direction to find local minimizers has been associated with the classical steepest descent method which behaves poorly except for very well conditioned problems. We stress out that the poor behavior of the steepest descent methods is due to the optimal Cauchy choice of steplength and not to the choice of the search direction. We discuss over and under relaxation of the optimal steplength. In fact, we study and extend recent nonmonotone choices of steplength that significantly enhance the behavior of the method. For a new particular case (Cauchy-Barzilai-Borwein method), we present a convergence analysis and encouraging numerical results to illustrate the advantages of using nonmonotone overrelaxations of the gradient method.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Similar content being viewed by others

References

  1. H. Akaike, “On a successive transformation of probability distribution and its application to the analysis of the optimum gradient method,” Ann. Inst. Statist. Math. Tokyo, vol. 11, pp. 1–16, 1959.

    Google Scholar 

  2. J. Barzilai and J.M. Borwein, “Two point step size gradient methods,” IMA J. Numer. Anal., vol. 8, pp. 141–148, 1988.

    Google Scholar 

  3. C. Brezinski, Padé—Type Approximation and General Orthogonal Polynomials, Birkhauser—Verlag: Basel, 1980.

    Google Scholar 

  4. A. Cauchy, “Méthodes générales pour la résolution des systèmes d'équations simultanées,” C. R. Acad. Sci. Par., vol. 25, pp. 536–538, 1847.

    Google Scholar 

  5. Y.H. Dai and L.Z. Liao, “R-linear convergence of the Barzilai and Borwein gradient method,” Technical Report AMSS 1999-081, Academy of Mathematics and Systems Sciences, Beijing, China, 1999.

    Google Scholar 

  6. A. Friedlander, J.M. Martinez, B. Molina, and M. Raydan, “Gradient method with retards and generalizations,” SIAM J. Numer. Anal., vol. 36, pp. 275–289, 1999.

    Google Scholar 

  7. W. Glunt, T.L. Hayden, and M. Raydan, “Molecular conformations from distance matrices,” J. Comp. Chem., vol. 14, pp. 114–120, 1993.

    Google Scholar 

  8. D. Luenberger, Linear and Nonlinear Programming, Addison-Wesley: Menlo Park, CA, 1984.

    Google Scholar 

  9. M. Raydan, “On the Barzilai and Borwein choice of steplength for the gradient method,” IMA J. Numer. Anal., vol. 13, pp. 321–326, 1993.

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Dpto. de Computación, Facultad de Ciencias, Universidad Central de Venezuela, Ap. 47002, Caracas, 1041-A, Venezuela

    Marcos Raydan

  2. Instituto de Matemática Pura e Aplicada, Estrada dona Castorina 110, Rio de Janeiro, RJ CEP, 22460-320, Brazil

    Benar F. Svaiter

Authors
  1. Marcos Raydan
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Benar F. Svaiter
    View author publications

    You can also search for this author in PubMed Google Scholar

Rights and permissions

Reprints and permissions

About this article

Cite this article

Raydan, M., Svaiter, B.F. Relaxed Steepest Descent and Cauchy-Barzilai-Borwein Method. Computational Optimization and Applications 21, 155–167 (2002). https://doi.org/10.1023/A:1013708715892

Download citation

  • Issue Date:

  • DOI: https://doi.org/10.1023/A:1013708715892

Search

Navigation