Abstract
The spectral gradient method is a nonmonotone gradient method for large-scale unconstrained minimization. We strengthen the algorithm by modifications which globalize the method and present strategies to apply preconditioning techniques. The modified algorithm replaces a condition of uniform positive definitness of the preconditioning matrices, with mild conditions on the search directions. The result is a robust algorithm which is effective on very large problems. Encouraging numerical experiments are presented for a variety of standard test problems, for solving nonlinear Poisson-type equations, an also for finding molecular conformations by distance geometry.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
B.M. Averick and J.M. Ortega, Solution of nonlinear Poisson-type equations, Appl. Numer. Math. 8(1991) 443–455.
B.M. Averick and J.M. Ortega, Fast solution of nonlinear Poisson-type equations, SIAM J. Sci. Comput. 14 (1993) 44–48.
O. Axelsson, A survey of preconditioned iterative methods for linear systems of algebraic equations, Bit 25 (1985) 166–187.
O. Axelsson and V.A. Barker, Finite Element Solution of Boundary Value Problems, Theory and Computation (Academic Press, New York, 1984).
H.E. Bailey and R.M. Beam, Newton's method applied to finite difference approximations for the steady-state compressible Navier-Stokes equations, J. Comput. Phys. 93 (1991) 108–127.
J. Barzilai and J.M. Borwein, Two point step size gradient methods, IMA J. Numer. Anal. 8 (1988) 141–148.
E.G. Birgin, R. Biloti, M. Tygel and L.T. Santos, Restricted optimization: A clue to fast and accurate implementation of the common reflection surface method, J. Appl. Geophys. 42 (1999) 143–155.
E.G. Birgin, I. Chambouleyron and J.M. Martínez, Estimation of the optical constants and the thickness of thin films using unconstrained optimization, J. Comput. Phys. 151 (1999) 862–880.
E.G. Birgin and Y.G. Evtushenko, Automatic differentiation and spectral projected gradient methods for optimal control problems, Optim. Methods Software 10 (1998) 125–146.
E.G. Birgin and J.M. Martínez, A spectral conjugate gradient method for unconstrained optimization, Appl. Math. Optim. 43 (2001) 117–128.
I. Borg and P.J.F. Groenen, Modern Multidimensional Scaling: Theory and Applications (Springer, Berlin, 1997).
Z. Castillo, D. Cores and M. Raydan, Low cost optimization techniques for solving the nonlinear seismic reflection tomography problem, Optim. Engrg. 1 (2000) 155–169.
D. Cores, G. Fung and R. Michelena, A fast and global two point low storage optimization technique for tracing rays in 2D and 3D isotropic media, J. Appl. Geophys. 45 (2000) 273–287.
W. Glunt, T.L. Hayden and M. Raydan, Molecular conformations from distance matrices, J. Comput. Chem. 14 (1993) 114–120.
W. Glunt, T.L. Hayden and M. Raydan, Preconditioners for distance matrix algorithms, J. Comput. Chem. 15 (1994) 227–232.
L. Grippo, F. Lampariello and S. Lucidi, A nonmonotone line search technique for Newton's method, SIAM J. Numer. Anal. 23 (19) 707–716, 1986.
P.J.F. Groenen, The Majorization Approach to Multidimensional Scaling: Some Problems and Extensions (DSWO Press, Univ. of Leiden, 1993).
P.J.F. Groenen, W. Glunt and T.L. Hayden, Fast algorithms for multidimensional scaling: A comparison of majorization and spectral gradient methods, Technical Report, Data Theory Department, University of Leiden, Leiden, Netherlands (1996).
P.J.F. Groenen and W.J. Heiser, The tunneling method for global optimization in multidimensional scaling, Psychometrika 61 (1996) 529–550.
P.J.F. Groenen, W.J. Heiser and J.J. Meulman, Global optimization in least-squares multidimensional scaling by distance smoothing, J. Classification 16 (1999) 225–254.
A.J. Kearsley, R.A. Tapia and M.W. Trosset, The solution of the metric stress and stress problems in multidimensional scaling using Newton's method, Comput. Statist. 13(3) (1998) 369–396.
B. Molina and M. Raydan, The preconditioned Barzilai-Borwein method for the numerical solution of partial differential equations, Numer. Algorithms 13 (1996) 45–60.
M. Mulato, I. Chambouleyron, E.G. Birgin and J.M. Martínez, Determination of thickness and optical constants of a-Si:H films from transmittance data, Appl. Phys. Lett. 77 (2000) 2133–2135.
J. Nocedal, Conjugate gradient methods and nonlinear optimization, in: Linear and Nonlinear Conjugate Gradient Related Methods, eds. L. Adams and J.L. Nazareth, AMS-IMS-SIAM Joint Summer Research Conference (SIAM, Philadelphia, PA, 1996) pp. 9–23.
M. Raydan, On the Barzilai and Borwein choice of steplength for the gradient method, IMA J. Numer. Anal. 13 (1993) 321–326.
M. Raydan, The Barzilai and Borwein gradient method for the large scale unconstrained minimization problem, SIAM J. Optim. 7 (1997) 26–33.
D.F. Shanno, On preconditioned conjugate gradient methods, in: Nonlinear Programming, Vol. 4, eds. R.R. Meyer, O.L. Mangasarian and S.M. Robinson (Academic Press, New York, 1981) pp. 201–221.
M.W. Trosset, Numerical algorithms for MDS, in: Classification and Knowledge Organization, eds. R. Klar and P. Opitz (Springer, Berlin, 1997) pp. 80–92.
C.Wells, W. Glunt and T.L. Hayden, Searching conformational space with the spectral distance geometry algorithm, J. Molecular Structure (Theochem) 308 (1994) 263–271.
Author information
Authors and Affiliations
Rights and permissions
About this article
Cite this article
Luengo, F., Raydan, M., Glunt, W. et al. Preconditioned Spectral Gradient Method. Numerical Algorithms 30, 241–258 (2002). https://doi.org/10.1023/A:1020181927999
Issue Date:
DOI: https://doi.org/10.1023/A:1020181927999