Abstract
This paper considers local convergence and rate of convergence results for algorithms for minimizing the composite functionF(x)=f(x)+h(c(x)) wheref andc are smooth buth(c) may be nonsmooth. Local convergence at a second order rate is established for the generalized Gauss—Newton method whenh is convex and globally Lipschitz and the minimizer is strongly unique. Local convergence at a second order rate is established for a generalized Newton method when the minimizer satisfies nondegeneracy, strict complementarity and second order sufficiency conditions. Assuming the minimizer satisfies these conditions, necessary and sufficient conditions for a superlinear rate of convergence for curvature approximating methods are established. Necessary and sufficient conditions for a two-step superlinear rate of convergence are also established when only reduced curvature information is available. All these local convergence and rate of convergence results are directly applicable to nonlinearing programming problems.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
P.T. Boggs, J.W. Tolle and P. Wang, “On the local convergence of quasi-Newton methods for constrained optimization”,SIAM Journal on Control and Optimization 20 (1982) 161–171.
R.M. Chamberlain, M.J.D. Powell, C. Lemaréchal and H.C. Pederson, “The watchdog technique in forcing convergence in algorithms for constrained optimization”,Mathematical Programming Study 16 (1982) 1–17.
T.F. Coleman and A.R. Conn, “Nonlinear programming via an exact penalty function: Asymptotic analysis”,Mathematical Programming 24 (1982) 123–136.
T.F. Coleman and D.C. Sorensen, “A note on the computation of an orthonormal basis for the null space of a matrix”,Mathematical Programming 29 (1984) 234–242.
L. Cromme, “Strong uniqueness. A far reaching criterion for the convergence analysis of iterative procedures”,Numerische Mathematik 29 (1978) 179–194.
J.E. Dennis and J.J. Moré, “A characterization of superlinear convergence and its application to quasi-Newton methods”,Mathematics of Computation 28 (1974) 549–560.
R. Fletcher,Practical methods of optimization, volume 2: Constrained optimization (Wiley, Chichester and New York, 1981).
R. Fletcher, “Numerical experiments with an exactl 1 penalty function method”, in: O.L. Mangasarian, R.R. Meyer and S.M. Robinson, eds.,Nonlinear programming 4 (Academic Press, New York and London, 1981).
R. Fletcher, “Second order corrections for nondifferentiable optimization”, in: G.A. Watson, ed.,Numerical analysis proceedings, Dundee 1981, Lecture Notes in Mathematics 912 (Springer-Verlag, 1982).
S.P. Han, “Superlinear convergence of a minimax method”, Report TR-78-336, Department of Computer Science, Cornell University (1978).
K. Jittorntrum and M.R. Osborne, “Strong uniqueness and second order convergence in nonlinear discrete approximation”,Numerische Mathematik 34 (1980) 439–455.
D.Q. Mayne, “On the use of exact penalty functions to determine step length in optimization algorithms”, in: G.A. Watson, ed.,Numerical analysis, Dundee 1979, Lecture notes in mathematics 773 (Springer-Verlag, Berlin, 1980).
W. Murray and M.L. Overton, “A projected Lagrangian algorithm for nonlinear minimax optimization”,SIAM Journal on Scientific and Statistical Computing 1 (1980) 345–370.
W. Murray and M.L. Overton, “A projected Lagrangian algorithm for nonlinearL 1 optimization”,SIAM Journal on Scientific and Statistical Computing 2 (1981) 207–224.
M.R. Osborne, “Algorithms for nonlinear discrete approximation”, in: C.T.H. Baker and C. Phillips, eds.,The numerical solution of nonlinear problems (Clarendon Press, Oxford, 1981).
M.R. Osborne, “Strong uniqueness in nonlinear approximation”, in: C.T.H. Baker and C. Phillips, eds.,The numerical solution of nonlinear problems (Clarendon Press, Oxford, 1981).
M.R. Osborne,Finite algorithms in optimization and data analysis (Wiley, in press).
M.J.D. Powell, “The convergence of variable metric methods for nonlinearly constrained optimization calculations”, in: O.L. Mangasarian, R.R. Meyer and S.M. Robinson, eds.,Nonlinear Programming 3 (Academic Press, New York, 1978).
M.J.D. Powell, “General algorithms for discrete nonlinear approximation calculations”, Report DAMTP 1982/NA5, University of Cambridge (to be published by Academic Press in Proceedings of the fourth Texas symposium on approximation theory, College Station, TX, 1983).
M.J.D. Powell and Y. Yuan, “Conditions for superlinear convergence inl 1 andl ∞ solutions of overdetermined nonlinear equations”, IMA Journal on Numerical Analysis 4 (1984) 241–251.
R.S. Womersley, “Minimizing nonsmooth composite functions”, Research Report No. 13-1984, Mathematical Sciences Research Centre, Australian National University (Canberra, Australia, 1984).
R.S. Womersley and R. Fletcher, “An algorithm for composite nonsmooth optimization problems”, Report NA/60, Department of Mathematics, University of Dundee (Dundee, Scotland, 1982).
Y. Yuan, “Global convergence of trust region algorithms for nonsmooth optimization”, Report DAMTP 1983/NA13, University of Cambridge (1983).
Y. Yuan, “An example of only linear convergence of trust region algorithms for nonsmooth optimization”,IMA Journal of Numerical Analysis 4 (1984) 327–335.
Author information
Authors and Affiliations
Additional information
This work was done while the author was a Research fellow at the Mathematical Sciences Research Centre, Australian National University.
Rights and permissions
About this article
Cite this article
Womersley, R.S. Local properties of algorithms for minimizing nonsmooth composite functions. Mathematical Programming 32, 69–89 (1985). https://doi.org/10.1007/BF01585659
Received:
Revised:
Issue Date:
DOI: https://doi.org/10.1007/BF01585659
Key words
- Composite Functions
- Nonsmooth Optimization
- Structure Functionals
- Superlinear Convergence
- Second Order Convergence
- Strong Uniqueness
- Reduced Curvature