Accelerated convergence for the Powell/Hestenes multiplier method

Abstract

It is known that augmented Lagrangian or multiplier methods for solving constrained optimization problems can be interpreted as techniques for maximizing an augmented dual functionD _c(λ). For a constantc sufficiently large, by considering maximizing the augmented dual functionD _c(λ) with respect toλ, it is shown that the Newton iteration forλ based on maximizingD _c(λ) can be decomposed into taking a Powell/Hestenes iteration followed by a Newton-like correction. Superimposed on the original Powell/Hestenes method, a simple acceleration technique is devised to make use of information from the previous iteration. For problems with only one constraint, the acceleration technique is equivalent to replacing the second (Newton-like) part of the decomposition by a finite difference approximation. Numerical results are presented.