Abstract
We develop an affine-scaling algorithm for box-constrained optimization which has the property that each iterate is a scaled cyclic Barzilai–Borwein (CBB) gradient iterate that lies in the interior of the feasible set. Global convergence is established for a nonmonotone line search, while there is local R-linear convergence at a nondegenerate local minimizer where the second-order sufficient optimality conditions are satisfied. Numerical experiments show that the convergence speed is insensitive to problem conditioning. The algorithm is particularly well suited for image restoration problems which arise in positron emission tomography where the cost function can be infinite on the boundary of the feasible set.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Akaike H. (1959). On a successive transformation of probability distribution and its application to the analysis of the optimum gradient method. Ann. Inst. Stat. Math. Tokyo 11: 1–17
Barzilai J. and Borwein J.M. (1988). Two point step size gradient methods. IMA J. Numer. Anal. 8: 141–148
Birgin E.G., Biloti R., Tygel M. and Santos L.T. (1999). Restricted optimization: a clue to a fast and accurate implementation of the common reflection surface stack method. J. Appl. Geophys. 42: 143–155
Birgin E.G., Chambouleyron I. and Martínez J.M. (1999). Estimation of the optical constants and the thickness of thin films using unconstrained optimization. J. Comput. Phys. 151: 862–880
Birgin E.G., Martínez J.M. and Raydan M. (2000). Nonmonotone spectral projected gradient methods for convex sets. SIAM J. Optim. 10: 1196–1211
Chang, J.-H., Anderson, J.M.M., Mair, B.A.: An accelerated penalized maximum likelihood algorithm for positron emission tomography. IEEE Trans. Nucl. Sci. 54(5): 1648–1659
Chang J.-H., Anderson J.M.M. and Votaw J.R. (2004). Regularized image reconstruction algorithms for positron emission tomography. IEEE Trans. Med. Imag. 23: 1165–1195
Coleman T.F. and Li Y. (1994). On the convergence of interior-reflective Newton methods for nonlinear minimization subject to bounds. Math. Prog. 67: 189–224
Coleman T.F. and Li Y. (1996). An interior trust region approach for nonlinear minimization subject to bounds. SIAM J. Optim. 6: 418–445
Coleman T.F. and Li Y. (2000). A trust region and affine scaling interior point method for nonconvex minimization with linear inequality constraints. Math. Program. 88: 1–31
Dai Y.H. (2003). Alternate stepsize gradient method. Optimization 52: 395–415
Dai Y.H. and Fletcher R. (2005). Projected Barzilai–Borwein methods for large-scale box-constrained quadratic programming. Numer. Math. 100: 21–47
Dai Y.H., Hager W.W., Schittkowski K. and Zhang H. (2006). The cyclic Barzilai–Borwein method for unconstrained optimization. IMA J. Numer. Anal. 26: 604–627
Dai Y.H. and Zhang H. (2001). An adaptive two-point stepsize gradient algorithm. Numer. Algorithms 27: 377–385
Dikin I.I. (1967). Iterative solution of problems of linear and quadratic programming. Sov. Math. Dokl. 8: 674–675
Dostál Z. (1997). Box constrained quadratic programming with proportioning and projections. SIAM J. Optim. 7: 871–887
Fletcher, R.: On the Barzilai–Borwein method. Technical Report, Department of Mathematics, University of Dundee, Dundee (2001)
Friedlander A., Martínez J.M., Molina B. and Raydan M. (1999). Gradient method with retards and generalizations. SIAM J. Numer. Anal. 36: 275–289
Glunt W., Hayden T.L. and Raydan M. (1993). Molecular conformations from distance matrices. J. Comput. Chem. 14: 114–120
Hager W.W. and Zhang H. (2006). A new active set algorithm for box constrained optimization. SIAM J. Optim. 17: 526–557
Heinkenschloss M., Ulbrich M. and Ulbrich S. (1999). Superlinear and quadratic convergence of affine-scaling interior-point Newton methods for problems with simple bounds without strict complementarity assumption. Math. Program. 86: 615–635
Johnson C.A., Seidel J. and Sofer A. (2000). Interior-point methodology for 3-D PET reconstruction. IEEE Trans. Med. Imag. 19: 271–285
Kanzow C. and Klug A. (2006). On affine-scaling interior-point Newton methods for nonlinear minimization with bound constraints. Comput. Optim. Appl. 35: 177–197
Kaufman L. (1987). Implementing and accelerating the EM algorithm for positron emission tomography. IEEE Trans. Med. Imag. 6: 37–51
Kaufman L. (1993). Maximum likelihood, least squares and penalized least squares for PET. IEEE Trans. Med. Imag. 12: 200–214
Raydan M. (1997). The Barzilai and Borwein gradient method for the large scale unconstrained minimization problem. SIAM J. Optim. 7: 26–33
Zhang H. and Hager W.W. (2004). A nonmonotone line search technique and its application to unconstrained optimization. SIAM J. Optim. 14: 1043–1056
Zhang H. and Hager W.W. (2005). PACBB: a projected adaptive cyclic Barzilai–Borwein method for box constrained optimization. In: Hager, W.W., Huang, S.-J., Pardalos, P.M. and Prokopyev, O.A. (eds) Multiscale Optimization Methods and Applications, pp 387–392. Springer, New York
Zhang, Y.: Interior-point gradient methods with diagonal-scalings for simple-bound constrained optimization. Technical Report, TR04-06, Department of Computational and Applied Mathematics. Rice University, Houston (2004)
Author information
Authors and Affiliations
Corresponding author
Additional information
This material is based upon work supported by the National Science Foundation under Grants 0203270, 0619080, and 0620286.
Rights and permissions
About this article
Cite this article
Hager, W.W., Mair, B.A. & Zhang, H. An affine-scaling interior-point CBB method for box-constrained optimization. Math. Program. 119, 1–32 (2009). https://doi.org/10.1007/s10107-007-0199-0
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10107-007-0199-0
Keywords
- Interior-point
- Affine-scaling
- Cyclic Barzilai–Borwein methods
- CBB
- PET
- Image reconstruction
- Global convergence
- Local convergence