Abstract.
The Kantorovich Theorem is a fundamental tool in nonlinear analysis which has been extensively used in classical numerical analysis. In this paper we show that it can also be used in analyzing interior point methods. We obtain optimal bounds for Newton’s method when relied upon in a path following algorithm for linear complementarity problems.
Given a point z that approximates a point z(τ) on the central path with complementarity gap τ, a parameter θ ∈ (0,1) is determined for which the point z satisfies the hypothesis of the affine invariant form of the Kantorovich Theorem, with regards to the equation defining z((1−θ)τ). The resulting iterative algorithm produces a point with complementarity gap less than ε in at most Newton steps, or simplified Newton steps, where ε0 is the complementarity gap of the starting point and n is the dimension of the problem. Thus we recover the best complexity results known to date and, in addition, we obtain the best bounds for Newton’s method in this context.
Similar content being viewed by others
References
Alefeld, G.E., Potra, F.A., Shen, Z.: On the existence theorems of Kantorovich, Moore and Miranda. Computing Supplementum 15, 21–28 (2001)
Amer, S.M.: On solution of nonlinear singular integral equations with shift in generalized Hölder space. Chaos Solitons Fractals 12 (7), 1323–1334 (2001)
Begehr, H., Efendiev, M.A.: On the asymptotics of meromorphic solutions for nonlinear Riemann-Hilbert problems. Math. Proc. Cambridge Philos. Soc. 127 (1), 159–172 (1999)
Cottle, R.W., Pang, J.-S., Stone, R.E.: m The Linear Complementarity Problem. Academic Press, Boston, MA, 1992
De Pascale, E., Zabrejko, P.P.: New convergence criteria for the Newton-Kantorovich method and some applications to nonlinear integral equations. Rend. Sem. Mat. Univ. Padova 100, 211–230 (1998)
Dennis, J.E. Jr.: On the Kantorovich hypothesis for Newton’s method. SIAM J. Numer. Anal. 6, 493–507 (1969)
Dennis, J.E. Jr.: On the convergence of Newton-like methods. In: Numerical methods for nonlinear algebraic equations (Proc. Conf., Univ. Essex, Colchester, 1969), pp. 163–181. Gordon and Breach, London, 1970
Deuflhard, P., Heindl, G.: Affine invariant convergence theorems for Newton’s method and extensions to related methods. SIAM J. Numer. Anal. 16, 1–10 (1979)
Ezquerro, J.A., Hernández, M.A.: Multipoint super-Halley type approximation algorithms in Banach spaces. Numer. Funct. Anal. Optim., 21 (7-8), 845–858 (2000)
Ezquerro, J.A., Hernández, M.A.: A special type of Hammerstein integral equations. Int. Math. J. 1 (6), 557–566 (2002)
Ferreira, O.P., Svaiter, B.F.: Kantorovich’s theorem on Newton’s method in Riemannian manifolds. J. Complexity 18 (1), 304–329 (2002)
Gragg, W.B., Tapia, R.A.: Optimal error bounds for the Newton-Kantorovich theorem. SIAM J. Numer. Anal. 11, 10–13 (1974)
Grigat, E., Sachs, G.: Predictor-corrector continuation method for optimal control problems. In: Variational calculus, optimal control and applications (Trassenheide, 1996), volume 124 of Internat. Ser. Numer. Math., pp. 223–232. Birkhäuser, Basel, 1998
Jansen, B., Roos, C., Terlaky, T., Vial, J.-P.: Primal-dual algorithms for linear programming based on the logarithmic barrier method. J. Optim. Theory Appl. 83 (1), 1–26 (1994)
Jansen, B., Roos, C., Terlaky, T., Vial, J.-Ph.: Primal-dual target-following algorithms for linear programming. Ann. Oper. Res. 62, 197–231 (1996) Interior point methods in mathematical programming
Kantorovich, L.: On Newton’s method for functional equations (Russian). Dokl. Akad. Nauk. SSSR 59, 1237–1240 (1948)
Kantorovich, L.V., Akilov, G.P.: Functional analysis in normed spaces, volume 46 of International Series of Monographs in Pure and Applied Mathematics. Pergamon Press, Oxford, 1964. Translated from the Russian by D. E. Brown and Dr. A. P. Robertson
Kojima, M., Megiddo, N., Noma, T., Yoshise, A.: A unified approach to interior point algorithms for linear complementarity problems, volume 538 of Lecture Notes in Comput. Sci. Springer-Verlag, New York, 1991
Laumen, M.: A Kantorovich theorem for the structured PSB update in Hilbert space. J. Optim. Theory Appl. 105 (2), 391–415 (2000)
Matveev, A.F., Yunganns, P.: On the construction of an approximate solution of a nonlinear integral equation of permeable profile. Differ. Uravn. 33 (9), 1242–1252, 1295 (1997)
Mayer, J.: A generalized theorem of Miranda and the theorem of Newton-Kantorovich. Numer. Funct. Anal. Optim. 23(3-4), 333–357 (2002)
Miel, G.: An updated version of the Kantorovich theorem for Newton’s method. Computing 27, 237 (1981)
Miel, G.J.: The Kantorovich theorem with optimal error bounds. American Mathematical Monthly 86 (3), 212–215 (1979)
Moret, I.: A note on Newton type iterative methods. Computing 33, 65–73 (1984)
Moret, I.: On a general iterative scheme for Newton-type methods. Numerical Functional Analysis and Optimization 9, 1115–1137 (1987)
Moret, I.: A Kantorovich-type theorem for inexact Newton methods. Numer. Funct. Anal. Optim., 10 (3-4), 351–365 (1989)
Mukaidani, H., Shimomura, T., Mizukami, K.: Asymptotic expansions and a new numerical algorithm of the algebraic Riccati equation for multiparameter singularly perturbed systems. J. Math. Anal. Appl. 267 (1), 209–234 (2002)
Nagatou, K., Yamamoto, N., Nakao, M.T.: An approach to the numerical verification of solutions for nonlinear elliptic problems with local uniqueness. Numer. Funct. Anal. Optim. 20 (5-6), 543–565 (1999)
Nashed, M.Z., Chen, X.: Convergence of Newton-like methods for singular operator equations using outer inverses. Numer. Math. 66 (2), 235–257 (1993)
Nayakkankuppam, M.V., Overton, M.L.: Conditioning of semidefinite programs. Math. Program. 85 (3, Ser. A), 525–540 (1999)
Nesterov, Y., Nemirovsky, A.: Interior Point Polynomial Methods in Convex Programming. SIAM Publications. SIAM, Philadelphia, 1994
Ortega, J.M.: The Newton-Kantorovich theorem. Amer. Math. Monthly 75, 658–660 (1968)
Ortega, J.M., Rheinboldt, W.C.: Iterative Solution of Nonlinear Equations in Several Variables. Academic Press, New York, 1970
Ostrowski, A.M.: Solution of Equations in Euclidean and Banach Spaces. Academic Press, New York, 1973
Potra, F.A.: An error analysis for the secant method. Numer. Math. 38, 427–445 (1982)
Potra, F.A.: On the aposteriori error estimates for Newton’s method. Beitraege Numer. Math. 12, 125–138 (1984)
Potra, F.A.: Sharp error bounds for a class of Newton–like methods. Libertas Mathematica 5, 71–84 (1985)
Potra, F.A.: A path-following method for linear complementarity problems based on the affine invariant Kantorovich theorem. ZIB-Report 00-30, Konrad-Zuse-Zentrum, Berlin, August 2000
Potra, F.A., Pták, V.: Sharp error bounds for Newton’s process. Numer. Math. 34, 63–72 (1980)
Potra, F.A., Pták, V.: Nondiscrete induction and iterative processes. Number 103 in Research Notes in Mathematics. John Wiley & Sons, Boston–London–Melbourne, 1984
Potra, F.A., Wright, S.J.: Interior-point methods. Journal of Computational and Applied Mathematics 124, 281–302 (2000)
Rall, L.B.: Computational solution of nonlinear operator equations. Krieger Huntington, New York, 1979
Renegar, J.: A polynomial-time algorithm, based on Newton’s method, for linear programming. Math. Programming 40 (1,Ser. A), 59–93 (1988)
Renegar, J.: A mathematical view of interior-point methods in convex optimization. MPS/SIAM Series on Optimization. Society for Industrial and Applied Mathematics (SIAM), Philadelphia, PA, 2001
Renegar, J., Shub, M.: Unified complexity analysis for Newton LP methods. Math. Programming 53 (1, Ser. A), 1–16 (1992)
Roos, C., Vial, J.-Ph., Terlaky, T.: Theory and algorithms for linear optimization: an interior point approach. Wiley-Interscience Series in Discrete Mathematics and Optimization. John Wiley and Sons, 1997
Smale, S.: Newton’s method estimates from data at one point. In: The merging of disciplines: new directions in pure, applied, and computational mathematics (Laramie, Wyo., 1985), pp. 185–196. Springer, New York, 1986
Stoffer, D., Palmer, K.J.: Rigorous verification of chaotic behaviour of maps using validated shadowing. Nonlinearity 12 (6), 1683–1698 (1999)
Tapia, R.A.: The Kantorovich theorem for Newton’s method. American Mathematical Monthly 78, 389–392 (1971)
Tsuchiya, T.: An application of the Kantorovich theorem to nonlinear finite element analysis. Numer. Math. 84 (1), 121–141 (1999)
Wang, X., Li, C., Lai, M.-J.: A unified convergence theory for Newton-type methods for zeros of nonlinear operators in Banach spaces. BIT 42 (1), 206–213 (2002)
Wright, S.J.: Primal–Dual Interior–Point Methods. SIAM Publications, Philadephia, 1997
Yamamoto, T.: A unified derivation of several error bounds for Newton’s process. Journal of Computation and Applied Mathematics 12, 179–191 (1985)
Yamamoto, T.: Error-bounds for Newton iterates derived from the Kantorovich-theorem. Numer. Math. 48, 91–98 (1986)
Yamamoto, T.: A method for finding sharp error-bounds for Newton method under the Kantorivich assumptions. Numer. Math. 49, 203–220 (1986)
Yamamoto, T.: Historical developments in convergence analysis for Newton’s and Newton-like methods. J. Comput. Appl. Math. 124 (1-2), 1–23 (2000), Numerical analysis 2000, Vol. IV, Optimization and nonlinear equations
Ye, Y.: Interior Point Algorithms : Theory and Analysis. Wiley-Interscience Series in Discrete Mathematics and Optimization. John Wiley and Sons, 1997
Author information
Authors and Affiliations
Corresponding author
Additional information
Mathematics Subject Classification (2000): 49M15, 65H10, 65K05, 90C33
Work supported by the National Science Foundation under Grants No. 9996154, 0139701
Acknowledgement The original version of this paper [38] was written when the author was visiting the Konrad-Zuse-Zentrum für Informationstechnik Berlin (ZIB). The author would like to thank Peter Deuflhard, President of ZIB, for the excellent working conditions at ZIB, and for very interesting discussions on various mathematical problems related to the subject of the present paper. The author would also like to thank two anonymous referees whose comments and suggestions led to a better presentation of our results.
Rights and permissions
About this article
Cite this article
Potra, F. The Kantorovich Theorem and interior point methods. Math. Program. 102, 47–70 (2005). https://doi.org/10.1007/s10107-003-0501-8
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10107-003-0501-8