Skip to main content
Log in

Complexity and algorithms for nonlinear optimization problems

  • Published:
Annals of Operations Research Aims and scope Submit manuscript

Abstract

Nonlinear optimization algorithms are rarely discussed from a complexity point of view. Even the concept of solving nonlinear problems on digital computers is not well defined. The focus here is on a complexity approach for designing and analyzing algorithms for nonlinear optimization problems providing optimal solutions with prespecified accuracy in the solution space. We delineate the complexity status of convex problems over network constraints, dual of flow constraints, dual of multi-commodity, constraints defined by a submodular rank function (a generalized allocation problem), tree networks, diagonal dominant matrices, and nonlinear knapsack problem’s constraint. All these problems, except for the latter in integers, have polynomial time algorithms which may be viewed within a unifying framework of a proximity-scaling technique or a threshold technique. The complexity of many of these algorithms is furthermore best possible in that it matches lower bounds on the complexity of the respective problems.

In general nonseparable optimization problems are shown to be considerably more difficult than separable problems. We compare the complexity of continuous versus discrete nonlinear problems and list some major open problems in the area of nonlinear optimization.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  • Ahuja, R. K., & Orlin, J. B. (2001b). Inverse optimization. Operations Research, 49, 771–783.

    Article  Google Scholar 

  • Ahuja, R. K., Batra, J. L., & Gupta, S. K. (1984). A parametric algorithm for the convex cost network flow and related problems. European Journal of Operational Research, 16, 222–235.

    Article  Google Scholar 

  • Ahuja, R. K., Hochbaum, D. S., & Orlin, J. B. (2003). Solving the convex cost integer dual network flow problem. Management Science, 49, 950–964.

    Article  Google Scholar 

  • Ahuja, R. K., Hochbaum, D. S., & Orlin, J. B. (2004). A cut based algorithm for the nonlinear dual of the minimum cost network flow problem. Algorithmica, 39, 189–208.

    Article  Google Scholar 

  • Ahuja, R. K., Magnanti, T. L., & Orlin, J. B. (1993). Network flows: Theory, algorithms and applications. New Jersey: Prentice Hall.

    Google Scholar 

  • Baldick, R. (1991). A unification of polynomially solvable cases of integer ‘non-separable’ quadratic optimization. Lawrence Berkeley Laboratory manuscript.

  • Baldick, R., & Wu, F. F. (1990). Efficient integer optimization algorithms for optimal coordination of capacitors and regulators. IEEE Transactions on Power Systems, 5, 805–812.

    Article  Google Scholar 

  • Barahona, F. (1986). A solvable case of quadratic 0-1 programming. Discrete Applied Mathematics, 13, 23–28.

    Article  Google Scholar 

  • Barlow, R. E., Bartholomew, D. J., Bremer, J. M., & Brunk, H. D. (1972). Statistical inference under order restrictions. New York: Wiley.

    Google Scholar 

  • Blum, M., Floyd, R. W., Pratt, V. R., Rivest, R. L., & Tarjan, R. E. (1972). Time bounds for selection. Journal of Computer Systems Science, 7, 448–461.

    Article  Google Scholar 

  • Brucker, P. (1984). An O(n) algorithm for quadratic knapsack problems. Operations Research Letters, 3, 163–166.

    Article  Google Scholar 

  • Burton, D., & Toint, Ph. L. (1992). On an instance of the inverse shortest paths problem. Mathematical Programming, 53, 45–61.

    Article  Google Scholar 

  • Burton, D., & Toint, Ph. L. (1994). On the use of an inverse shortest paths algorithm for recovering linearly correlated costs. Mathematical Programming, 63, 1–22.

    Article  Google Scholar 

  • Busacker, R. G., & Gowen, P. J. (1961) A procedure for determining minimal-cost network flow patterns. Operational Research Office, John Hopkins University, Baltimore, MD.

  • Cosares, S., & Hochbaum, D. S. (1994). A strongly polynomial algorithm for the quadratic transportation problem with fixed number of suppliers. Mathematics of Operations Research, 19, 94–111.

    Google Scholar 

  • Dantzig, G. B. (1963). Linear programming and extensions. New Jersey: Princeton University Press.

    Google Scholar 

  • Dennis, J. B. (1959). Mathematical programming and electrical networks. In Technology press research monographs (pp. 74–75). New York: Technology Press and Wiley.

    Google Scholar 

  • Edmonds, J., & Karp, R. M. (1972). Theoretical improvements in algorithmic efficiency for network flow problems. Journal of ACM, 19, 248–264.

    Article  Google Scholar 

  • Erickson, R. E., Monma, C. L., & Veinott, A. F. (1987). Send-and-split method for minimum-concave-cost network flows. Mathematics of Operations Research, 12, 634–664.

    Google Scholar 

  • Federgruen, A., & Groenevelt, H. (1986a). The greedy procedure for resource allocation problems: Necessary and sufficient conditions for optimality. Operations Research, 34, 909–918.

    Google Scholar 

  • Federgruen, A., & Groenevelt, H. (1986b). Optimal flows in networks with multiple sources and sinks, with applications to oil and gas lease investment programs. Operations Research, 34, 218–225.

    Google Scholar 

  • Frederickson, G. N., & Johnson, D. B. (1982). The complexity of selection and rankings in X+Y and matrices with sorted columns. Journal of Computing System Science, 24, 197–208.

    Article  Google Scholar 

  • Fourer, R. (1988). A simplex algorithm for piecewise-linear programming: Finiteness, feasibility and degeneracy. Mathematical Programming, 41, 281–316.

    Article  Google Scholar 

  • Gale, D. (1957). A theorem of flows in networks. Pacific Journal of Mathematics, 7, 1073–1082.

    Google Scholar 

  • Gallo, G., Grigoriadis, M. D., & Tarjan, R. E. (1989). A fast parametric maximum flow algorithm and applications. SIAM Journal of Computing, 18, 30–55.

    Article  Google Scholar 

  • Garey, M., & Johnson, D. (1979). Computers and intractability, a guide to the theory of NP-completeness. New York: Freeman.

    Google Scholar 

  • Goldberg, A. V., & Tarjan, R. E. (1988). A new approach to the maximum flow problem. Journal of the ACM, 35, 921–940.

    Article  Google Scholar 

  • Granot, F., & Skorin-Kapov, J. (1990). Some proximity and sensitivity results in quadratic integer programming. Mathematical Programming, 47, 259–268.

    Article  Google Scholar 

  • Guisewite, G., & Pardalos, P. M. (1990). Minimum concave cost network flow problems: Applications, complexity, and algorithms. Annals of Operations Research, 25, 75–100.

    Article  Google Scholar 

  • Hansen, P., & Simeone, B. (1986). Unimodular functions. Discrete Applied Mathematics, 14, 269–281.

    Article  Google Scholar 

  • Hochbaum, D. S. (1989). On a polynomial class of nonlinear optimization problems. Manuscript, U.C. Berkeley.

  • Hochbaum, D. S. (1993). Polynomial algorithms for convex network optimization. In D. Du, M. Pardalos (Eds.), Network optimization problems: algorithms, complexity and applications (pp. 63–92). Singapore: World Scientific.

    Google Scholar 

  • Hochbaum, D. S. (1994). Lower and upper bounds for allocation problems. Mathematics of Operations Research, 19, 390–409.

    Google Scholar 

  • Hochbaum, D. S. (1995). A nonlinear knapsack problem. Operations Research Letters, 17, 103–110.

    Article  Google Scholar 

  • Hochbaum, D. S. (1998). The pseudoflow algorithm for the maximum flow problem. Manuscript, UC Berkeley, revised 2003. Extended abstract in: Boyd & Rios-Mercado (Eds.), Lecture notes in computer science: Vol. 1412. The pseudoflow algorithm and the pseudoflow-based simplex for the maximum flow problem. Proceedings of IPCO98 (pp. 325–337), Bixby, June 1998. New York: Springer.

  • Hochbaum, D. S. (2001). An efficient algorithm for image segmentation, Markov random fields and related problems. Journal of the ACM, 48, 686–701.

    Article  Google Scholar 

  • Hochbaum, D. S. (2002). The inverse shortest paths problem. Manuscript, UC Berkeley.

  • Hochbaum, D. S. (2003). Efficient algorithms for the inverse spanning tree problem. Operations Research, 51, 785–797.

    Article  Google Scholar 

  • Hochbaum, D. S. (2005). Complexity and algorithms for convex network optimization and other nonlinear problems. 4OR, 3, 171–216.

    Article  Google Scholar 

  • Hochbaum, D. S., & Hong, S. P. (1995). About strongly polynomial time algorithms for quadratic optimization over submodular constraints. Mathematical Programming, 69, 269–309.

    Google Scholar 

  • Hochbaum, D. S., & Hong, S. P. (1996). On the complexity of the production-transportation problem. SIAM Journal on Optimization, 6, 250–264.

    Article  Google Scholar 

  • Hochbaum, D. S., & Queyranne, M. (2003). The convex cost closure problem. SIAM Journal on Discrete Mathematics, 16, 192–207.

    Article  Google Scholar 

  • Hochbaum, D. S., & Seshadri, S. (1993). The empirical performance of a polynomial algorithm for constrained nonlinear optimization. Annals of Operations Research, 43, 229–248.

    Article  Google Scholar 

  • Hochbaum, D. S., & Shanthikumar, J. G. (1990). Convex separable optimization is not much harder than linear optimization. Journal of the ACM, 37, 843–862.

    Article  Google Scholar 

  • Hochbaum, D. S., Shamir, R., & Shanthikumar, J. G. (1992). A polynomial algorithm for an integer quadratic nonseparable transportation problem. Mathematical Programming, 55, 359–372.

    Article  Google Scholar 

  • Hoffman, A. J. (1960). Some recent applications of the theory of linear inequalities to extremal combinatorial analysis. In R. Bellman, M. Hall Jr. (Eds.), Proceedings of Symposia in Applied Mathematics: Vol. X. Combinatorial analysis (pp. 113–127). Providence: American mathematical Society.

    Google Scholar 

  • Ibaraki, T., & Katoh, N. (1988). Resource allocation problems: Algorithmic approaches. Boston: MIT.

    Google Scholar 

  • Iri, M. (1960). A new method of solving transportation network problems. Journal of the Operations Research Society of Japan, 3, 27–87.

    Google Scholar 

  • Jewell, W. S. (1958). Optimal flow through networks. Technical report No. 8, Operations research Center, MIT, Cambridge.

  • Kapoor, S., & Vaidya, P. M. (1986). Fast algorithms for convex quadratic programming and multicommodity flows. In Proceedings of the 18th symposium on theory of computing (pp. 147–159).

  • Karzanov, A. V., & McCormick, S. T. (1997). Polynomial methods for separable convex optimization in unimodular linear spaces with applications. SIAM Journal on Computing, 26, 1245–1275.

    Article  Google Scholar 

  • Knuth, D. (1973). The art of computer programming: Vol. 3. Sorting and searching. Reading: Addison Wesley.

    Google Scholar 

  • Kozlov, M. K., Tarasov, S. P., & Khachian, L. G. (1979). Polynomial solvability of convex quadratic programming. Doklady Akad. Nauk SSSR, 5, 1051–1053 (Translated in Soviet Mathematics Doklady 20 (1979), 1108–1111).

    Google Scholar 

  • Lawler, E. (1979). Fast approximation algorithms for knapsack problems. Mathematics of Operations Research, 4, 339–356.

    Google Scholar 

  • Mansour, Y., Schieber, B., & Tiwari, P. (1991). Lower bounds for computations with the floor operation. SIAM Journal on Computing, 20, 315–327.

    Article  Google Scholar 

  • Megiddo, N., & Tamir, A. (1993). Linear time algorithms for some separable quadratic programming problems. Operations Research Letters, 13, 203–211.

    Article  Google Scholar 

  • Minoux, M. (1984). A polynomial algorithm for minimum quadratic cost flow problems. European Journal of Operational Research, 18, 377–387.

    Article  Google Scholar 

  • Minoux, M. (1986). Solving integer minimum cost flows with separable convex cost objective polynomially. Mathematical Programming Study, 26, 237–239.

    Google Scholar 

  • Minoux, M. (1986). Mathematical programming, theory and algorithms. Wiley: New York, Chaps. 5, 6.

    Google Scholar 

  • Monteiro, R. D. C., & Adler (1989). Interior path following primal-dual algorithms. Part II: Convex quadratic programming. Mathematical Programming, 44, 43–66.

    Article  Google Scholar 

  • Moriguchi, S., & Shioura, A. (2004). On Hochbaum’s proximity-scaling algorithm for the general resource allocation problem. Mathematics of Operations Research, 29, 394–397.

    Article  Google Scholar 

  • Nemirovsky, A. S., & Yudin, D. B. (1983). Problem complexity and method efficiency in optimization. New York: Wiley.

    Google Scholar 

  • Papadimitiou, C. H., & Steiglitz, K. (1982). Combinatorial optimization: algorithms and complexity. New Jersey: Prentice Hall.

    Google Scholar 

  • Picard, J. C. (1976). Maximal closure of a graph and applications to combinatorial problems. Management Science, 22, 1268–1272.

    Article  Google Scholar 

  • Pinto, Y., & Shamir, R. (1994). Efficient algorithms for minimum-cost flow problems with piecewise-linear convex costs. Algorithmica, 11(3), 256–276.

    Article  Google Scholar 

  • Radzik, T. (1993). Parametric flows, Weighted means of cuts, and fractional combinatorial optimization. In P.M. Pardalos (Ed.), Complexity in numerical optimization (pp. 351–386). Singapore: World Scientific.

    Google Scholar 

  • Renegar, J. (1987). On the worst case arithmetic complexity of approximation zeroes of polynomials. Journal of Complexity, 3, 90–113.

    Article  Google Scholar 

  • Rote, G., & Zachariasen, M. (2007, to appear). Matrix scaling by network flow. In Proceedings of SODA07.

  • Sahni, S. (1974). Computationally related Problems. SIAM Journal on Computing, 3, 262–279.

    Article  Google Scholar 

  • Shub, M., & Smale, S. (1996). Computational complexity: On the geometry of polynomials and a theory of cost, II. SIAM Journal on Computing, 15, 145–161.

    Article  Google Scholar 

  • Sun, J., Tsai, K. -H., & Qi, L. (1993). A simplex method for network programs with convex separable piecewise linear costs and its application to stochastic transshipment problems. In D. Du & P. M. Pardalos (Eds.), Network optimization problems: Algorithms, complexity and applications (pp. 283–300). Singapore: World Scientific.

    Google Scholar 

  • Tamir, A. (1993). A strongly polynomial algorithm for minimum convex separable quadratic cost flow problems on series-parallel networks. Mathematical Programming, 59, 117–132.

    Article  Google Scholar 

  • Tardos, E. (1985). A strongly polynomial minimum cost circulation algorithm. Combinatorica, 5, 247–255.

    Article  Google Scholar 

  • Tardos, E. (1986). A strongly polynomial algorithm to solve combinatorial linear programs. Operations Research, 34, 250–256.

    Google Scholar 

  • Värbrand, P., Tuy, H., Ghannadan, S., & Migdalas, A. (1995). The minimum concave cost network flow problems with fixed number of sources and non-linear arc costs. Journal of Global Optimisation, 6, 135–151.

    Article  Google Scholar 

  • Värbrand, P., Tuy, H., Ghannadan, S., & Migdalas, A. (1996). A strongly polynomial algorithm for a concave production-transportation problem with a fixed number of non-linear variables. Mathematical Programming, 72, 229–258.

    Google Scholar 

Download references

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Dorit S. Hochbaum.

Additional information

An earlier version of this paper appeared in 4OR, 3:3, 171–216, 2005.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Hochbaum, D.S. Complexity and algorithms for nonlinear optimization problems. Ann Oper Res 153, 257–296 (2007). https://doi.org/10.1007/s10479-007-0172-6

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10479-007-0172-6

Keywords

Navigation