Skip to main content

Advertisement

Springer Nature Link
Log in

Limitations of Incremental Dynamic Programming

  • Published:
Algorithmica Aims and scope Submit manuscript

Abstract

We consider so-called “incremental” dynamic programming algorithms, and are interested in the number of subproblems produced by them. The classical dynamic programming algorithm for the Knapsack problem is incremental, produces nK subproblems and nK 2 relations (wires) between the subproblems, where n is the number of items, and K is the knapsack capacity. We show that any incremental algorithm for this problem must produce about nK subproblems, and that about nKlogK wires (relations between subproblems) are necessary. This holds even for the Subset-Sum problem. We also give upper and lower bounds on the number of subproblems needed to approximate the Knapsack problem. Finally, we show that the Maximum Bipartite Matching problem and the Traveling Salesman problem require exponential number of subproblems. The goal of this paper is to leverage ideas and results of boolean circuit complexity for proving lower bounds on dynamic programming.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

Notes

  1. All logarithms in this paper are to the basis 2.

  2. We prefer to use the term “node” instead of “vertex” as well as “wire” instead of “edge” while talking about branching programs, because inputs to programs may also be edges and vertices of a customary graph.

  3. The Min-Plus semiring (R+,⊕,⊗,∞,0) with operations x⊕y=min(x,y) and x⊗y=x+y, is called “tropical” in honor of Imre Simon who lived in Sao Paulo (south tropic). Tropical algebra and tropical geometry are intensively studied topics in mathematics.

References

  1. Aho, A., Hopcroft, J., Ullman, J.: The Design and Analysis of Computer Algorithms. Addison-Wesley, Reading (1974)

    MATH  Google Scholar 

  2. Ajtai, M.: Determinism versus non-determinism for linear time RAMs with memory restrictions. J. Comput. Syst. Sci. 65(1), 2–37 (2002)

    Article  MATH  MathSciNet  Google Scholar 

  3. Alekhnovich, M., Borodin, A., Buresh-Oppenheim, J., Impagliazzo, R., Magen, A.: Toward a model for backtracking and dynamic programming. Comput. Complex. 20(4), 679–740 (2011)

    Article  MATH  MathSciNet  Google Scholar 

  4. Angelopoulos, S., Borodin, A.: The power of prioritized algorithms for facility location and set cover. Algorithmica 40(4), 271–291 (2004)

    Article  MATH  MathSciNet  Google Scholar 

  5. Arora, S., Lund, C., Motwani, R., Sudan, M., Szegedy, M.: Proof verification and the hardness of approximation problems. J. ACM 45(3), 501–555 (1998)

    Article  MATH  MathSciNet  Google Scholar 

  6. Barvinok, A., Gimadi, E.K., Serdyukov, A.I.: The maximum traveling salesman problem. In: Gutin, G., Punnen, A. (eds.) The Traveling Salesman Problem and Its Variations, pp. 585–607. Kluwer, Amsterdam (2002)

    Google Scholar 

  7. Beame, P., Saks, M., Sun, X., Vee, E.: Time-space trade-off lower bounds for randomized computation of decision problems. J. ACM 50(2), 154–195 (2003)

    Article  MathSciNet  Google Scholar 

  8. Bellman, R.: Combinatorial processes and dynamic programming. In: Proc. of the 10-th Symp. in Applied Math of the AMS, pp. 24–26. AMS, Providence (1958)

    Google Scholar 

  9. Bompadre, A.: Exponential lower bounds on the complexity of a class of dynamic programs for combinatorial optimization problems. Algorithmica 62(3–4), 659–700 (2012)

    Article  MATH  MathSciNet  Google Scholar 

  10. Borodin, A., Razborov, A., Smolensky, R.: On lower bounds for read-k times branching programs. Comput. Complex. 3, 1–18 (1993)

    Article  MATH  MathSciNet  Google Scholar 

  11. Borodin, A., Nielsen, M.N., Rackoff, C.: (Incremental) prioritized algorithms. Algorithmica 37(4), 295–326 (2003)

    Article  MATH  MathSciNet  Google Scholar 

  12. Borodin, A., Boyar, J., Larsen, K.S., Mirmohammadi, N.: Priority algorithms for graph optimization problems. Theor. Comput. Sci. 411(1), 239–258 (2010)

    Article  MATH  MathSciNet  Google Scholar 

  13. Buresh-Oppenheim, J., Davis, S., Impagliazzo, R.: A stronger model of dynamic programming algorithms. Algorithmica 60(4), 938–968 (2011)

    Article  MATH  MathSciNet  Google Scholar 

  14. Davis, S., Impagliazzo, R.: Models of greedy algorithms for graph problems. Algorithmica 54(3), 269–317 (2009)

    Article  MATH  MathSciNet  Google Scholar 

  15. Furman, M.E.: Application of a method of fast multiplication of matrices in the problem of finding the transitive closure of a graph. Sov. Math. Dokl. 11(5), 1252 (1970)

    MATH  MathSciNet  Google Scholar 

  16. Garey, M.R., Johnson, D.S.: Strong NP-completeness results: motivation, examples, and implications. J. ACM 25, 499–508 (1978)

    Article  MATH  MathSciNet  Google Scholar 

  17. Hansel, G.: Nombre minimal de contacts de fermeture necessaires pour realiser une function booleenne symetrique de n variables. C. R. Acad. Sci. 258(25), 6037–6040 (1964). In French

    MATH  MathSciNet  Google Scholar 

  18. Held, M., Karp, R.M.: A dynamic programming approach to sequencing problems. SIAM J. Appl. Math. 10, 196–210 (1962)

    Article  MATH  MathSciNet  Google Scholar 

  19. Helman, P.: A common schema for dynamic programming and branch and bound algorithms. J. ACM 36(1), 97–128 (1989)

    Article  MATH  MathSciNet  Google Scholar 

  20. Helman, P., Rosenthal, A.: A comprehensive model of dynamic programming. SIAM J. Algebr. Discrete Methods 6, 319–334 (1985)

    Article  MATH  MathSciNet  Google Scholar 

  21. Jerrum, M., Snir, M.: Some exact complexity results for straight-line computations over semirings. J. ACM 29(3), 874–897 (1982)

    MATH  MathSciNet  Google Scholar 

  22. Jukna, S.: A note on read-k times branching programs. RAIRO Theor. Inform. Appl. 29(1), 75–83 (1995)

    MATH  MathSciNet  Google Scholar 

  23. Jukna, S.: A nondeterministic space-time tradeoff for linear codes. Inf. Process. Lett. 109(5), 286–289 (2009)

    Article  MATH  MathSciNet  Google Scholar 

  24. Jukna, S.: Boolean Function Complexity: Advances and Frontiers. Springer, Berlin (2012)

    Book  Google Scholar 

  25. Jukna, S., Razborov, A.A.: Neither reading few bits twice nor reading illegally helps much. Discrete Appl. Math. 85(3), 223–238 (1998)

    Article  MATH  MathSciNet  Google Scholar 

  26. Karchmer, M., Wigderson, A.: Monotone circuits for connectivity require super-logarithmic depth. SIAM J. Discrete Math. 3(2), 255–265 (1990)

    Article  MATH  MathSciNet  Google Scholar 

  27. Karp, R., Held, M.: Finite state processes and dynamic programming. SIAM J. Appl. Math. 15, 693–718 (1967)

    Article  MATH  MathSciNet  Google Scholar 

  28. Kerr, L.R.: The effect of algebraic structure on the computation complexity of matrix multiplications. PhD Thesis, Cornell Univ., Ithaca, NY (1970)

  29. Kowalik, L., Mucha, M.: 35/44-approximation for asymmetric Maximum TSP with triangle inequality. Algorithmica 59(2), 240–255 (2011)

    Article  MATH  MathSciNet  Google Scholar 

  30. Munro, I.: Efficient determination of the transitive closure of a directed graph. Inf. Process. Lett. 1(2), 56–58 (1971)

    Article  MATH  MathSciNet  Google Scholar 

  31. Nechiporuk, E.I.: On a boolean function. Sov. Math. Dokl. 7(4), 999–1000 (1966)

    MATH  Google Scholar 

  32. Okolnishnikova, E.A.: Lower bounds on the complexity of realization of characteristic functions of binary codes by branching programs. In: Diskretnii Analiz, Novosibirsk, vol. 51, pp. 61–83 (1991). In Russian

    Google Scholar 

  33. Poloczek, M.: Bounds on greedy algorithms for MAX SAT. In: Demetrescu, C., Halldórsson, M.M. (eds.) Proc. of 19th Ann. European Symp. on Algorithms. LNCS, vol. 6942, pp. 37–48. Springer, Berlin (2011)

    Google Scholar 

  34. Regev, O.: Priority algorithms for makespan minimization in the subset model. Inf. Process. Lett. 84(3), 153–157 (2002)

    Article  MATH  MathSciNet  Google Scholar 

  35. Rychkov, K.L.: On the complexity of generalized contact circuits. Diskretn. Anal. Issled. Oper. 16(5), 78–87 (2009). In Russian

    MATH  MathSciNet  Google Scholar 

  36. Wegener, I.: Branching Programs and Binary Decision Diagrams. SIAM, Philadelphia (2000)

    Book  MATH  Google Scholar 

  37. Woeginger, G.J.: When does a dynamic programming formulation guarantee the existence of a fully polynomial time approximation scheme (fptas)? Informs J. Comput. 12(1), 57–74 (2000)

    Article  MATH  MathSciNet  Google Scholar 

Download references

Acknowledgements

I am thankful to Allan Borodin, Joshua Buresh-Oppenheim, Russell Impagliazzo, Gerhard Paseman, and Georg Schnitger for enlightening discussions. My thanks to anonymous referees for stimulating comments; I am especially obligated to one of them for suggesting to use even-odd pairs in the proof of Theorem 5.

Author information

Authors and Affiliations

  1. Faculty of Computer Science and Mathematics, Goethe-University Frankfurt, Robert-Mayer Str. 11-15, Frankfurt am Main, Germany

    Stasys Jukna

  2. Institute of Mathematics and Informatics, Vilnius University, Akademijos 4, Vilnius, Lithuania

    Stasys Jukna

Authors
  1. Stasys Jukna
    View author publications

    You can also search for this author inPubMed Google Scholar

Corresponding author

Correspondence to Stasys Jukna.

Additional information

Research supported by the DFG grant SCHN 503/5-1.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Jukna, S. Limitations of Incremental Dynamic Programming. Algorithmica 69, 461–492 (2014). https://doi.org/10.1007/s00453-013-9747-6

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00453-013-9747-6

Keywords